by OpenBMB
UltraData-Math is an OpenBMB math pre-training dataset of over 290B tokens in three quality tiers, built to improve mathematical reasoning in language models.
DatasetFree
Step 1: Open the dataset page on Hugging Face and review the tier table.
Step 2: Pick a tier: L1 for scale, L2 for quality-selected text, L3 for structured reasoning formats.
Step 3: Load the chosen tier with the Hugging Face datasets library.
Step 4: Mix it into your pre-training data and compare results on math benchmarks.
Step 5: Read the linked paper for the tiered data management framework.
L1 is a filtered web math corpus, L2 is a quality-selected subset of it, and L3 is rewritten or synthetic data in formats like Q&A, dialogue and textbooks.
More than 290B tokens in total: 170.5B in L1, 33.7B in L2 and 88B in L3.
The card says the filtering aims for mainly English and Chinese mathematical content.
Yes. The card says it has been applied to mathematical pre-training of the MiniCPM series.