by OpenBMB
UltraX Preview is a collection of five English pre-training corpora refined by UltraX, a framework that edits web text with structured function calls. Each corpus has about 20B tokens.
DatasetFree
Step 1: Open the dataset page on Hugging Face and pick one of the five corpora.
Step 2: Load it with load_dataset, passing the corpus name such as UltraX-FineWeb and split train.
Step 3: Read cleaned_content for the refined text and raw_content for the original.
Step 4: Inspect processed_functions to see which edits were applied.
Step 5: Check the license of the source corpus before using the data.
Refined versions of FineWeb, RedPajama-v2, AICC, Ultra-FineWeb and FineWeb-ProX-Doc, each with about 20B tokens.
Apache 2.0, but you should also check each source dataset's license. Redistributing or mirroring the artifacts without written permission is prohibited.
uid, raw_content, cleaned_content, processed_functions and source.
HH-RLHFHH-RLHF is Anthropic's dataset of human preference comparisons on helpfulness and harmlessness, plus red teaming transcripts. It is meant for research on making models less harmful.
MONETMONET is a dataset of 103.8 million English image-text pairs for training text-to-image models. It was distilled from 2.9 billion raw pairs through safety, domain and duplicate filtering.
GDPvalGDPval is OpenAI's dataset of 220 real-world knowledge tasks across 44 occupations. It supports evaluating AI models on economically valuable work.