by Wikimedia
The Wikimedia Wikipedia dataset holds cleaned Wikipedia articles with one subset per language, built from the Wikipedia dumps.
DatasetFree
Step 1: Pick a language and dump date from the dataset page.
Step 2: Install the Hugging Face datasets library.
Step 3: Load it with load_dataset("wikimedia/wikipedia", "20231101.en").
Step 4: Read the text and title fields from the train split.
Step 5: Check the license terms before redistributing the text.
The card states GFDL and CC BY-SA 3.0. Some text may be available only under the Creative Commons license.
An article id, its url, its title and its cleaned text.
Each language is a separate configuration named by dump date and language code, for example 20231101.en.
The card says it is generally used for language modeling.
GSM8KGSM8K is OpenAI's dataset of about 8.5K grade-school math word problems with step-by-step natural-language solutions. It is for researchers testing and training language models on multi-step arithmetic reasoning.MMLUMMLU is a multiple-choice benchmark covering 57 subjects, from elementary mathematics to law, published by CAIS on Hugging Face. It measures how much knowledge a language model has across many fields.SQuADSQuAD (Stanford Question Answering Dataset) is a reading comprehension dataset of crowdworker questions on Wikipedia articles, where each answer is a span of text from the passage. It is for researchers training and evaluating extractive question-answering models.