by CAIS
MMLU is a multiple-choice benchmark covering 57 subjects, from elementary mathematics to law, published by CAIS on Hugging Face. It measures how much knowledge a language model has across many fields.
DatasetFree
Step 1: Open the cais/mmlu dataset page on Hugging Face and read the dataset card.
Step 2: Load a subject configuration with the Hugging Face datasets library.
Step 3: Use the dev split to build few-shot prompts for each subject.
Step 4: Run your model on the test split and compare its chosen letter with the answer label.
Step 5: Report accuracy per subject group and overall.
It covers 57 tasks across the humanities, social sciences, hard sciences and other areas.
It has a question string, a list of four choices and an answer label.
The dataset card states the MIT License.
It holds 5 examples per subject and is meant for few-shot prompting.
GSM8KGSM8K is OpenAI's dataset of about 8.5K grade-school math word problems with step-by-step natural-language solutions. It is for researchers testing and training language models on multi-step arithmetic reasoning.Wikipedia DatasetThe Wikimedia Wikipedia dataset holds cleaned Wikipedia articles with one subset per language, built from the Wikipedia dumps.SQuADSQuAD (Stanford Question Answering Dataset) is a reading comprehension dataset of crowdworker questions on Wikipedia articles, where each answer is a span of text from the passage. It is for researchers training and evaluating extractive question-answering models.