Ploba logo

Find the right AI tool for the job: apps, agents, APIs, MCP servers and datasets, explained plainly.

Get the AI Digest

Discover

  • Explore
  • Top Charts
  • Categories
  • Agents
  • MCP servers
  • Developer tools

Learn & community

  • Learn AI
  • AI Glossary
  • Blog
  • Hackathons
  • People
  • AI Digest

For makers

  • List your tool
  • Docs
  • Claim a listing
  • Advertise
  • Write for us

Company

  • About
  • Careers
  • Changelog
  • Trust
  • Security
  • Contact

© 2026 Ploba. All rights reserved.

  • Privacy
  • Terms
  • Cookies
  • Advertising
  • All policies
llms.txt
Ploba logo
DiscoverHackathonsPeopleLearnAI Digest
Sign inGet started
HomeTop ChartsExploreCategories
TypesWeb apps1,894Agents585Plugins416APIs376Mobile apps152Datasets85
AdvertisingYour product hereReach people browsing AI tools on Ploba.Advertise hereAdvertisingYour product hereReach people browsing AI tools on Ploba.Advertise here
  1. Explore
  2. /
  3. Data
  4. /
  5. MMLU
Data
MM

MMLU

by CAIS

MMLU is a multiple-choice benchmark covering 57 subjects, from elementary mathematics to law, published by CAIS on Hugging Face. It measures how much knowledge a language model has across many fields.

Dataset·Free

View on Hugging Face(opens in a new tab)

About MMLU

Key features of MMLU

  • 57 subject tasks from elementary mathematics to law
  • Four-choice multiple-choice questions
  • Test split of 14,042 questions
  • Dev split with 5 examples per subject for few-shot prompts
  • Validation split of 1,531 questions
  • Auxiliary training questions drawn from several reading and science sets

Who MMLU is for

  • Researchers evaluating language models
  • Teams comparing models on broad knowledge
  • Developers building evaluation harnesses

How to use MMLU

  1. 1

    Step 1: Open the cais/mmlu dataset page on Hugging Face and read the dataset card.

  2. 2

    Step 2: Load a subject configuration with the Hugging Face datasets library.

  3. 3

    Step 3: Use the dev split to build few-shot prompts for each subject.

  4. 4

    Step 4: Run your model on the test split and compare its chosen letter with the answer label.

  5. 5

    Step 5: Report accuracy per subject group and overall.

MMLU FAQ

How many subjects does MMLU cover?

It covers 57 tasks across the humanities, social sciences, hard sciences and other areas.

What does a record look like?

It has a question string, a list of four choices and an answer label.

What is the license?

The dataset card states the MIT License.

What is the dev split for?

It holds 5 examples per subject and is meant for few-shot prompting.

Details

Pricing
FreeFree. The card lists the MIT License.
Type
Dataset
Categories
DataEducationResearch
Added
Sep 17, 2026
CAPublisherCAIS

Built from public sources

We put this page together from MMLU’s own website and docs so you can compare it with other tools. Ploba isn’t affiliated with MMLU.

Work at MMLU? Claim the listing to manage it yourself.

Something wrong here?

More from CAIS

View publisher
HLHumanity's Last ExamCAIS · FreeHumanity's Last Exam (HLE) is a benchmark of 2,500 expert-written questions across dozens of subjects, built to test AI models at the frontier of human knowledge. It is for researchers and model builders evaluating frontier AI.AI · Free

Related tools

See more
GSM8K iconGSM8KOpenAI · FreeGSM8K is OpenAI's dataset of about 8.5K grade-school math word problems with step-by-step natural-language solutions. It is for researchers testing and training language models on multi-step arithmetic reasoning.Data · FreeWDWikipedia DatasetWikimedia · FreeThe Wikimedia Wikipedia dataset holds cleaned Wikipedia articles with one subset per language, built from the Wikipedia dumps.Data · FreeSQSQuADPranav Rajpurkar · FreeSQuAD (Stanford Question Answering Dataset) is a reading comprehension dataset of crowdworker questions on Wikipedia articles, where each answer is a span of text from the passage. It is for researchers training and evaluating extractive question-answering models.Data · Free