SQuAD (Stanford Question Answering Dataset) is a reading comprehension dataset of crowdworker questions on Wikipedia articles, where each answer is a span of text from the passage. It is for researchers training and evaluating extractive question-answering models.
DatasetFree
Step 1: Install the Hugging Face datasets library with pip install datasets.
Step 2: Load the data with load_dataset("rajpurkar/squad").
Step 3: Feed each context and question to your model and train it to predict the answer span using answer_start.
Step 4: Evaluate on the validation split by comparing predicted spans with the reference answers.
SQuAD 1.1 contains more than 100,000 question-answer pairs on over 500 Wikipedia articles: 87,599 for training and 10,570 for validation.
CC BY-SA 4.0.
Each answer is a span of text taken from the reading passage, with its starting position.
English.
GSM8KGSM8K is OpenAI's dataset of about 8.5K grade-school math word problems with step-by-step natural-language solutions. It is for researchers testing and training language models on multi-step arithmetic reasoning.Wikipedia DatasetThe Wikimedia Wikipedia dataset holds cleaned Wikipedia articles with one subset per language, built from the Wikipedia dumps.