GAIA is a benchmark of more than 450 questions with unambiguous answers for evaluating LLMs that use tools, search and prompting. It is for teams measuring how well AI assistants handle multi-step tasks.
DatasetFree
Step 1: Sign in to Hugging Face and accept the access conditions on the GAIA page.
Step 2: Download the repository with snapshot_download from huggingface_hub.
Step 3: Load a config such as 2023_level1 with load_dataset from the downloaded directory.
Step 4: Read each Question and open any attachment using its file_path.
Step 5: Compare your system's answers with the Final answer column on the dev set, and check the leaderboard for results.
More than 450 questions, divided into 3 levels of difficulty.
No, not in a crawlable form. To avoid contamination you agree not to reshare it outside a gated or private repository on the Hugging Face Hub.
No. Each level has a public dev set, while the test set keeps its answers and metadata private.
Parquet files that mirror the former JSONL structure, with columns such as task_id, Question, Level, Final answer, file_name and file_path.
HH-RLHFHH-RLHF is Anthropic's dataset of human preference comparisons on helpfulness and harmlessness, plus red teaming transcripts. It is meant for research on making models less harmful.