by Mercor
APEX-Agents is Mercor's benchmark of 480 tasks for testing whether AI agents can do long-horizon professional services work across applications. Tasks were written by investment banking analysts, management consultants and corporate lawyers.
DatasetFree
Step 1: Sign in to Hugging Face and accept the access conditions on the dataset page.
Step 2: Load the data with load_dataset("mercor/apex-agents").
Step 3: Set up the open-source execution service linked from the card to run your agent in each world.
Step 4: Give the agent each task prompt and collect its output.
Step 5: Grade the output against the rubric criteria with a judge model and compare against the leaderboard.
No. The card says it is for model evaluation only and forbids training, fine-tuning or parameter fitting.
A judge model grades each binary rubric criterion as met or not met, using the prompt, the agent output and relevant artifacts.
CC-BY 4.0, with the evaluation-only restriction stated on the card.
Investment banking, management consulting and corporate law, with 160 tasks each.
HH-RLHFHH-RLHF is Anthropic's dataset of human preference comparisons on helpfulness and harmlessness, plus red teaming transcripts. It is meant for research on making models less harmful.