What is GDPval?
GDPval is OpenAI's benchmark of real work tasks across 44 occupations, graded by experts against human deliverables. How it is built, scored and what it misses.
GDPval is a benchmark from OpenAI that tests whether AI models can produce the work products of professionals. OpenAI’s announcement describes tasks drawn from the real work of professionals in 44 occupations, judged by experts from the same occupations. The paper, by Tejal Patwardhan and colleagues, was submitted in October 2025. Unlike a multiple-choice test such as MMLU, a GDPval task asks for a deliverable like a document, spreadsheet or slide deck.
What GDPval contains
The paper covers nine sectors, each contributing more than 5% of U.S. GDP: real estate and rental and leasing, manufacturing, professional, scientific and technical services, government, health care and social assistance, finance and insurance, retail trade, wholesale trade, and information. Across them it picks 44 occupations.
| Set | Tasks | Notes |
|---|---|---|
| Full set | 1,320 | 30 per occupation |
| Gold subset | 220 | 5 per occupation, open-sourced |
Each task has a request, often with reference files, and an expected deliverable. The Hugging Face dataset card lists a prompt, reference files, deliverable files and a rubric for every task.
How the tasks were made
Industry experts with an average of 14 years of experience wrote each task from their own work. The paper says they mapped requests to O*NET occupational tasks and estimated completion time, so each task gets a dollar value: its estimated time multiplied by the occupation’s median hourly wage. Every task went through automated model screening and human review, averaging five human reviews and never fewer than three.
The paper reports that 67.7% of gold tasks need at least one reference file, and that the human baseline on the gold subset took about 404 minutes and $361 per task.
How GDPval is graded
The main method is a blinded pairwise comparison. Occupational experts rank an unlabeled human deliverable against an unlabeled model deliverable and pick the better one. Results come as win rates, often reported as wins plus ties. Each gold-subset comparison took over an hour on average, which is why grading is the expensive part.
In the paper, Claude Opus 4.1 scored 47.6% wins plus ties on the gold subset, the best result in that study. OpenAI’s announcement says performance on GDPval tasks “more than tripled in a year” from GPT-4o to GPT-5. Those numbers are from the autumn 2025 release, so look at the current results before quoting any.
To cut the cost, the authors also built an experimental automated grader. It agreed with human experts 65.7% of the time, against 70.8% agreement between two human raters. That is the same trade-off covered in LLM-as-a-judge and LLM judge limitations: cheap, close to a person, and biased in ways that need checking. The paper also notes that the grader, built on GPT-5-high, agrees less with human experts on outputs from capable OpenAI models.
GDPval-AA
Artificial Analysis publishes a variant called GDPval-AA, at version 2.1 as of November 2026. Its methodology page says it uses the 220 gold tasks, gives models shell access and web browsing in an agentic loop through its Stirrup harness, and has an LLM judge pick the winner of anonymized head-to-head comparisons. Those results are aggregated into an Elo rating, so a score is a rating against other models, not a win rate against human experts. The two numbers are not interchangeable. See agent benchmarks for other tests that run models as agents.
What GDPval does not tell you
The authors list several limits:
- Small. It has 44 occupations and 30 tasks per occupation, and the authors say it is not a comprehensive evaluation.
- Narrow kind of work. Tasks are self-contained and computer-based. Manual or physical work, heavy tacit knowledge, personal data, proprietary tools and interpersonal communication are left out.
- One shot. Tasks are precisely specified and not interactive, and performance dropped on deliberately under-specified prompts.
- Grading is a proxy. The automated grader is less reliable than expert graders.
A strong GDPval result also does not show a model will do well on your documents, templates and review standards. For that, build a golden dataset from your own cases and grade with a rubric, adding human evaluation where judgment matters. The broader picture is in LLM evaluation.
Running a GDPval-style test on your own tasks
The gold subset is public, so you can load it as a dataset. In HoneyHive you can import a dataset and compare runs as described in the experiments guide.
FAQ
Where does the name GDPval come from?
The paper is titled "GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks" and draws its tasks from the top nine sectors contributing to U.S. GDP.
How many tasks are in GDPval?
The paper describes a full set of 1,320 tasks, 30 for each of 44 occupations, and an open-sourced gold subset of 220 tasks, 5 per occupation.
How is GDPval scored?
Occupational experts compare unlabeled model and human deliverables in blinded pairs and pick the better one. OpenAI also offers an experimental automated grader.
What is GDPval-AA?
It is Artificial Analysis's version of the benchmark. It runs models as agents with shell and web access on the 220 gold tasks and ranks them with Elo ratings from LLM-judged pairwise comparisons.