# What is GDPval?

> GDPval is OpenAI's benchmark of real work tasks across 44 occupations, graded by experts against human deliverables. How it is built, scored and what it misses.

Source: https://www.honeyhive.ai/learn/gdpval (updated 2026-11-13)

GDPval is a benchmark from OpenAI that tests whether AI models can produce the work products of professionals. OpenAI's [announcement](https://openai.com/index/gdpval/) describes tasks drawn from the real work of professionals in 44 occupations, judged by experts from the same occupations. The [paper](https://arxiv.org/abs/2510.04374), by Tejal Patwardhan and colleagues, was submitted in October 2025. Unlike a multiple-choice test such as [MMLU](/learn/mmlu), a GDPval task asks for a deliverable like a document, spreadsheet or slide deck.

## What GDPval contains

The [paper](https://arxiv.org/html/2510.04374) covers nine sectors, each contributing more than 5% of U.S. GDP: real estate and rental and leasing, manufacturing, professional, scientific and technical services, government, health care and social assistance, finance and insurance, retail trade, wholesale trade, and information. Across them it picks 44 occupations.

| Set | Tasks | Notes |
|---|---|---|
| Full set | [1,320](https://arxiv.org/html/2510.04374) | 30 per occupation |
| Gold subset | [220](https://huggingface.co/datasets/openai/gdpval) | 5 per occupation, open-sourced |

Each task has a request, often with reference files, and an expected deliverable. The Hugging Face [dataset card](https://huggingface.co/datasets/openai/gdpval) lists a prompt, reference files, deliverable files and a rubric for every task.

## How the tasks were made

Industry experts with an [average of 14 years](https://arxiv.org/abs/2510.04374) of experience wrote each task from their own work. The [paper](https://arxiv.org/html/2510.04374) says they mapped requests to O*NET occupational tasks and estimated completion time, so each task gets a dollar value: its estimated time multiplied by the occupation's median hourly wage. Every task went through automated model screening and human review, averaging five human reviews and never fewer than three.

The paper [reports](https://arxiv.org/html/2510.04374) that 67.7% of gold tasks need at least one reference file, and that the human baseline on the gold subset took about 404 minutes and $361 per task.

## How GDPval is graded

The main method is a blinded pairwise comparison. Occupational experts [rank](https://arxiv.org/html/2510.04374) an unlabeled human deliverable against an unlabeled model deliverable and pick the better one. Results come as win rates, often reported as wins plus ties. Each gold-subset comparison [took](https://arxiv.org/html/2510.04374) over an hour on average, which is why grading is the expensive part.

In the paper, Claude Opus 4.1 [scored](https://arxiv.org/html/2510.04374) 47.6% wins plus ties on the gold subset, the best result in that study. OpenAI's [announcement](https://openai.com/index/gdpval/) says performance on GDPval tasks "more than tripled in a year" from GPT-4o to GPT-5. Those numbers are from the autumn 2025 release, so look at the current results before quoting any.

To cut the cost, the authors also built an experimental automated grader. It [agreed](https://arxiv.org/html/2510.04374) with human experts 65.7% of the time, against 70.8% agreement between two human raters. That is the same trade-off covered in [LLM-as-a-judge](/learn/llm-as-a-judge) and [LLM judge limitations](/learn/llm-judge-limitations): cheap, close to a person, and biased in ways that need checking. The paper also [notes](https://arxiv.org/html/2510.04374) that the grader, built on GPT-5-high, agrees less with human experts on outputs from capable OpenAI models.

## GDPval-AA

[Artificial Analysis](/learn/artificial-analysis-intelligence-index) publishes a variant called GDPval-AA, at version 2.1 as of November 2026. Its [methodology page](https://artificialanalysis.ai/evaluations/gdpval-aa) says it uses the 220 gold tasks, gives models shell access and web browsing in an agentic loop through its Stirrup harness, and has an LLM judge pick the winner of anonymized head-to-head comparisons. Those results are aggregated into an Elo rating, so a score is a rating against other models, not a win rate against human experts. The two numbers are not interchangeable. See [agent benchmarks](/learn/agent-benchmarks) for other tests that run models as agents.

## What GDPval does not tell you

The authors [list](https://arxiv.org/html/2510.04374) several limits:

- **Small.** It has 44 occupations and 30 tasks per occupation, and the authors say it is not a comprehensive evaluation.
- **Narrow kind of work.** Tasks are self-contained and computer-based. Manual or physical work, heavy tacit knowledge, personal data, proprietary tools and interpersonal communication are left out.
- **One shot.** Tasks are precisely specified and not interactive, and performance dropped on deliberately under-specified prompts.
- **Grading is a proxy.** The automated grader is less reliable than expert graders.

A strong GDPval result also does not show a model will do well on your documents, templates and review standards. For that, build a [golden dataset](/learn/golden-dataset) from your own cases and grade with a [rubric](/learn/llm-evaluation-rubric), adding [human evaluation](/learn/human-evaluation-llm) where judgment matters. The broader picture is in [LLM evaluation](/learn/llm-evaluation).

## Running a GDPval-style test on your own tasks

The gold subset is public, so you can load it as a dataset. In HoneyHive you can [import a dataset](https://docs.honeyhive.ai/v2/datasets/import.md) and compare runs as described in the [experiments guide](https://docs.honeyhive.ai/v2/evaluation/introduction.md).

## FAQ

### Where does the name GDPval come from?

The [paper](https://arxiv.org/abs/2510.04374) is titled "GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks" and draws its tasks from the top nine sectors contributing to U.S. GDP.

### How many tasks are in GDPval?

The [paper](https://arxiv.org/html/2510.04374) describes a full set of 1,320 tasks, 30 for each of 44 occupations, and an open-sourced gold subset of 220 tasks, 5 per occupation.

### How is GDPval scored?

Occupational experts [compare](https://arxiv.org/html/2510.04374) unlabeled model and human deliverables in blinded pairs and pick the better one. OpenAI also offers an experimental automated grader.

### What is GDPval-AA?

It is Artificial Analysis's [version](https://artificialanalysis.ai/evaluations/gdpval-aa) of the benchmark. It runs models as agents with shell and web access on the 220 gold tasks and ranks them with Elo ratings from LLM-judged pairwise comparisons.
