# LMArena (now Arena) explained: how its leaderboard ranks models

> LMArena (now Arena) ranks models from anonymous pairwise human votes using a Bradley-Terry model. What the score means and where it misleads.

Source: https://www.honeyhive.ai/learn/lmarena (updated 2026-11-09)

LMArena, renamed Arena in [January 2026](https://arena.ai/blog/lmarena-is-now-arena/), is a public leaderboard that ranks AI models by human preference. A visitor sends one prompt, sees two anonymous answers, and votes for the better one. The platform fits a Bradley-Terry model to those pairwise votes and sorts models by the result. The score measures which answer people preferred, not whether the answer was correct.

The project started as Chatbot Arena, described in a [March 2024 paper](https://arxiv.org/abs/2403.04132) by Chiang, Zheng, Stoica and colleagues, which reported over 240K votes at the time. It later became LMArena, and in January 2026 the company [dropped the "LM"](https://arena.ai/blog/lmarena-is-now-arena/) and moved to arena.ai, where its [methodology write-up](https://arena.ai/blog/arena-rank) now sits.

## How the ranking is built

1. **Anonymous pairs.** The Chatbot Arena paper [says](https://arxiv.org/html/2403.04132) the platform filters conversations that contain model or company names, and reveals model identities only after the user votes.
2. **Adaptive pairing.** Pairs are not drawn at random. The authors sample a pair in proportion to how much it would shrink the confidence interval, which they report needs fewer samples than random sampling for the same precision.
3. **Bradley-Terry fit.** The [Arena-Rank write-up](https://arena.ai/blog/arena-rank) (December 2025) describes Bradley-Terry as the core model over battles with a winner, plus an extension for contextual features. A reweighting option is meant to treat models with few battles fairly.
4. **Confidence intervals.** The early paper [chose](https://arxiv.org/html/2403.04132) sandwich robust intervals over a pivot bootstrap because they are smaller in large samples. Arena-Rank now [computes](https://arena.ai/blog/arena-rank) intervals in closed form and ships as the open-source `arena-rank` package under Apache 2.0.

### Elo or Bradley-Terry?

Early versions of the interface reported Elo scores, and the paper's authors [explain](https://arxiv.org/html/2403.04132) that they moved to Bradley-Terry coefficients because those suit statistical estimation better. Many articles still call the number an Elo rating.

## Style control

Raters can prefer longer or more heavily formatted answers regardless of substance. The LMSYS team [introduced](https://lmsys.org/blog/2024-08-28-style-control/) style control in August 2024. It adds four features to the Bradley-Terry regression: answer token length and counts of markdown headers, bold text and lists. Each feature is the normalized difference between the two answers.

In their [analysis](https://lmsys.org/blog/2024-08-28-style-control/), length was the dominant factor and markdown effects were second order. With style control, GPT-4o-mini and Grok-2-mini dropped, while Claude 3.5 Sonnet, Claude 3 Opus and Llama-3.1-405B rose. The authors note the analysis is observational, so unobserved confounders such as a real link between length and quality may remain. Arena-Rank [lists](https://arena.ai/blog/arena-rank) style-controlled leaderboards as a use of its feature extension.

## What the votes can and cannot tell you

| Question | What the sources say |
|---|---|
| Do voters match experts? | The paper [reports](https://arxiv.org/html/2403.04132) 72% to 83% crowd-expert agreement, and 79.4% and 89.8% expert-expert agreement. |
| Can providers test privately? | The Leaderboard Illusion [authors](https://arxiv.org/abs/2504.20879) say undisclosed private testing lets providers try several variants and withdraw scores, and name 27 private variants Meta tested before Llama-4. |
| Is data shared evenly? | The same authors estimate Google and OpenAI received about 19.2% and 20.4% of Arena data, against about 29.7% for 83 open-weight models combined. |
| Does Arena data help a model? | They estimate up to 112% relative gain on the Arena distribution from extra Arena data, and warn of overfitting to Arena dynamics. |
| Does it transfer to other tasks? | The RouteLLM authors [found](https://lmsys.org/blog/2024-07-01-routellm/) routers trained only on Chatbot Arena data performed near random on MMLU. |

These are the cited authors' findings, not settled facts. Treat a rank as one signal.

## Using it to choose a model

Use the leaderboard to build a shortlist, then test the shortlist on your own task. Votes come from open-ended prompts typed by visitors, so they say little about your domain, your output format or a tool-using agent. For agent tasks with graded outcomes, see [agent benchmarks](/learn/agent-benchmarks). For how human preference compares with automated scoring, see [LLM-as-a-judge](/learn/llm-as-a-judge) and [human evaluation](/learn/human-evaluation-llm). The wider picture is in [LLM evaluation](/learn/llm-evaluation), and [model routing](/learn/model-routing) covers sending queries to different models.

## Running your own preference test

The same pairwise idea works on your data. HoneyHive supports [human evaluators](https://docs.honeyhive.ai/v2/evaluators/human.md) for review of outputs, and [datasets with experiments](https://docs.honeyhive.ai/v2/datasets/run-experiments.md) to run candidate models over a fixed set of your own prompts.

## FAQ

### What is LMArena?

A public platform where people compare two anonymous model answers to the same prompt and vote for the better one. The votes become a ranked leaderboard. The project began as [Chatbot Arena](https://arxiv.org/abs/2403.04132), described in a March 2024 paper, and LMArena [renamed itself Arena](https://arena.ai/blog/lmarena-is-now-arena/) in January 2026.

### Is the LMArena score an Elo rating?

Not in the original sense. The Chatbot Arena authors [wrote](https://arxiv.org/html/2403.04132) that they switched from Elo to Bradley-Terry coefficients because they are better for statistical estimation.

### What is style control on LMArena?

A version of the ranking that adds answer length and markdown counts as variables in the Bradley-Terry regression, so a model gets less credit for long or heavily formatted answers. The [LMSYS post](https://lmsys.org/blog/2024-08-28-style-control/) introduced it.

### Can a high LMArena rank tell me which model to use in my product?

Only as a starting point. The votes reflect whatever prompts visitors typed, and the Leaderboard Illusion [authors](https://arxiv.org/abs/2504.20879) argue that private testing and uneven sampling bias scores. Run your own evaluation on your own data.

### Do LMArena voters agree with experts?

Mostly. The Chatbot Arena paper [reports](https://arxiv.org/html/2403.04132) agreement of 72% to 83% between crowd users and experts, against 79.4% and 89.8% between pairs of experts.