# What is synthetic data?

> Synthetic data is artificial data made by an algorithm to mimic real data.

Source: https://www.honeyhive.ai/learn/what-is-synthetic-data (updated 2026-11-02)

Synthetic data is data created by an algorithm instead of collected from real events. AWS defines it as ["data generated by an algorithm rather than observed from real-world measurements"](https://aws.amazon.com/what-is/synthetic-data/). IBM calls it ["artificial data designed to mimic real-world data"](https://www.ibm.com/think/topics/synthetic-data), made with statistical methods or AI. In AI work the generator is often a language model, and the data is text: instructions, questions, conversations or test cases; [how to generate synthetic data with LLMs](/learn/synthetic-data-generation) covers the methods.

## Types of synthetic data

[IBM](https://www.ibm.com/think/topics/synthetic-data) groups it two ways.

| By | Kinds |
|---|---|
| Form | Tabular, text, multimedia |
| How much real data is used | Fully synthetic (no real information), partially synthetic (sensitive values swapped for artificial ones), hybrid (real records mixed with fully synthetic ones) |

## How it is made

AWS lists [three families of methods](https://aws.amazon.com/what-is/synthetic-data/):

- **Statistical distributions.** Find the distributions in real data, then sample new data from them.
- **Model-based.** Train a machine learning model on real data, then generate data that follows the same distribution.
- **Deep learning.** GANs and VAEs, which AWS says work well for images and time series.

For language models, a common pattern is to have a model write the data. [Self-Instruct](https://arxiv.org/abs/2212.10560) (Wang et al., 2022, ACL 2023) had a pretrained model generate instructions, inputs and outputs, filtered them, and finetuned on the result. On GPT-3 it gave a "33% absolute improvement over the original model on Super-NaturalInstructions", and the authors called it an "almost annotation-free method".

## Why teams use it

- **Scarce or costly data.** Liu et al. say model quality depends on large, diverse, high-quality data that is [hard to get because of scarcity, privacy and cost](https://arxiv.org/abs/2404.07503).
- **Privacy.** AWS says synthetic records can stand in for personal data in fields like healthcare and finance [without exposing sensitive data](https://aws.amazon.com/what-is/synthetic-data/).
- **Labels and edge cases.** IBM notes it [comes pre-labeled and can fill gaps with rare cases](https://www.ibm.com/think/topics/synthetic-data).
- **Balance.** AWS says it can [rebalance skewed data](https://aws.amazon.com/what-is/synthetic-data/).

## Where it goes wrong

- **Model collapse.** Shumailov et al. (2023) found that [use of model-generated content in training "causes irreversible defects"](https://arxiv.org/abs/2305.17493), as "tails of the original content distribution disappear". IBM says it happens when [a model is repeatedly trained on AI-generated data](https://www.ibm.com/think/topics/synthetic-data) and that mixing in real data helps.
- **Inherited bias.** IBM says synthetic data [can carry the biases of its source data](https://www.ibm.com/think/topics/synthetic-data).
- **Missing outliers.** AWS says real data has [outliers and anomalies that generators struggle to reproduce](https://aws.amazon.com/what-is/synthetic-data/).
- **Privacy against accuracy.** AWS says [stronger privacy may cost accuracy](https://aws.amazon.com/what-is/synthetic-data/).
- **Unchecked quality.** Liu et al. say it should be [checked for factuality, fidelity and lack of bias](https://arxiv.org/abs/2404.07503). IBM says [extra checks and tests must be run](https://www.ibm.com/think/topics/synthetic-data) after generation.

## Synthetic data for evaluation

Evaluation needs test cases with inputs and, often, expected outputs, and real ones can be thin before launch. Synthetic cases can add coverage, such as rare inputs or hard cases. The cautions above still apply: generated data tends to lose the tails of the real distribution and miss real outliers, so check synthetic cases against real traffic and review samples by hand. Anthropic's guidance on grading also applies: its eval examples note it is [best practice to grade with a different model from the one that produced the output](https://platform.claude.com/docs/en/test-and-evaluate/develop-tests). That line is about the grader, but the same logic argues for not letting one model both write and answer its own test set.

Synthetic cases support [LLM evaluation](/learn/llm-evaluation) and probing for failures in [AI red teaming](/learn/ai-red-teaming). They are not a replacement for cases taken from real use.

## Using synthetic cases in HoneyHive

HoneyHive's [datasets documentation](https://docs.honeyhive.ai/v2/datasets/introduction.md) lists five ways to create a dataset: from production traces, a JSON, JSONL or CSV upload in the UI, an upload with the Python SDK, YAML files applied with the CLI, or in code. It does not describe a built-in generator, so generate synthetic cases elsewhere, then upload them. Keep them in a separate dataset from cases taken from traces so you can compare results on each, and run [experiments](https://docs.honeyhive.ai/v2/evaluation/introduction.md) against both.

## FAQ

### What is synthetic data in simple terms?

Data produced by an algorithm instead of collected from real events. AWS defines it as ["data generated by an algorithm rather than observed from real-world measurements"](https://aws.amazon.com/what-is/synthetic-data/), and IBM as ["artificial data designed to mimic real-world data"](https://www.ibm.com/think/topics/synthetic-data).

### What are the types of synthetic data?

IBM splits it by form (tabular, text, multimedia) and by how much real data is involved, which gives [fully synthetic, partially synthetic and hybrid data](https://www.ibm.com/think/topics/synthetic-data).

### Can you train a model only on synthetic data?

Not safely without checks. Shumailov et al. found that [training on model-generated content causes "irreversible defects"](https://arxiv.org/abs/2305.17493) called model collapse, and IBM says [mixing in real data helps](https://www.ibm.com/think/topics/synthetic-data).

### Is synthetic data private?

It can protect personal data, but AWS notes that making sure [no one can trace points back to real information may require a reduction in accuracy](https://aws.amazon.com/what-is/synthetic-data/).