What is DSPy? How it works and how to evaluate it

DSPy is an open-source Python framework that started at Stanford NLP for building LLM programs without hand-writing prompts. Its PyPI page describes it as “programming, rather than prompting, language models”. You declare what a step takes in and puts out, pick a module that decides how the model does it, and let an optimizer tune the prompts or model weights against a metric you define (docs).

The parts

The DSPy docs describe a conventional prompt as four things tangled together, and DSPy pulls them apart:

Because the pieces are separate, the docs say you can swap the LM or its adapter without changing the rest of your logic, or exchange dspy.ChainOfThought for another module without touching the signature. The DSPy homepage shows ReAct for tool use; for the pattern itself see ReAct agents.

The loop: program, evaluate, optimize

DSPy’s learning guide names three stages and suggests learning them in order: programming, evaluation, then optimization. It says launching optimization runs “using a poorly designed program or a bad metric” is unproductive.

1. Program. Define inputs and outputs, then start simple, perhaps with a single dspy.ChainOfThought module, and add steps based on what you see.
2. Evaluate. Collect a development set. The evaluation guide says even 20 examples can be useful and 200 goes a long way. Then define a metric, a function that takes an example and your system’s output and returns a score. For long-form output the guide says the metric will be a smaller DSPy program that checks several properties, and warns that it is “hard to consistently improve what you aren’t able to define”. Run the pipeline, read the outputs and scores, and keep the result as your baseline.
3. Optimize. Optimizers such as MIPROv2, GEPA and BootstrapFewShot tune the prompts or weights against the metric. Which one fits, what it costs and where it fails is covered in prompt optimization.

The metric is the part DSPy cannot write for you. For the general method of building one, see LLM evaluation.

What to check before you rely on it

• Version. HoneyHive’s integration guide was last tested with DSPy 3.1.x (April 2026) and requires 3.x. The latest release on PyPI was 3.4.1 as of November 2026. Tutorials for 2.x may not run unchanged.
• Held-out data. An optimizer improves the score on the data it sees. Keep examples it never touched to confirm that the gain holds.
• Visibility. A compiled program hides its final prompts and intermediate calls behind modules. Trace a run to see what each module sent to the model and what the optimizer changed. For what to watch in agent runs, see agent observability.

Tracing DSPy in HoneyHive

HoneyHive’s integration page uses OpenInference instrumentors for DSPy and OpenAI, attached to the HoneyHive tracer. Install honeyhive, dspy, openinference-instrumentation-dspy and openinference-instrumentation-openai, set HH_INGESTION_API_KEY, and instrument before any module runs:

tracer = HoneyHiveTracer.init()  # reads HH_INGESTION_API_KEY
DSPyInstrumentor().instrument(tracer_provider=tracer.provider)
OpenAIInstrumentor().instrument(tracer_provider=tracer.provider)

Existing DSPy code then runs unchanged. The guide lists what is captured: each module forward() call with inputs and outputs, LLM requests and responses with token usage, tool arguments and results for ReAct agents, and parent and child spans when a module calls sub-modules. The @trace decorator wraps your own logic in a parent span, and enrich_span() adds metadata. In long-running processes, call force_flush() and uninstrument() on shutdown to avoid duplicate spans.

A coding agent can do the setup with HoneyHive’s installable skills (AI coding agents): ask it to use honeyhive-instrument to add tracing to the DSPy app. Full setup is on the HoneyHive DSPy integration page.

FAQ

What does DSPy stand for?

Declarative Self-improving Python, as its PyPI listing says.

Is DSPy free to use?

Yes. It is MIT licensed and installs with pip install -U dspy on Python 3.10 or later. You pay your model provider for tokens, and optimizer runs make many model calls.

Does DSPy replace prompt engineering?

It moves the work. DSPy separates the task definition from the formatting and the trial and error of wording prompts, and an optimizer searches the wording. You still have to define the task and a metric. See prompt optimization.

Do older DSPy tutorials work with DSPy 3?

Not always. HoneyHive's integration guide requires DSPy 3.x and notes that the API changed significantly from 2.x, so check the version a tutorial targets.