Introducing HoneyHive Skills

A few months ago, we published the Agent Development Lifecycle — a six-stage process distilled from working with hundreds of teams shipping agents. As agents are increasingly built and maintained using coding agents such as Claude Code and Cursor, it is essential to provide agent development knowledge in a way that is ergonomic for coding agents.

Today, we’re releasing 5 skills that capture key agent development workflows and supply them to any coding agent — Claude Code, Cursor, Codex, Copilot, and Gemini CLI. These make it easier than ever to build, monitor, and evaluate your agents.

‍

Here’s an overview of the new skills:

1. honeyhive-cli — Install and use the HoneyHive CLI, with schema introspection so agents work from real commands instead of guessed flags; the shared foundation for every other skill.
2. honeyhive-instrument — Wire HoneyHive tracing into your LLM, agent, or RAG application with minimal edits — SDK install, OTEL setup, and the right framework instrumentor, verified against a trace-quality rubric.
3. honeyhive-evaluate — Stand up the smallest useful experiment: datasets, evaluators, runs, and baseline comparisons that plug straight into regression tracking.
4. honeyhive-improve — Debug failing agent workflows from trace evidence — reconstruct the failure, find the first plausible cause, and ship the minimal fix.
5. honeyhive-alert-root-cause — Turn a HoneyHive Discover alert URL into a root-cause investigation: decode the filters, pull flagged sessions, separate true from false positives, and recommend guardrails.

‍

Installation

npx skills add honeyhiveai/skills --skill '*'

‍

The repo is open source at github.com/honeyhiveai/skills.

What are Agent Skills?

Agent Skills is an open standard, originally developed by Anthropic, for packaging procedural knowledge into folders an agent loads on demand. Each skill is a directory with a SKILL.md — YAML frontmatter that tells the agent when the skill is relevant, plus markdown instructions for how to execute — along with optional reference docs and scripts. Skills use progressive disclosure: only the name and description sit in context by default; the full instructions load when your prompt matches. The result is an agent that carries a lot of expertise with a small context footprint. If you've used skills from Sentry, Datadog, or Langfuse, the install flow here will feel familiar.

The suite at a glance

The four skills map onto the lifecycle of an AI application, and they hand off to each other:

The architecture follows a pattern we think is the right one for vendor skills: the skill is the procedural knowledge, the CLI is the deterministic transport. Agents are great at judgment — reading your codebase, picking the right instrumentor, deciding what to evaluate. They're unreliable at remembering exact API payloads. So every skill shells out to the honeyhive CLI, which exposes --show-file-schema and --show-argument-schema so the agent introspects real schemas instead of guessing flags from training data. No hallucinated payloads, no stale API knowledge.

One more thing the foundation gives you: a doc-gap protocol. When a skill hits a framework or CLI operation we don't cover yet, it doesn't improvise a brittle workaround and move on — it tells you, and files a GitHub issue with the framework, versions, and what it did instead. The skills report their own blind spots, which means they get better with every install base mile.

‍

honeyhive instrument — wire tracing in, then grade the trace

The job: add HoneyHive tracing to an LLM app, agent, or RAG pipeline with the smallest possible edit set, and don't stop until the resulting trace is actually useful.

The skill opens with its own rule of engagement: "You are an instrumentation surgeon, not a rewriter." It runs three phases, and each phase has a checkpoint — a concrete artifact proving the phase ran:

Two design decisions are worth calling out.

‍

Validation gates everything. Before touching a line of code, the skill curls the healthcheck and runs a read-only scan of your repo: language, framework and pinned version, instrumentor family, existing HoneyHive wiring, existing OTel providers from other vendors. A wrong HH_API_URL or rejected key means every later step builds on sand, so the skill refuses to proceed past a failed check — with the exact status code and recovery step instead of a vague retry loop.

‍

Verification grades quality, not just plumbing. Most setup automation declares victory when a span arrives. This skill then loads a qualitative rubric (success.md) and grades the trace: are event names meaningful? Are LLM calls easy to find rather than buried five levels deep? Do tool calls link cleanly to the agent that invoked them? Are failures surfaced instead of hidden in metadata? Is there a final event with enough context to hang end-to-end evaluators on? The skill's own words: "a mechanically-correct integration that produces a messy trace is not done." If the trace fails the rubric, the agent circles back and adjusts session boundaries or adds manual spans — auto-instrumentation is the starting point, not the ceiling.

‍

Where you'd use it:

• "Add HoneyHive tracing to my LangGraph agent." The skill detects Python, picks the matching instrumentor family, pulls the framework's integration doc, and applies the minimal wiring.
• Monorepo that already ships traces to another vendor. Runtime validation detects the existing TracerProvider, loads the co-existence reference, and shares the provider instead of hijacking it.
• Spans arrive but the trace is unreadable. The rubric fails it; the agent adds manual spans around retrieval and tool calls until the trace is debuggable.

‍

The guardrails: the skill carries a list of hard MUSTs, each tied to a named evaluator we run against the skill itself. It will not hardcode your API key, will not register a global tracer provider, will not construct duplicate tracers, will not bump your framework versions to make its life easier, and will not keep iterating after success criteria are met. For anyone who has watched an agent "fix" instrumentation by upgrading half the lockfile, this is the section to read twice.

‍

honeyhive-evaluate — the smallest useful experiment

The job: set up a HoneyHive experiment that answers a real regression question — dataset, evaluators, run, comparison — without flooding your project with resources you didn't approve.

This skill encodes evaluation philosophy, not just API calls. The evaluator guidance baked into it is the advice we give every team in onboarding:

• Start with one or two high-impact evaluators, not a long list. Metric sprawl makes every event harder to read.
• Prefer narrow checks over holistic scores. "Did the answer match ground truth?" beats "was this response good overall?"
• Use code checks before LLM judges for anything objective — schema conformance, required fields, format rules, thresholds.
• Prefer binary pass/fail over 1–5 grades. Binary decisions are easier to calibrate and compare across runs.

And the validation phase has a specific bar: it must prove the experiment is usable for regression tracking — resources visible, results retrievable, comparison against a baseline working programmatically — not just that a script exited zero.

Where you'd use it:

• "Take these 10 failing examples and turn them into a dataset with an exact-match evaluator." Seed dataset, compact evaluator, first run — small enough to review by hand.
• "Compare this run against the baseline before I merge the prompt change." The skill reuses existing runs and wires up the comparison instead of inventing new resources.
• A TypeScript codebase. The skill drives everything through the CLI and the TS SDK rather than pretending Python-only helpers exist — a failure mode you've probably seen from agents working off generic training data.

The guardrails: read-only discovery first, an explicit plan you approve before anything mutates, no hidden dataset or evaluator creation, no version upgrades by default, and never writing secrets to source.

‍

honeyhive-improve — evidence first, minimal fix, then stop

The job: set up a HoneyHive experiment that answers a real regression question — dataset, evaluators, run, comparison — without flooding your project with resources you didn't approve.

The discipline here is epistemic. The skill instructs the agent to write observations before explanations — "tool call returned no documents before the bad answer" is admissible; "the model was confused" is not. Every investigation separates three things that engineers (and agents) love to conflate:

• Symptom — what the user noticed
• First plausible cause — the earliest step that appears wrong
• Owner — prompt, retrieval, tool call, model call, parser, evaluator, or infra

Find the first plausible failure point, not every downstream symptom. Fix that. Validate that the failing case no longer reproduces. Then stop — no scope creep into refactors, no forced experiment setup just to close one bug. If you want regression coverage, the skill hands off to honeyhive-evaluate explicitly.

‍

Where you'd use it:

• "Why did session abc123 fail?" Direct reconstruction from the exact failing trace.
• "Users say answers got worse this week." The skill narrows by time window and evaluator signal until the vague complaint becomes one concrete failing example — then investigates that.
• An evaluator went red on last night's experiment run. The skill inspects the run and its metrics before proposing anything, and is just as willing to conclude the evaluator is wrong as the app.

The guardrails: no trace-backed claims without traces — if you're not instrumented, the skill says so plainly and offers honeyhive-instrument or an offline loop instead of bluffing. No hidden mutations, no chasing every symptom, no framework-lore rabbit holes unless the observed failure actually points there.

‍

The skills compose: a Tuesday-morning production bug, end to end

It's Tuesday morning and a HoneyHive alert fires: groundedness on the pricing assistant dropped below threshold overnight, right after yesterday's deploy.

1. You paste the alert in your agent. You paste the HoneyHive URL: "This alert just fired. What's going on?" honeyhive-alert-root-cause triggers. It decodes the alert's filters and time window, pulls every flagged session, walks the trace trees, and sorts them with evidence: genuinely wrong pricing answers (true positives) versus a few sessions where the assistant correctly declined questions about unreleased SKUs and the evaluator flagged them anyway (false positives). The verdict is specific: a real regression in one retrieval path, with the noise already filtered out.

2. You point honeyhive-improve at the failures. "Dig into these true-positive sessions and find the root cause." It reconstructs them fast: in every bad session, the retrieval step returned zero documents, starting right after yesterday's query-rewrite change shipped. The symptom is wrong pricing answers, the likely cause is that the rewritten query drops the product SKU before retrieval, and the owner is the retrieval step. It gets there by reading the sessions from honeyhive.

3. It proposes the minimal fix and waits. The change is small: one function in the query builder. You approve. It applies the edit, re-runs a failing case, and confirms the documents come back and the answer is grounded. Then it waits for your next instruction.

4. You ask for a regression check. "Now set this up so it never regresses." honeyhive-evaluate lays out a plan and waits for your approval, then curates the failing sessions into a small dataset, adds a binary groundedness evaluator, runs the experiment, and checks that the run compares cleanly against a baseline. Next prompt or retrieval change, you have a tripwire.

‍

Three prompts and two approvals take you from a single alert URL to a fixed bug with a permanent regression test, with no need to re-explain how HoneyHive works.

Install

You need Node 18+ (for the skills CLI) and the HoneyHive CLI:

# 1. HoneyHive CLI — Homebrew (macOS, or Linux with Homebrew)
brew tap honeyhiveai/tap
brew install honeyhive
# Linux install script: see github.com/honeyhiveai/honeyhive-cli

# 2. Credentials in your env — never in source
export HH_API_KEY="<your-project-api-key>"
export HH_API_URL="https://api.dp1.us.honeyhive.ai"   # dedicated/self-host deployments have their own

# 3. Healthcheck
honeyhive events search --filters '[]' --limit 1

# 4. Install the skills
npx skills add honeyhiveai/skills --skill '*'

‍

The skills CLI installs into the per-agent skills directory for Cursor, Claude Code, Copilot, Codex, and others — see npx skills add --help for --agent and --global options. Then just talk to your agent: "add HoneyHive tracing to this app", "evaluate my agent", "debug this trace". The right skill triggers on its own.

How we built them (and what's next)

Three design principles run through all four skills, and they're worth stating because they're what separates a skill from a pasted doc:

‍Checkpoints that evals can grade. Every phase produces a concrete artifact — a recorded finding, a created resource, a graded trace. That makes skipped steps detectable, which means we can run automated evaluations against the skills themselves and catch regressions in agent behavior the same way you'd catch regressions in code.

‍

Qualitative success criteria. "Spans are flowing" is a mechanical bar. "An on-call engineer can debug this trace at 2am" is the real one. Encoding that judgment into a rubric the agent grades against is, we think, the most underrated part of writing good skills.

‍

Guardrails as hard rules. Agents fail in predictable ways — hardcoded secrets, dependency bumps, scope creep, confident clai

ms without evidence. Each skill carries explicit MUSTs targeting exactly those failure modes, so the floor is high even when the model has a bad day.

If you followed our recent work on Observability-Driven Development, you'll recognize the shape of this release: ODD argues that for long-running agents, the development loop runs through production traces — instrument, observe, evaluate, improve, repeat. These skills are that loop made executable. Not a framework you adopt, but a set of workflows your coding agent already knows how to run.

We eval these skills the same way you'd eval any agent — with traces, datasets, and experiments in HoneyHive. How we do that (and what the skills' own traces taught us about writing better skills) is the next post.

Try them, and when you hit a gap, the skills will file the issue for you — but we'd love to hear from you directly too: github.com/honeyhiveai/skills/issues.