Get started for free

An event refers to a single trace span, structured log, or metric label combination sent to our API as OTLP or JSON. It captures any relevant data from your system, including all context fields generated by your application's instrumentation.

Automated Evaluators: An automated evaluator is a function (code or LLM) that helps you unit test any arbitrary event or combinations of events to generate a measurable score (and explanation, in case of LLM evaluators). Common examples of auto-evaluators include Context Relevance, Answer Faithfulness, ROUGE, BERTScore, and more. We provide many common evaluators out-of-the-box and allow defining custom evaluators within the platform.

Human Evaluators: We strongly encourage a hybrid-evaluation approach, i.e. combining automated techniques with human oversight. This helps you account for evaluation criteria bias and better align your evaluators with your domain experts' scoring rubric. To enable this, you can define custom scoring rubrics in HoneyHive for domain experts to use when evaluating outputs.

HoneyHive allows you to filter and curate datasets from your production logs. These datasets can be annotated by domain experts within the platform and exported programmatically for fine-tuning models.

You can export datasets curated within HoneyHive using our SDK and use your preferred 3rd-party provider to fine-tune custom models. Our DSL's flexibility also supports more advanced use-cases like Active Learning. Contact us to learn more.

All data is secure and encrypted at rest and in transit. We are SOC-2 compliant and GDPR aligned, conduct regular penetration tests via 3rd-party auditors, and provide flexible hosting solutions to meet your security and compliance needs. Contact us to learn more.

You can self-host HoneyHive in your Virtual Private Cloud (VPC) on the Enterprise plan. We support self-hosting in AWS, Azure, and GCP via Kubernetes, and are happy to provide additional support for highly custom deployments. Contact us to learn more.

No, we do not proxy your requests via our servers. Instead, we store prompts as YAML configurations, which can be deployed and fetched in your application logic using the GET Configuration API or by setting up a custom GitHub Workflow.

You can log traces using our SDKs and API endpoints, or async via our batch ingestion endpoint. We offer native SDKs in Python and Typescript with OpenTelemetry support, and provide automatic integrations with popular frameworks like LangChain, LlamaIndex, Vercel AI SDK, and others.

For users using other languages, you can send your OpenTelemetry traces to our OTel collector or manually instrument your application using our APIs.