Product Update: OpenTelemetry-native SDK

Few months ago, we announced our shift to a trace and span native data model to better support emerging agent architectures. Today, we're excited to share the next big step in our journey: our new OpenTelemetry-native SDKs and collectors for Python and TypeScript.

‍

Why We Built This

‍

We've been listening to your feedback, and we know that adding observability to AI applications can be a headache, especially with the complexity of stitching together various requests across distributed systems. That's why we've focused on making our new tracers as simple, powerful, and interoperable with your existing stack as possible.

‍

Automated instrumentation: Just 3 lines of code to start automatically instrumenting popular LLMs and vector databases like OpenAI, Anthropic, and Pinecone.
OpenTelemetry goodness: We're fully embracing the OpenTelemetry standard, making it easy to collect traces and feedback, run evaluations on your logs, generate metrics, and export your data.
AI-focused abstractions: Our SDK has built-in support for distributed tracing, multi-modal applications, and custom spans and properties along with automatic cost, latency, and token calculation, perfect for building powerful AI agents.
Support for additional languages: If you're developing applications in languages other than Python and TypeScript, you can easily send traces to our OTEL-compatible collector, or use our APIs for fine-grained instrumentation of your application.
Scalability & performance: Our cloud platform has been rigorously tested to ingest over 1000 requests per second, enabling seamless scaling to production workloads.

‍

A Peek Under the Hood

‍

Our OpenTelemetry tracer works by automatically logging any /POST and /GET calls to the below providers and automatically generating trace spans that are received by our OTEL-compatible collector. Any other providers and steps can be manually instrumented using @trace method.

‍

Model Providers: OpenAI, Azure OpenAI, Anthropic, AWS Bedrock, Google AI Studio (Gemini), Google Vertex AI, Mistral AI, Cohere, Ollama, HuggingFace, Replicate, IBM WatsonX, and Together AI
Vector Databases: Pinecone, Chroma, Qdrant, Weaviate, and Milvus

‍

Let's take a quick look at how easy it is to get started:

It takes 3 lines of code to get started with our Python SDK

‍

And for you TypeScript fans out there, it's just as simple.

Our Typescript SDK automatically traces any /PUT and /GET calls

‍

That's it! With these simple steps, HoneyHive automatically captures input/output data, visualizes your application's data flow, and enables comprehensive tracing.

You can easily explore traces, provide feedback, and use auto-evals to debug issues.

‍

After setting up the integration, you can:

‍

Debug: Easily explore your traces, filter down on specific spans, and share traces with your team.
Evaluate: Run auto-evaluations on your logs, or manually annotate logs with domain experts.
Analyze Trends: Easily slice and dice your data and analyze common trends across user cohorts, input topics, and more.
Curate Datasets: Leverage your logs to curate domain-specific datasets, that can be used to run offline evaluations to test your system’s performance or to fine-tune custom models for improved results.
Build Automations: Export traces and metrics programmatically using our DSL to build automations like active learning pipelines on top of HoneyHive.

‍

Other Product Updates

Introducing Keyword Search: You can now search over any input and output field in HoneyHive, allowing you to quickly search for similar keywords, narrow results, and debug specific traces faster.‍
New Models in Playground: We added GPT-4o and Claude 3.5 Sonnet to our Playground.‍
Query speed improvements: We migrated our core databases to Clickhouse, resulting in over 10x improvements in query speeds. Querying 1M+ events (~100GB of raw logs) now takes less than 10 seconds.‍
New Documentation: We rewrote our docs from scratch for easier onboarding and enablement.‍
Semantic Versioning: We now strictly follow semantic versioning for all our releases, ensuring easier migration and backwards compatibility.‍
Bug Fixes: Resolved various bugs throughout the platform for improved reliability.

‍

Getting started

‍

We're excited about embracing and contributing to open-source standards for AI observability. We welcome your thoughts, feedback, and feature requests. Join our beta waitlist or email us at [email protected] to share your feedback.

About the author:

Mohak Sharma

Co-Founder and CEO

Product Update: OpenTelemetry-native SDK

About the author:

Mohak Sharma

Join our monthly newsletter