Tracing RAG applications in production with LanceDB and HoneyHive

Modern AI applications face two key challenges: efficiently managing vector data and gaining visibility into performance. LanceDB and HoneyHive together provide a powerful solution by combining a serverless vector database with comprehensive tracing capabilities.
‍

Traditional vector databases require complex server management and lack integrated monitoring. Meanwhile, most monitoring solutions aren't designed for vector search operations. The LanceDB and HoneyHive integration addresses these gaps, offering engineering teams better control over their AI applications.

‍

‍

The Solution: LanceDB + HoneyHive

LanceDB stands out as a developer-friendly, serverless vector database built specifically for AI applications. Its embedded architecture, persistent storage capabilities, and true multimodal support make it an ideal foundation for modern AI applications. Unlike other vector databases that only store embeddings and metadata, LanceDB can store the actual data alongside vectors, simplifying your data architecture.

HoneyHive complements LanceDB perfectly by providing specialized tracing and monitoring for vector database operations. By integrating HoneyHive with LanceDB, engineering teams gain visibility into embedding quality, retrieval performance, and optimization opportunities throughout their RAG pipelines.

‍

‍

What You'll Learn in This Blog

In this comprehensive guide, we'll explore:
‍

1. The unique features of LanceDB that make it ideal for AI applications
2. How HoneyHive enhances vector database operations through specialized tracing
3. Step-by-step implementation of a complete RAG pipeline with LanceDB and HoneyHive
4. Performance insights and optimization strategies enabled by this integration
5. Real-world benefits for engineering teams and leaders

Whether you're an engineering leader looking to improve your team's AI infrastructure or an engineer seeking to optimize your vector search applications, this integration offers tangible benefits that we'll explore in detail. Let's dive into how LanceDB and HoneyHive work together to supercharge your AI applications.

‍

‍

LanceDB: The Serverless Vector Database for AI

‍

‍

Serverless/Embedded Architecture

LanceDB operates in-process within your application, eliminating the need for separate database servers. This approach offers:
‍

• Zero infrastructure overhead
• Simplified deployment in serverless environments
• Lower latency by eliminating network round-trips
• Reduced operational costs

‍

On-Disk/Persisted Storage

Unlike memory-based vector databases, LanceDB uses disk-based, persisted storage:
‍

• Store billions of vectors without RAM constraints
• Cost-effective scaling with economical disk storage
• Automatic versioning without additional infrastructure
• Data durability across application restarts

‍

True Multimodal Support

LanceDB stores actual multimodal data alongside vector embeddings:
‍

• Text documents, images, videos, audio, and point clouds
• Original content alongside vector representations
• Rich metadata with comprehensive filtering options
• Simplified data architecture by eliminating separate storage systems

‍

HoneyHive: Tracing for Vector Database Operations

HoneyHive provides specialized tracing for AI applications, offering unprecedented visibility into vector operations. When integrated with LanceDB, it enables teams to:

‍

Track Vector Operations

• Monitor embedding generation processes
• Capture query execution paths and performance metrics
• Analyze data ingestion workflows

‍

Monitor Embedding Quality

• Compare embedding model performance
• Track embedding drift over time
• Evaluate dimension choice impact

‍

Evaluate Retrieval Performance

• Measure retrieval latency across query types
• Assess retrieved document relevance
• Compare similarity metrics effectiveness

‍

Identify Optimization Opportunities

• Pinpoint RAG pipeline bottlenecks
• Detect suboptimal index configurations
• Recognize caching opportunities

‍

‍

The Engineering Perspective: Why Tracing Matters

‍

From an engineering standpoint, the addition of HoneyHive tracing to LanceDB operations addresses several critical needs:

‍

• Debugging complex pipelines: Trace the full execution path from query to response
• Performance optimization: Identify and address bottlenecks with precision
• Quality assurance: Monitor retrieval relevance and embedding quality over time
• Cost management: Understand resource utilization and optimize accordingly
• Continuous improvement: Make data-driven decisions about model and parameter choices

‍

As one engineering leader put it: "Before adding HoneyHive tracing to our LanceDB operations, optimizing our RAG pipeline was like flying blind. Now we have a cockpit with all the instruments we need to navigate effectively."

‍

In the next section, we'll explore a complete implementation example that demonstrates how to build a RAG pipeline with LanceDB and HoneyHive tracing from the ground up.

‍

‍

Implementation Guide: Building a RAG Pipeline with LanceDB and HoneyHive

‍

In this section, we'll walk through a complete implementation of a Retrieval Augmented Generation (RAG) pipeline using LanceDB for vector storage and HoneyHive for tracing. This practical example demonstrates how these technologies work together to create a powerful, observable AI application.

‍

Prerequisites

Before we begin, ensure you have the following:
‍

• Python 3.8+
• A HoneyHive account and API key
• An OpenAI API key (for embeddings and LLM generation)
• Basic understanding of RAG pipelines

‍

First, let's install the required packages:
‍

pip install lancedb honeyhive sentence-transformers openai pandas

Implementing HoneyHive tracing with LanceDB is remarkably straightforward. The integration uses a decorator-based approach that adds minimal overhead to your existing code.

‍

This simple addition of the @trace decorator provides comprehensive visibility into your LanceDB operations without requiring complex instrumentation or code changes.

‍

‍

Step 1: Initialize Clients and Setup

We'll start by setting up the necessary clients and configuration for HoneyHive, OpenAI, and LanceDB:
‍

import os
import sys
import logging
import pandas as pd
import lancedb
from lancedb.pydantic import LanceModel, Vector
from lancedb.embeddings import get_registry
from openai import OpenAI
from honeyhive import HoneyHiveTracer, trace
from typing import List, Dict, Any

‍

Configure logging

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler("rag_pipeline.log"),
        logging.StreamHandler(sys.stdout)
    ]
)
logger = logging.getLogger("lancedb_rag")

‍

Initialize HoneyHive tracer

HONEYHIVE_API_KEY = os.environ.get("HONEYHIVE_API_KEY", "your_honeyhive_api_key")
HONEYHIVE_PROJECT = os.environ.get("HONEYHIVE_PROJECT", "your_project_name")

HoneyHiveTracer.init(
    api_key=HONEYHIVE_API_KEY,
    project=HONEYHIVE_PROJECT,
    source="dev",
    session_name="lancedb_rag_session"
)

‍

Set OpenAI API key

OPENAI_API_KEY = os.environ.get("OPENAI_API_KEY", "your_openai_api_key")
openai_client = OpenAI(api_key=OPENAI_API_KEY)

This setup establishes logging, initializes the HoneyHive tracer with your project details, and configures the OpenAI client. The HoneyHive tracer will be used to monitor and analyze each step of our RAG pipeline.

‍

Step 2: Define Document Class

Next, let's create a simple document class to hold our text chunks:

‍

class Document:
    """Simple document class to hold text chunks."""
    def __init__(self, text: str, metadata: Dict[str, Any] = None):
        self.text = text
        self.metadata = metadata or {}

‍

This class provides a consistent structure for our documents, including both the text content and associated metadata.

‍

Step 3: Load and Process Documents with Tracing

Now, let's create functions to load and chunk documents with HoneyHive tracing:

‍

@trace
def load_documents(file_path: str) -> List[Document]:
    """
    Load documents from a text file.
    Each line is treated as a separate document.
    """
    logger.info(f"Loading documents from {file_path}")
    documents = []
    
    try:
        with open(file_path, 'r') as f:
            lines = f.readlines()
            
        for i, line in enumerate(lines):
            if line.strip():  # Skip empty lines
                doc = Document(
                    text=line.strip(),
                    metadata={"source": file_path, "line_number": i}
                )
                documents.append(doc)
                
        logger.info(f"Loaded {len(documents)} documents")
        return documents
    except Exception as e:
        logger.error(f"Error loading documents: {e}")
        raise

@trace
def chunk_documents(documents: List[Document], chunk_size: int = 1000) -> List[str]:
    """
    Split documents into smaller chunks.
    """
    logger.info(f"Chunking {len(documents)} documents with chunk size {chunk_size}")
    chunks = []
    
    for doc in documents:
        text = doc.text
        # Simple chunking by character count
        if len(text) <= chunk_size:
            chunks.append(text)
        else:
            # Split into chunks of approximately chunk_size characters
            for i in range(0, len(text), chunk_size):
                chunk = text[i:i + chunk_size]
                chunks.append(chunk)
    
    logger.info(f"Created {len(chunks)} chunks")
    return chunks

‍

The @trace decorator logs these operations to HoneyHive, capturing metadata about document loading and chunking processes. This visibility is crucial for understanding how your data preparation affects downstream performance.

‍

Step 4: Create LanceDB Table with Tracing

Next, let's set up a LanceDB table with embeddings:

‍

@trace
def create_lancedb_table(chunks: List[str], table_name: str = "docs"):
    """
    Create a LanceDB table with embeddings.
    """
    logger.info(f"Creating LanceDB table '{table_name}' with {len(chunks)} chunks")
    
    # Connect to LanceDB
    db = lancedb.connect("/tmp/lancedb")
    
    # Get embedding model
    model = get_registry().get("sentence-transformers").create(
        name="BAAI/bge-small-en-v1.5", 
        device="cpu"
    )
    
    # Define schema
    class Docs(LanceModel):
        text: str = model.SourceField()
        vector: Vector(model.ndims()) = model.VectorField()
    
    # Create table
    df = pd.DataFrame({'text': chunks})
    
    # Check if table exists and drop if it does
    if table_name in db.table_names():
        db.drop_table(table_name)
    
    # Create new table
    table = db.create_table(table_name, schema=Docs)
    
    # Add data
    table.add(data=df)
    
    logger.info(f"Created table '{table_name}' with {len(chunks)} rows")
    return table

‍

This function creates a LanceDB table and adds document chunks with embeddings. The @trace decorator logs information about the embedding model used and table creation process, providing visibility into this critical step.

‍

Step 5: Retrieve Documents with Tracing

Now, let's create a function to retrieve relevant documents from LanceDB:

‍

@trace
def retrieve_documents(query: str, table_name: str = "docs", limit: int = 3):
    """
    Retrieve relevant documents from LanceDB.
    """
    logger.info(f"Retrieving documents for query: '{query}'")
    
    # Connect to LanceDB
    db = lancedb.connect("/tmp/lancedb")
    
    # Get table
    table = db.open_table(table_name)
    
    # Search
    results = table.search(query).limit(limit).to_list()
    
    logger.info(f"Retrieved {len(results)} documents")
    return results

The @trace decorator logs information about the retrieval process, including the query and number of results. This visibility helps you understand how well your vector search is performing and identify potential improvements.

‍

Step 6: Generate Response with Tracing

Let's create a function to generate a response using OpenAI with tracing:

‍

@trace
def generate_answer(query: str, context: List[Dict[str, Any]]):
    """
    Generate an answer using OpenAI's API.
    """
    logger.info(f"Generating answer for query: '{query}'")
    
    # Extract text from context
    context_text = "\n\n".join([item["text"] for item in context])
    
    # Create prompt
    prompt = f"""
    Answer the following question based on the provided context:
    
    Context:
    {context_text}
    
    Question: {query}
    
    Answer:
    """
    
    # Call OpenAI API
    response = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that answers questions based on the provided context."},
            {"role": "user", "content": prompt}
        ],
        temperature=0.3,
        max_tokens=500
    )
    
    answer = response.choices[0].message.content
    logger.info(f"Generated answer: '{answer[:100]}...'")
    
    return answer

‍

This function generates a response using OpenAI based on the retrieved documents. The @trace decorator logs information about the generation process, including the prompt construction and response.

‍

Step 7: Complete RAG Pipeline with Tracing

Finally, let's create a function that combines all the previous steps into a complete RAG pipeline:

‍

@trace
def rag_pipeline(query: str, data_path: str):
    """
    End-to-end RAG pipeline.
    """
    logger.info(f"Starting RAG pipeline for query: '{query}'")
    
    # 1. Load documents
    documents = load_documents(data_path)
    
    # 2. Chunk documents
    chunks = chunk_documents(documents)
    
    # 3. Create vector store
    table = create_lancedb_table(chunks)
    
    # 4. Retrieve relevant documents
    results = retrieve_documents(query)
    
    # 5. Generate answer
    answer = generate_answer(query, results)
    
    logger.info("RAG pipeline completed successfully")
    return answer

‍

The @trace decorator logs the entire RAG pipeline process, creating a parent span that contains all the child spans from the individual functions. This hierarchical view in HoneyHive provides a comprehensive understanding of your pipeline's performance.

‍

Step 8: Run the Example

Let's create a main function to run our example:

‍

def main():
    """
    Main function to demonstrate the RAG pipeline.
    """
    # Sample data path - replace with your actual data file
    data_path = "data/sample_data.txt"
    
    # Create sample data if it doesn't exist
    os.makedirs(os.path.dirname(data_path), exist_ok=True)
    if not os.path.exists(data_path):
        with open(data_path, 'w') as f:
            f.write("LanceDB is a vector database for AI applications.\n")
            f.write("It provides high-performance vector search capabilities.\n")
            f.write("LanceDB can be used for RAG applications to improve LLM responses.\n")
            f.write("RAG stands for Retrieval Augmented Generation.\n")
            f.write("Vector databases store embeddings which are numerical representations of data.\n")
    
    # Sample query
    query = "What is LanceDB and how can it be used for RAG?"
    
    # Run RAG pipeline
    answer = rag_pipeline(query, data_path)
    
    print("\n=== Final Answer ===")
    print(answer)
    
    # End HoneyHive tracing session
    HoneyHiveTracer.init(
        api_key=HONEYHIVE_API_KEY,
        project=HONEYHIVE_PROJECT,
        source="dev",
        session_name="new_session"  # This ends the previous session and starts a new one
    )

if __name__ == "__main__":
    main()

‍

This implementation uses the @trace decorator to log operations to HoneyHive, providing visibility into each step of the RAG pipeline.

How it helps

The LanceDB and HoneyHive integration offers significant advantages for AI applications:

‍

• Simplified Infrastructure
  • The serverless, embedded nature of LanceDB eliminates the need for complex database infrastructure, reducing operational overhead and allowing teams to focus on application logic rather than database management.
• Cost-Effective Scaling
  • LanceDB's disk-based, persistent storage approach enables economical scaling to billions of vectors without proportional increases in infrastructure costs—a critical consideration as vector datasets grow rapidly.
• True Multimodal Support
  • Unlike traditional vector databases that only store embeddings, LanceDB's ability to store the actual multimodal data alongside vectors simplifies architecture and enables more sophisticated retrieval strategies.
• Comprehensive Visibility
  • HoneyHive's specialized tracing capabilities provide unprecedented insights into every aspect of vector operations, from embedding generation to retrieval performance, enabling data-driven optimization.
• Continuous Improvement
  • The combination of performance metrics and quality indicators from HoneyHive allows teams to iteratively improve their RAG pipelines based on real-world usage patterns and outcomes.

‍

Real-World Impact

Organizations using this integration have reported:
‍

• Large reduction in infrastructure costs
• Faster and more accurate improvement in retrieval relevance
• 2-3x faster development cycles
• Significant reduction in LLM hallucinations

‍

By combining efficient vector storage with comprehensive tracing, this integration addresses two critical challenges in modern AI applications: data management and performance visibility.

‍

Resources

‍

• LanceDB GitHub
• HoneyHive Documentation

‍

‍