Skip to main content

Observability with Clickhouse

The gateway supports OpenTelemetry tracing with ClickHouse as the storage backend. All traces are stored in the langdb.traces table.

Setting up​

  1. Create the traces table in ClickHouse:
# Create langdb database if it doesn't exist
clickhouse-client --query "CREATE DATABASE IF NOT EXISTS langdb"

# Import the traces table schema
clickhouse-client --query "$(cat sql/traces.sql)"
  1. Enable tracing by providing the ClickHouse URL when running the server:
ai-gateway serve --clickhouse-url "clickhouse://localhost:9000"

Or in config.yaml:

clickhouse:
  url: "http://localhost:8123"

Querying Traces​

Traces are stored in the langdb.traces table. Example query:

-- Get recent traces
SELECT
    trace_id,
    operation_name,
    start_time_us,
    finish_time_us,
    (finish_time_us - start_time_us) as duration_us
FROM langdb.traces
WHERE finish_date >= today() - 1
ORDER BY finish_time_us DESC
LIMIT 10;

Leveraging LangDB APIs within ClickHouse​

LangDB APIs can be called directly within ClickHouse. Check out our UDF documentation to learn how to use LLMs in SQL queries.

Running with Docker Compose​

For a complete setup, including ClickHouse for analytics and tracing, follow these steps:

  1. Start the services using Docker Compose:
docker-compose up -d

This will start:

  • ClickHouse server on ports 8123 (HTTP)
  • All necessary configurations loaded from docker/clickhouse/server/config.d
  1. Build and run the gateway:
ai-gateway run

The gateway will now be running with full analytics and logging capabilities, storing data in ClickHouse.