Variables & Functions

Available Metrics

LangDB provides a rich set of real-time metrics for making dynamic, data-driven routing decisions. These metrics are aggregated at the provider level, giving you a live view of model performance.

Metric Name	Description	Example Value	Business Value
ttft	Time to First Token (ms)	450	Optimize for user-perceived speed
tps	Tokens Per Second (output_tokens/latency)	300	Measure model generation speed
error_rate	Fraction of failed requests	0.01	Route around unreliable models
latency	Average end-to-end response time (ms)	1100	Route based on overall performance
requests	Total number of requests processed	1500	Monitor traffic and usage patterns
cost	Estimated request cost (USD)	0.02	Track and control spend in real time

Available Variables

Variables provide contextual information from the incoming request and user metadata. Unlike metrics, they are not performance indicators but are essential for conditional logic.

User Information

Variable Path	Description	Example Value	Business Use
extra.user.id	User identifier from request	"u-12345"	Custom user identification
extra.user.name	User name from request	"john.doe"	Personalization
extra.user.email	User email from request	"john@example.com"	User-based routing
extra.user.tier	User tier from request	"premium"	SLA-based routing
extra.user.tiers	Array of user tiers from request	["premium", "beta"]	Multi-tier access control
extra.user.*	Any custom field in extra.user	Various	Custom business logic

Request Metadata

Variable Path	Description	Example Value	Business Use
metadata.ip	IP address of the requester	"203.0.113.42"	Geo-fencing, fraud detection
metadata.region	Geographical region (based on IP)	"Europe"	Data residency, compliance
metadata.country	Country code (based on IP)	"DE"	Country-specific routing
metadata.user_agent	Client application	"Google ADK/ CrewAI"	Agentic library detection
metadata.user_id	LangDB user ID	"langdb-u-12345"	Internal user identification
metadata.group_id	LangDB cost group ID	"grp-456"	Cost group identification
metadata.group_name	LangDB cost group name	"development"	Cost group-based routing
metadata.tags.{tag}	LangDB user tags	"premium"	User classification

Pre-Request Interceptor Results

Variable Path	Description	Example Value	Business Use
pre_request.{interceptor_name}.passed	Whether interceptor check passed	true	Conditional routing based on checks
pre_request.rate_limiter.passed	Rate limit check result	true	Enforce usage quotas
pre_request.semantic_guardrail.passed	Content policy check result	false	Block policy violations
pre_request.{guardrail}.result.topic	Detected topic from guardrail	"billing"	Topic-based model routing
pre_request.{guardrail}.result.*	Any custom result from interceptor	Various	Custom business logic

Provider Performance Metrics (for Filtering)

Variable Path	Description	Example Value	Business Use
provider.error_rate	Provider's current error rate	0.02	Avoid unreliable providers
model_capabilities	Supported features (vision, code)	["vision"]	Feature-based routing
model_tags	Model classification tags	["fast"]	Tag-based filtering

Target Configuration

Configuration Type	Description	Example Usage	When to Use
Simple List	Direct array of model names	["openai/gpt-4o", "anthropic/claude-4-sonnet"]	Fixed model selection
Provider Wildcards	All models from specific provider	["openai/*", "anthropic/*"]	Provider-level routing
Single Target	Direct model specification	"openai/gpt-4o"	Fixed routing to one model
$any Pool	Pool of models with selection logic	{ "$any": [...], "sort_by": "price" }	Dynamic model selection

Model Target Formats

Format	Description	Example	Use Case
Exact Model Name	Specific model identifier	"openai/gpt-4o"	Direct routing
Provider Wildcard	All models from provider	"openai/*"	Provider-level failover

Sorting Options

Sort Configuration	What It Does	When to Use	Example JSON Usage
"sort_by": "price", "sort_order": "min"	Picks the cheapest model	Cost control, bulk/low-priority tasks	"sort_by": "price", "sort_order": "min"
"sort_by": "price", "sort_order": "max"	Picks the most expensive model	Premium users, highest quality needed	"sort_by": "price", "sort_order": "max"
"sort_by": "ttft", "sort_order": "min"	Picks the fastest model (time-to-first-token)	VIP, real-time, or user-facing tasks	"sort_by": "ttft", "sort_order": "min"
"sort_by": "latency", "sort_order": "min"	Picks the lowest latency model	Performance-critical applications	"sort_by": "latency", "sort_order": "min"
"sort_by": "error_rate", "sort_order": "min"	Picks the most reliable model	Mission-critical or regulated workflows	"sort_by": "error_rate", "sort_order": "min"
"sort_by": "tps", "sort_order": "max"	Picks the fastest token generation model	Bulk generation, high throughput	"sort_by": "tps", "sort_order": "max"
"sort_by": "requests", "sort_order": "min"	Picks the least loaded model	Load balancing	"sort_by": "requests", "sort_order": "min"

Complete Target Examples

Simple Array Targets

{
  "targets": ["openai/gpt-4o", "anthropic/claude-4-sonnet"]
}

Provider Wildcard Targets

{
  "targets": ["openai/*", "anthropic/*"]
}

Single Target

{
  "targets": "openai/gpt-4o"
}

Advanced $any Pool with Filtering and Sorting

{
  "targets": {
    "$any": [
      "openai/*",
      "anthropic/claude-4-sonnet",
      "mistral/mistral-large-latest"
    ],
    "filter": {
      "ttft": { "$lt": 1000 },
      "provider.error_rate": { "$lt": 0.05 }
    },
    "sort_by": "price",
    "sort_order": "min"
  }
}

Filter Options

Filter Type	Description	Operators Supported	Example Usage
filter: { ttft: {...} }	Filter models by time-to-first-token	$lt, $gt, $lte, $gte, $eq	"ttft": { "$lt": 1000 }
filter: { price: {...} }	Filter models by cost	$lt, $gt, $lte, $gte, $eq	"price": { "$lt": 0.02 }
filter: { error_rate: {...} }	Filter models by reliability	$lt, $gt, $lte, $gte, $eq	"error_rate": { "$lt": 0.05 }
filter: { provider.error_rate: {...} }	Filter by provider reliability	$lt, $gt, $lte, $gte, $eq	"provider.error_rate": { "$lt": 0.02 }
filter: { model_tags: {...} }	Filter by model classification tags	$contains, $in	"model_tags": { "$contains": "fast" }

Comparison Operators

Operator	Description	Example Usage	Use Case
$eq	Equals	"tier": { "$eq": "premium" }	Exact matching
$ne	Not equals	"tier": { "$ne": "free" }	Exclusion logic
$lt	Less than	"ttft": { "$lt": 1000 }	Performance thresholds
$lte	Less than or equal	"price": { "$lte": 0.02 }	Budget constraints
$gt	Greater than	"tps": { "$gt": 100 }	Minimum performance requirements
$gte	Greater than or equal	"reliability": { "$gte": 0.95 }	Quality standards
$in	In array	"region": { "$in": ["US", "EU"] }	Multiple value matching
$contains	Array contains value	"tags": { "$contains": "GDPR" }	Tag-based filtering

Important Notes:

• Targets: Can be a simple array, single string, or $any object with filtering/sorting
• Wildcards: Use * for provider-level routing (e.g., "openai/*" = all OpenAI models)
• Sorting: Use "sort_by" and "sort_order" fields separately
• Operators: Always use $ prefix for comparison operators ($eq, $lt, etc.)
• Values: Use lowercase for sort order ("min", "max")