Try it — type any question below
Total theoretical savings
$0.0335
vs GPT-4o baseline · $0.00 actual spend
Incoming Request
POST /v1/chat/completions
--Semantic Cache
Redis · cosine ≥ 0.92
17% hitComplexity Classifier
semantic-router · 3 routes
83% routedgroq/openai/gpt-oss-20b
~150ms · simplest queries
20%groq/llama-3.3-70b-versatile
~400ms · explanations
20%groq/openai/gpt-oss-120b
~800ms · reasoning
60%Ollama Fallback
qwen2.5 / llama3.2
0%cache hit rate
16.7%
fallback rate
0.0%
classifier accuracy
88.3%
total requests
6
| Time ↓ | Model ↕ | Tokens | Latency ↕ | Savings ↕ | Routing |
|---|---|---|---|---|---|
| 21:20:05 | groq/openai/gpt-oss-120b | 73→39 | 900ms | $0.000000 | classifier · complex |
| 18:47:41 | groq/llama-3.3-70b-versatile | 39→275 | 1615ms | $0.004149 | classifier · medium |
| 12:31:37 | groq/openai/gpt-oss-120b | 77→1786 | 4764ms | $0.000000 | classifier · complex |
| 12:30:50 | cache | 8→990 | 798ms | $0.014890 | cache · cosine match |
| 12:29:53 | groq/openai/gpt-oss-20b | 77→1024 | 1961ms | $0.014446 | classifier · simple |