Cutting LLM cost by 60% in production while keeping quality, with full observability.
We run LLM workloads through OpenRouter and track regular and reasoning tokens, model response time and whether answers match the expected schema. Cost came down through cheaper models where they are good enough, caching and prompt corrections, and quality is checked by a second model plus test benches that run the same data through different models many times. Queues and a fallback model keep the service up, and sensitive data is masked or replaced with placeholders before it reaches a model. The hard part was balancing price against quality.
- OpenRouter
- Token tracking
- Model evaluation
- Fallback models
- Data masking