Caching stable context, trimming repeated instructions, and measuring token waste did more than model shopping. What optimization produced your biggest practical gain?

I have evals, abuse cases, latency budgets, fallbacks, data retention, monitoring, and a way to report bad outputs. What is missing?

Our prototype answered beautifully but ignored permissions, handoffs, and the place users needed the result. What helped you cross the gap from impressive demo to useful product?

Schema-constrained generation helps, but edge cases remain. How do you balance automatic repair with the risk of silently changing meaning?

Describe the smallest stack that reliably serves real users: model gateway, retrieval, jobs, storage, observability, and the pieces you intentionally left out.