Developer Tooling
Building LLM Applications for Production
Three foundational guides on taking LLM applications from prototype to production. The common thread is that the model is the easy part — everything around it is where teams struggle.
What We’ve Learned from a Year of Building with LLMs
TLDR: A landmark collaborative guide distilling tactical, operational, and strategic lessons from production LLM development. Written by practitioners who have shipped real systems, not theorists.
Key Insight: Build robust evals before building features — if you cannot measure quality, you cannot improve it.
Building LLM Applications for Production
TLDR: Chip Huyen’s guide to the real challenges of productionizing LLMs: prompt ambiguity, control flow design, fewshot learning, and testing strategies. The gap between demo and production is wider than most teams expect.
Key Insight: The hardest part of production LLM work is not the model — it is control flow: retries, fallbacks, output validation, and graceful degradation.
Emerging Architectures for LLM Applications
TLDR: A reference architecture from a16z mapping the LLM application stack: data pipelines, embedding models, orchestration layers, vector databases, and model APIs. The post established vocabulary the industry still uses.
Key Insight: The LLM stack is stabilizing around a clear pattern: data to embeddings to vector store to orchestration to model to validation.
What does this mean for developers?
Production LLM engineering is a discipline distinct from both traditional software engineering and ML research. Developers building LLM applications should prioritize evaluation frameworks, control flow design, and architecture patterns over model selection. The stack is maturing, and the patterns documented here are becoming industry standard.