Notes on making agents survive production
Roughly monthly. Each one aims to contain at least one number you can check and one thing you can do on Monday. If a post is just an opinion, I have not finished it.
- 7 min read
What to look for in an agent design review
Reviewing an agent feature before it ships is the cheapest reliability work available, and most teams review the prompt rather than the architecture. Nine questions that catch the expensive problems while they are still free.
agent design reviewAI code review checklistLLM architecture reviewagent feature review
- 8 min read
Where your context window actually goes
A million-token window does not mean you should use it. Decompose one real conversation and the proportions are usually a surprise — history dominates, tool definitions are larger than expected, and the useful content is a minority of what you pay for.
LLM context windowcontext window budgetconversation history costLLM token usage
- 9 min read
Event-driven agents: what Kafka taught me about agent architecture
A 20% input spike once turned a healthy pipeline into an outage because it had no backpressure — it had optimism. Every concept that prevents that has a direct agent equivalent, and almost no agent codebase implements any of them.
event-driven agentsKafka LLM architectureasync agent designagent backpressureLLM streaming data
- 7 min read
LLM-as-judge is overused
It is slow, it costs money on every CI run, and it is non-deterministic — which means your test suite has flaky tests by construction. About 70% of the checks teams reach for a judge to make can be written as assertions instead.
LLM as judgeLLM evaluation methodsAI eval assertionsLLM testing CI
- 9 min read
MCP in production: what the protocol gives you and what it does not
Model Context Protocol moves your tool boundary into a place where validation and authorisation naturally live. That is the real benefit, and it is not the one usually advertised. What it does not give you: security, cheaper tokens, or a reason to trust tool arguments.
Model Context Protocol productionMCP best practicesMCP securityMCP tool designMCP vs function calling
- 8 min read
Model routing is the biggest cost lever, and it needs evals first
Routing tasks by difficulty typically recovers 40–70% of spend. It is also the only cost lever that can silently degrade quality — which is why the teams that most need it are usually the ones least equipped to do it safely.
LLM model routingreduce LLM costsmodel selection by taskLLM cost optimisation
- 8 min read
Prompt caching and semantic caching are not the same thing
They get discussed interchangeably and they solve different problems with different risks. One is a billing mechanism with a write premium that can lose you money; the other returns a previous answer to a different question and can leak across tenants.
prompt cachingsemantic caching LLMLLM cache hit rateLLM cost caching
- 7 min read
Prompt versioning: your prompts are deployable artefacts
If a customer reports a bad answer from last Tuesday, can you reconstruct the exact prompt that produced it? Most teams cannot, and it makes every subsequent investigation archaeology.
prompt versioningLLM prompt managementprompt rollbackLLM reproducibility
- 7 min read
The tool-definition tax: what your agent pays before it does anything
Every tool schema you attach is billed on every single call, whether the agent uses it or not — plus a per-model tool-use system prompt of 286 to 804 tokens. Here is how to measure that fixed cost and what it does to your bill at scale.
LLM cost optimisationtool definitions tokensagent costprompt cachingMCP
- 8 min read
Your agent doesn't have a model problem
Almost every failing agent I have looked at is failing on engineering discipline, not model capability. The evidence is that upgrading the model rarely fixes it — and the four things that do are unglamorous.
why AI agents fail in productionAI agent reliabilityLLM engineering disciplineproduction AI systems