Decision support
Build or buy: LLM observability
The decision costs more in engineering time than in licence fees, which is the part most comparisons omit.
The question
Do we adopt an LLM observability product, extend what we already run, or build our own?
The short answer
Buy, unless you have a hard data-residency requirement or an existing observability platform with a team who owns it — in which case extend that. Building from scratch is almost never right: the initial version takes days, and the maintenance you did not budget for is what actually costs you. What matters far more than the vendor choice is field coverage, because a tool without cost, validity and stop-reason fields cannot answer the questions you will ask it.
Written for
- Engineering leaders choosing a tracing approach
- Platform teams that already run observability infrastructure
- CTOs with data residency or procurement constraints
Why the answer is that
The comparison people run is licence cost against zero, which makes building look free. The real comparison is licence cost against engineering time — and specifically against maintenance time, which is where home-built observability quietly consumes a quarter of someone's capacity forever.
The initial build genuinely is easy. Emitting spans with prompt, response, tokens and latency is a few days of work and it feels like a win. What follows is a trace viewer nobody likes using, retention and storage decisions, search that does not scale, and a slow realisation that you are maintaining an internal product with one grudging maintainer and no roadmap.
There is one case where building is correct: you already run a mature observability platform, a team owns it, and LLM spans are a schema addition rather than a new system. That is extending, not building, and it is genuinely cheaper than adopting a second vendor.
The decision that actually matters is not the vendor. It is whether whatever you pick captures the fields that make cost and correctness debuggable — and most teams discover the gaps only when they need an answer they cannot get.
Where each option genuinely wins
Pick the row that describes you rather than comparing feature lists.
| Your situation | Do this | Because |
|---|---|---|
| No existing observability practice, small team | Buy managed | You need answers this month, not a platform project. Setup is hours. |
| Hard data-residency or air-gap requirement | Self-host an open-source tool | Gets you the product without the data leaving. Budget for someone to run it. |
| Mature observability platform with an owning team | Extend it | LLM spans become a schema addition. Cheapest real option, and one pane of glass. |
| Procurement will take six months | Self-host now, revisit later | Six months without tracing costs more than the tool. Do not wait. |
| Highly unusual agent architecture | Buy anyway, supplement | Use the tool for what fits and add custom spans. Rebuilding the viewer is not the win. |
| "We could build this in a sprint" | Buy | You could build the first version in a sprint. The sprint is not the cost. |
Notice that "build from scratch" is not the answer in any row. That is deliberate — I have not yet seen a case where a from-zero build beat buying or extending, once maintenance was counted honestly.
The costs each option actually carries
The line items that get left out of the business case. Engineering time is the dominant term in every column and is usually the one nobody prices.
| Cost | Buy managed | Self-host | Build |
|---|---|---|---|
| Initial setup | Hours | Days | Days — deceptively cheap |
| Licence or infrastructure | Per-trace or seat pricing | Infrastructure only | Storage only |
| Ongoing maintenance | None | Upgrades, storage, uptime | It is now an internal product |
| Trace viewer and search | Included | Included | You build and maintain it |
| Eval integration | Usually included | Usually included | Build it too |
| Onboarding a new engineer | Documented product | Documented product | Tribal knowledge |
| What happens when the owner leaves | Nothing | Find a new operator | Quiet decay, then a rebuild |
| Opportunity cost | Low | Medium | High — this is the real number |
The last row decides it. Engineering time spent on an internal trace viewer is time not spent on the product, and the comparison is rarely made explicitly enough for that to be obvious.
Get the answer for your own system in four minutes
The free scorecard produces a score out of 100, a breakdown across reliability, cost and observability, and your gaps ranked by production risk with a fix and effort estimate for each. It prints to a PDF you can take into a planning meeting — which is usually more persuasive than an argument.
Run the scorecardIf you are the engineer who found this page: Written to be forwarded. If you are the engineer who found this, send the relevant one up — it is framed for the person who has to approve the work.
If you decide to do the work
The engagements this decision leads to, each with the full method and effort estimates — whether your team runs it or I do.
Questions
Should we build or buy LLM observability?
Buy, unless you have a hard data-residency requirement — in which case self-host an open-source option — or you already run a mature observability platform with a team who owns it, in which case extend that.
Building from scratch is almost never right. The first version takes days, which is what makes it tempting; the maintenance, the trace viewer, the search and the onboarding cost are what you have not budgeted.
Which LLM observability tool is best?
Field coverage matters far more than vendor. Langfuse, LangSmith, Braintrust, Arize Phoenix and plain OpenTelemetry can all carry what you need — a tool with complete fields beats a better tool with gaps.
Decide on data residency and pricing model rather than features, because the features converge and those two constraints do not.
Is OpenTelemetry enough?
Technically yes, and if you already run OpenTelemetry it is the lowest-friction path — LLM spans are conventions on top of ordinary tracing.
What you give up is the purpose-built interface: prompt diffing, replay, and eval integration. The practical test is whether your team will open a tracing UI unprompted. If they already live in traces, OpenTelemetry is fine. If not, a dedicated tool gets used, and a tool that gets used beats a better one that does not.
How much should we budget for LLM observability?
Budget engineering time first, licence cost second — the ratio surprises people. Adoption is hours of setup plus a few days to instrument properly. Self-hosting adds an ongoing operational load that needs a named owner.
Building from scratch is where budgets break: not the initial days, but the standing maintenance that never appears in the original estimate.
Want a second opinion on the decision?
Describe your situation and I will tell you what I would do — including when that is "nothing" or "handle it internally". No charge for that, and it is genuinely how a lot of these conversations end.