Telecom & data platforms
Keep the language model out of your hot path
At 520 million records every 15 minutes, per-record inference is off by six orders of magnitude. There are still good places for an agent — just not there.
What I have shipped here
520M
parameters processed every 15 minutes
I architected a configuration-driven telecom data platform processing 200 million configuration and 320 million performance parameters per 15-minute ingestion cycle on Golang, Kafka, Kubernetes, PostgreSQL and Oracle, where new data sources could be onboarded through configuration rather than a deployment.
Read the full case studyWho I work with here
- Telecom and network operations teams with CM/PM data platforms
- Data platform teams being asked to add AI features
- Observability and AIOps products adding agent capabilities
Systems I see in this domain
- Anomaly explanation over network performance metrics
- Pipeline and connector configuration generation
- Alert triage, deduplication and enrichment
- Runbook automation with human gates
- Natural-language querying over analytical stores
What makes agents hard in telecom & data platforms
This is the one domain where my main advice is usually to use less AI than the roadmap assumes, and the arithmetic makes the case better than any argument. At 520 million records per 15-minute cycle you have roughly 1.7 microseconds of budget per record. A model call takes on the order of a second. That is about six orders of magnitude of gap, and no amount of batching or caching closes it.
So per-record inference in the hot path is not expensive, it is arithmetically impossible. The useful question is not whether to add AI, but which surfaces genuinely tolerate a one-second, occasionally-wrong, per-call-billed operation.
The second constraint is a cultural advantage rather than a problem. Data platform teams already think in bounded queues, backpressure, consumer lag and p95. Those are exactly the concepts agent systems are currently missing — so a platform team adopting agents starts ahead, provided nobody tells them to abandon that discipline because "AI is different".
It is not different. An agent run with no step ceiling is an unbounded queue. Retrying a failing tool without backoff is redelivery amplification. No per-run token budget is a pipeline accepting work it cannot afford to finish. I learned all three from a delayed telecom batch that turned a 20% input spike into an outage, and they transfer directly.
Where an LLM belongs in a high-throughput pipeline
The distinction that matters is per-record versus per-aggregate, and synchronous versus offline. Everything workable sits on the aggregate or offline side.
| Surface | Verdict | Why |
|---|---|---|
| Per-record inference in the ingestion path | Never | 1.7 µs of budget per record against ~1 s per model call. Six orders of magnitude — not an optimisation problem. |
| Per-record classification at moderate volume | Only with a small local model | If it must be per-record, it is a fine-tuned classifier or a rules engine, not a hosted LLM call. |
| Anomaly explanation on detected events | Yes — async | Detection stays statistical and fast. The model explains the handful of anomalies that surfaced, off the hot path. |
| Alert triage, dedup and enrichment | Yes | Operates on alerts, not raw records. Volume drops by orders of magnitude and latency tolerance is in seconds. |
| Pipeline and connector config generation | Yes — offline, human-reviewed | This is the highest-value use I have found: generate the configuration, have an engineer approve it, let the deterministic framework execute it. |
| Natural-language querying over the warehouse | Yes — with a query gate | Interactive, low volume, high value. Needs the same AST validation and bounds as any text-to-SQL system. |
| Runbook automation | Yes — human-gated on anything destructive | Diagnosis and proposed remediation from the model; execution behind an approval step with a dry-run. |
| Report and summary generation | Yes — batch | Not latency-sensitive, so it also qualifies for the 50% batch discount. |
The pattern across every "yes" row: the model operates on aggregates or events, off the critical path, and its output is either reviewed by a human or validated by deterministic code before it does anything.
Bounded-resource discipline, translated to agents
The list a data platform team already knows, mapped onto agent systems. If you run pipelines, you have applied all of these — the point is that almost no agent codebase has.
- 1Every queue and buffer is explicitly bounded — an agent run with no maximum step count is an unbounded buffer, which is a delayed crash
- 2Load is shed or rejected rather than accepted when it cannot be completed — refusing work is a valid, observable behaviour; accepting it and dying is not
- 3Retries use backoff and are idempotent, or redelivery amplifies the incident that caused it
- 4Alerting is on trends, not thresholds — cycle time creeping from 9 to 12 minutes is the signal; hitting 15 is the incident
- 5Cost has a per-unit ceiling, not just a monthly budget — a per-run token cap is the agent equivalent of a per-batch memory limit
- 6Consumer lag has an agent analogue: queue depth of pending runs, and it needs the same dashboard
- 7Reprocessing a single entity in isolation is possible — the agent equivalent is replaying one run from its trace
- 8Headroom is maintained deliberately so a spike is absorbed rather than amplified
This transfer is the most useful thing I bring to a platform team adopting agents: not new AI concepts, but permission to apply the operational discipline they already have.
The failure modes that dominate here
Every agent can fail in all nine ways. These are the three that cause the most damage in this domain specifically.
medium · common
Average latency looks fine but some users wait forty seconds
Multi-step agents have long tails by construction: each step adds latency, and retries multiply it.
critical · very common
The agent loops, repeating the same step until something kills it
The agent calls a tool, dislikes the result, and calls it again with near-identical arguments — indefinitely.
high · common
Spend jumped several times over with no corresponding traffic increase
Month-over-month spend multiplies while usage looks flat.
How I would approach it
The engagements that apply most often in this domain. Each has the full method and effort estimates.
LLM cost reduction
Usually 40–70% recoverable, and the levers are ranked.
A materially lower bill, with evidence that quality held.
The methodLLM observability & tracing
One trace ID that explains the whole run.
Any bad run can be pulled up, replayed, and explained in minutes.
The methodText-to-SQL reliability
The hard part is not generating SQL. It is knowing when not to run it.
Answers users trust, because every number traces back to a query they can inspect.
The methodQuestions from this domain
Can we run an LLM on streaming data in real time?
Not per record at high volume. At 520 million records per 15 minutes the budget is about 1.7 microseconds per record, against roughly a second for a model call — a gap of six orders of magnitude that batching does not close.
What works is running the model on aggregates or detected events instead: statistical detection stays in the hot path, and the model explains the small number of things that surfaced.
What is the highest-value use of AI in a data platform?
Configuration generation with human review, in my experience. Onboarding a new data source means writing connector, transformation and destination config — tedious, pattern-heavy work that a model does well and an engineer can verify in minutes.
The execution stays deterministic. You get the speed-up on the authoring step without putting a probabilistic system in the processing path.
Does agent tracing add too much overhead for high-volume systems?
Not where agents belong. Agent surfaces in a data platform run on aggregates and events — thousands of operations, not hundreds of millions — so per-span overhead of a few milliseconds is irrelevant.
The systems where tracing overhead genuinely matters are the ones that should not have an LLM in them anyway.
Our team runs pipelines, not AI. What actually transfers?
Nearly all of your operational instincts, and that is a bigger advantage than it sounds. Bounded queues, backpressure, load shedding, idempotent retries, alerting on trends, maintaining headroom — every one has a direct agent equivalent, and agent codebases routinely lack all of them.
The gap is usually not AI knowledge. It is that nobody told you the discipline you already have is the missing piece.
Building this in telecom & data platforms?
Tell me what your system does and what worries you. I have shipped in this domain, so I can usually name the specific constraint you are about to hit — and that answer is free.
Other domains