Skip to content

Recruitment & ATS

Candidate-facing AI has to be defensible, not just accurate

If the same candidate scores differently on a re-run, you cannot defend the decision — and from August 2026 you may have to.

Talk about your systemor email instead

What I have shipped here

B2B + B2C

recruitment platform with AI matching in production

I built an enterprise recruitment platform with an AI-powered candidate and job matching engine using NLP, vector embeddings and similarity scoring on AWS Bedrock, alongside Elasticsearch for high-performance search, RChilli resume parsing, Twilio candidate communication, and synchronisation with third-party ATS platforms.

Who I work with here

  • Recruitment platforms and job marketplaces with AI matching
  • ATS vendors adding screening, ranking or sourcing agents
  • Talent-acquisition teams running AI screening in-house

Systems I see in this domain

  • Candidate–job matching and ranking engines
  • Resume parsing and enrichment pipelines
  • Automated screening and shortlisting
  • Sourcing and outreach agents
  • Interview scheduling and scoring assistants

What makes agents hard in recruitment & ats

In most domains a non-deterministic answer is an annoyance you paper over with caching. In recruitment it is a defect you cannot defend. If a candidate asks why they were rejected, or a regulator asks you to reproduce a ranking, "the model gives slightly different scores each run" is not an acceptable answer.

That single property reshapes the architecture. Ranking has to be reproducible from a recorded input, a pinned model version and a recorded feature set. Which means version pinning, decision logging and structured output stop being nice engineering hygiene and become the product's legal foundation.

The second constraint is that ranking is the product. In an HR analytics tool a wrong answer is a bad report. In recruitment a wrong ranking changes someone's employment outcome, and "the model decided" is not a defence anywhere.

Third: ATS integration means your failures leave your system. A truncated candidate list or a hallucinated status write-back lands in someone else's system of record, where you cannot see it and your customer's recruiters treat it as fact.

Controls candidate-facing AI needs

Engineering controls, not policy statements. Each one is something you can point at in code or a log — which is what makes a decision defensible rather than merely well-intentioned.

  1. 1Ranking is reproducible: the same candidate, same job, same model version and same feature set produces the same score, and you can prove it by re-running
  2. 2Model version is pinned per decision and recorded with it — never a floating alias that changes underneath a completed hiring round
  3. 3Protected characteristics are not features, and neither are their proxies: postcode, school and university, name-derived ethnicity or gender inference, photo, age-revealing dates, career-gap length
  4. 4Free-text resume content is checked for proxy leakage — a model reading "maternity leave 2023" or a fraternity name has inferred a protected characteristic whether or not you gave it a field
  5. 5Every ranking decision logs its inputs, the retrieved evidence, the score, and the reasoning in a form a human can read months later
  6. 6Adverse decisions have a defined human review path, and the log records what the reviewer actually saw — not merely that someone approved
  7. 7Output is structured, never prose: scores, reasons and flags as fields, so decisions are comparable across candidates and time
  8. 8Score distributions are monitored for drift across demographic groups where you lawfully hold that data for audit purposes, separated from the data used to make decisions
  9. 9Truncation is treated as an error: a shortlist cut off by a token limit is a silently wrong hiring decision
  10. 10Write-backs to an ATS are validated and idempotent, because a hallucinated status is now the customer's system of record
  11. 11Candidates receive the notice your jurisdiction requires, and the notice text is versioned like code
  12. 12Your eval suite contains candidates who must rank consistently, and adversarial resumes with prompt-injection attempts in the text

Item one is the one teams skip and the one that matters most. Run the same 50 candidates through your ranker twice and diff the order. If it changes, you have a defensibility problem today, regardless of what any regulator does next.

Where wrong hiring recommendations come from

Ranked by how often I find each one. Note how few are about the model.

CauseWhat it looks likeThe fix
Parsing lossA skill exists in the resume and never reaches the model — tables, columns, headers and PDFs defeat parsers routinelyMeasure parse recall against a labelled set before touching the ranker
Proxy leakage in free textModel infers a protected characteristic from context you never intended as a featureRedact or neutralise before embedding; test with matched resumes differing only in a proxy
Semantic similarity mistaken for fitA resume that talks like the job description outranks one that can do the jobScore against explicit requirements, not description similarity; keep requirements structured
Non-determinism across runsSame candidate, different rank, no explanation availablePin versions, structure output, assert consistency in evals
Silent truncation of the candidate listShortlist cut at a token limit; the omitted candidates are invisibleTreat a length-based stop reason as an error, and paginate rather than raising the cap
Stale enrichment dataRanking on a job history that is two years out of dateTrack freshness per record and exclude or flag stale enrichment

Parsing loss is first for a reason. Teams tune ranking prompts for weeks when the relevant skill never made it into the context — measure the pipeline before the model.

What the rules imply for how you build

Not a compliance summary — the engineering consequence of each obligation.

EU AI Act — Annex III, point 4 (recruitment and candidate screening)

High-risk obligations apply from 2 August 2026

AI that screens CVs, ranks applicants, filters candidate pools, shortlists for human review, or determines whether a candidate advances is explicitly high-risk. Providers carry design, documentation and bias-testing duties; deployers owe human oversight, input-data quality, logging and candidate information. Penalties under Article 99 reach €15 million or 3% of global turnover.

What it means for the build: Per-decision logging, reproducibility and a documented human-oversight path become build requirements. A ranker that cannot reproduce a past decision cannot demonstrate compliance. Note that the Commission's November 2025 Digital Omnibus proposal may move this date, but it is not enacted law — building for the current date is the conservative call.

NYC Local Law 144 — automated employment decision tools

In force since July 2023

Automated employment decision tools used for hiring or promotion of NYC candidates require an annual independent bias audit, publication of audit results, and advance notice to candidates.

What it means for the build: You need a data pipeline that can produce audit-ready selection-rate data by category, kept separate from the data used to make decisions. Retrofitting this is far harder than logging it from the start.

Regulatory notes on these pages are engineering guidance, not legal advice. They describe what the rules imply for how you build and log a system. Get a qualified opinion for your jurisdiction and use case before relying on any of it.

The failure modes that dominate here

Every agent can fail in all nine ways. These are the three that cause the most damage in this domain specifically.

medium · common

The same question gives different answers each time

Some variation is inherent to sampling.

high · common

Output is cut off mid-way and nothing reports an error

The model hits its output limit and stops.

high · very common

The answer is wrong because retrieval returned the wrong context

The model is behaving correctly given what it was handed — and what it was handed was wrong.

How I would approach it

The engagements that apply most often in this domain. Each has the full method and effort estimates.

Agent eval suites

Stop shipping prompt changes as unmeasured bets.

A suite your team owns that fails the build when agent quality drops.

The method

LLM observability & tracing

One trace ID that explains the whole run.

Any bad run can be pulled up, replayed, and explained in minutes.

The method

Multi-tenant AI isolation

A prompt is not an access-control mechanism.

Isolation that holds even when the model behaves unexpectedly.

The method

Questions from this domain

Is our AI recruitment tool high-risk under the EU AI Act?

If it screens CVs, ranks or filters applicants, shortlists candidates, or decides who advances, then yes — that is explicitly Annex III point 4, and high-risk obligations apply from 2 August 2026.

Whether you are a provider or a deployer changes which duties you carry, and both sets have engineering consequences: logging, human oversight, data quality and reproducibility. This is engineering guidance, not legal advice.

How do we make AI candidate ranking explainable?

Score against explicit structured requirements rather than similarity to the job description, return reasons as fields rather than prose, and record the inputs, model version, retrieved evidence and score for every decision.

Explainability is mostly an architecture property, not a post-hoc feature. If ranking is a similarity number out of an embedding comparison, no amount of generated explanation makes it defensible — the explanation is a story about a number, not the reason for it.

Is it enough to remove protected characteristics from candidate data?

No, and this is the most common misconception. Models infer protected characteristics from proxies: postcode, school, university, name, photo, career-gap length, and language in free text like "maternity leave 2023".

Removing the explicit fields while embedding the full resume text leaves every proxy in place. Test it directly: matched resumes differing only in a proxy should score the same. Most systems have never been tested that way.

Why does non-determinism matter more in recruitment than elsewhere?

Because you may have to reproduce a decision. A candidate appeal, a customer dispute or a regulator's request all assume the ranking can be re-derived — and "the score varies by a few points each run" ends that conversation badly.

It is also a straightforward test you can run today: put 50 candidates through your ranker twice and diff the order. If it changes, fix that before adding features.

Building this in recruitment & ats?

Tell me what your system does and what worries you. I have shipped in this domain, so I can usually name the specific constraint you are about to hit — and that answer is free.

Message meBook a 20-minute callor email instead