The Check Passes. We Still Do Not Believe It.

TL;DR: Most personal data scanners return a boolean. That works while every identifier has a published checksum, and stops working the moment one does not. Two of the identifiers a UK firm asks about most cannot be checked arithmetically at all, so our results carry a confidence rather than a yes: observed when a citable algorithm agreed, asserted when only the shape did. The uncomfortable case is the one where the check runs, passes, and we still report asserted.

Four numbers, four different amounts of certainty

Building the UK profile put four identifiers side by side, and they turn out to sit at four different distances from the truth.

An NHS number is ten digits with a modulus 11 check digit, specified by NHS Digital in a published standard. Run the weights, compare, done. If it matches, arithmetic said so and anyone can check our working.

A sort code and account number can be modulus checked against the published weight table that the payments industry maintains. Same story.

A National Insurance number has no check digit. None. What makes QQ123456C invalid is not arithmetic but convention: which letters may open it, which pairs are never issued, and that the final letter must be A, B, C or D. Every one of those is a rule about shape.

A Unique Taxpayer Reference is the interesting one, and it is where the design decision actually lives.

The check that runs and proves nothing

A UTR has ten digits and a modulus 11 check digit. The weights are 6, 7, 8, 9, 10, 5, 4, 3, 2, applied to the last nine and compared against the first.

Those weights are correct. Every major UK accounting package implements them. They hold against real UTRs. We run them.

HMRC has never published them.

So when the check passes, what do we actually know? That the number is consistent with an algorithm nobody at HMRC has committed to. That is genuinely useful: it removes roughly ten in eleven of the random ten-digit strings that would otherwise be reported as a taxpayer reference, which is a large reduction in noise. It is not the same as establishing a fact, and a compliance officer reading a scan report has no way to tell the two apart unless the report tells them.

So the result says asserted, even when the check passes. The comment in the source says the quiet part:

ASSERTED even when the check PASSES, which looks wrong and is not. A check we cannot cite is a check that reduces false positives without establishing anything. If HMRC ever publishes the algorithm, this becomes OBSERVED and the change is one word.

That last sentence is the test of whether the distinction is real. If it were a hedge, publication would not change anything. It is a claim about provenance, so it does.

The same decision, in the Gulf

The Emirates ID took the identical shape and we made the identical call.

The number is 784-YYYY-NNNNNNN-C, and the check digit is widely believed to be Luhn over the leading fourteen digits. The belief is well tested against real cards. There is no official specification, and published analyses disagree about whether the check covers the whole number or a truncated sequence.

What is actually doing the work here is not the checksum. It is the 784 prefix and a registration year that falls in a plausible range, both of which are structure. So the Emirates ID reports asserted, and a UAE IBAN sitting beside it in the same scan reports observed, because ISO 13616 mod-97 is a published standard.

A reader who sees both in one result learns exactly how much arithmetic stood behind each. A reader who sees two green ticks learns nothing.

A collision we did not expect

Putting UTR and NHS number in the same profile created something new. Both are ten bare digits. Both use modulus 11. They are the first pair in the product to share a pattern and a jurisdiction, which means every ten-digit number in a UK document is tested against both.

We measured the overlap rather than reasoning about it. Over 200,000 random ten-digit strings, each check passes about 9 percent of the time, and 0.82 percent satisfy both. Roughly one ten-digit number in 122 looks simultaneously like a valid taxpayer reference and a valid NHS number.

This is not a bug, and there is no arithmetic that resolves it. A bare ten-digit number genuinely is ambiguous, and the honest behaviour is to report both candidates and let context decide, rather than pick one and sound confident. It matters because the two carry different obligations: an NHS number is UK GDPR Article 9 special category data, and a UTR is not.

We found this by listing every family’s pattern and grouping them, not by thinking hard. Worth doing whenever a profile gains a member.

Then the grid found twenty-one things we had missed

All of the above was covered by unit tests, and the unit tests were green. So was the rest of the suite, all 1,567 of them.

We then ran the adversarial grid, which crosses every identifier against every mutation an attacker or an ordinary document might apply, and it returned 21 defect cells: 14 in the UK profile, 7 in the UAE one, against zero for the mature EU profile. That comparison is what made the number obviously wrong rather than merely a number.

Two of the three causes were not attacks at all. They were ordinary printed forms.

ab123456c. The NINO pattern required uppercase letters. That is how the number appears on a card and not how a person types it into a form.

00-00-0019740417. The sort code and account pattern required a separator between the two. That is how they appear on a statement and not how they appear in a bank file, where they run together.

Both were fixed by loosening a quantifier. Neither was findable by a test that called the verifier function directly with a well-formed string, because the defect was never in the verifier. It was in what reached it.

What we changed about how we work

A feature is not finished when its own tests pass. It is finished when we have asked what its arrival changed about everything already there. Concretely, after this we now check four things before calling a profile done:

Run the adversarial harness, and compare against something mature. An absolute defect count means little. EU at zero next to UK at fourteen means a great deal.

Look for what the new thing silently shrinks. Six new identifier packs shipped with no red-team seed, so all six landed in the untested list and the grid said nothing about them at all. The code was honest about this and the effect was still a denominator quietly getting smaller. Seeds now ship with packs.

Check for collisions with what already exists. That is how the UTR and NHS overlap surfaced.

Re-run everything generated. A build script with a hand-maintained lookup and a silent fallback produced a table row pointing at the wrong page, and nothing complained. It derives the mapping now.

Unit tests check the thing you thought of. The harnesses check the thing you did not.

Why any of this belongs in a governance product

Because the alternative is a number that cannot be acted on.

A scan that reports “47 National Insurance numbers found” invites a response proportional to certainty that does not exist. A scan that reports 47 asserted and 12 observed lets somebody decide where to spend the afternoon, and lets them say, in front of a supervisor, what stood behind each figure.

The UK profile and the UAE profile both ship this way, and both say in the product what they cannot see: names, addresses and dates of birth carry no checksum, so a clean scan never means no personal data.