Deckname has to find the personal data in a document before it can replace it. In German legal text, that is harder than it sounds, and the reason is a grammar rule most people never think about.

Here is the whole problem in one line. German capitalizes every noun, so capitalization tells you almost nothing about whether a word is a name. The base models miss exactly the cases that matter, custom recognizers fix most of it, and an eval harness with blocking CI gates keeps the result honest.

The capitalization problem

In English, a capital letter in the middle of a sentence is a hint. “I met Baker” is probably a person. German removes that hint. “Bauer”, “Vogel”, and “Koch” are ordinary words (farmer, bird, cook) and common surnames, spelled the same, capitalized either way. Capitalization is not a signal. It is noise.

The base stack is Microsoft Presidio with spaCy (de_core_news_lg). With no custom work, the first eval run told the whole story. Email and IBAN sat at 1.0, because Presidio ships solid patterns for those. Every German-specific type (case numbers, tax IDs, social-security numbers, license plates, addresses) sat at 0.0, because the recognizers did not exist yet. German person recall started at 0.86, missing exactly the predicted hard cases: surnames that are nouns, and lowercase names inside email addresses.

Recognizers that encode the domain

The fix was not a smarter model. It was encoding how German legal text is actually written. A few recognizers I was happy to get right:

  • A tax ID with a real ISO 7064 check digit. The algorithm matched the official example value on the first try, which almost never happens.
  • A social-security number with a birthdate plausibility check built into the format.
  • A context-aware date of birth. A bare date is left alone, or every filing deadline becomes a false hit. A date in the right context is flagged.

The strongest person recognizers came from legal conventions, not from the model:

  • Parties around the versus sign ./., where spaCy reads a bare surname as a location.
  • A name before a bracketed contract role, like “Gerlinde A. (Darlehensgeberin)”, with a postcode look-behind so it does not swallow an address.
  • Lowercase names in sign-offs, like “danke, fabian”.

The bugs were the lesson

Two of them still make me smile.

Presidio compiles its pattern regexes with IGNORECASE by default. My salutation recognizer turned “Herr Bauer” into a person. With IGNORECASE, it also matched “Herr über seine Daten” (master of his own data). Setting the flags explicitly, without IGNORECASE, fixed it, and the unit test had already caught it.

The second: “Uerdinger Straße 214” would not match while glued forms did. My address pattern required at least one character before the “Straße” suffix, so a standalone “Straße” could never match. Glued versus standalone suffixes are a classic German-address trap.

Eval before features

A detection tool is only as trustworthy as the evidence that it detects. So the eval came before most of the features.

The dataset is 40 synthetic documents with about 150 annotated entities. Synthetic on purpose: no real personal data goes into a repository for a privacy product. The documents are deliberately mean. Genitive forms (“Wexlers”). Noun-surnames. Lowercase emails. IBANs with and without spaces. Three date formats. One emoji-only document as a UTF-16 offset test.

The best decision in the harness was to never count offsets by hand. Gold spans are generated from inline markers in the source:

In der Sache [[PERSON|Jonas Wexler]] gegen die [[ORG|Beispiel GmbH]] ...

A script turns those markers into exact character offsets. Hand-counting UTF-16 offsets across 40 documents would have been a festival of off-by-one errors, and every one would have looked like a detection bug.

Four gates run in CI and block the build: structured recall at least 0.98, German person recall at least 0.95, overall German recall at least 0.95, and precision at least 0.80. The baseline sat well below all four (structured recall 0.636). That was the point. The gates describe where the product has to be, not where it starts. By the end of the core milestone all four were green (structured recall 1.0, German person recall 1.0, overall German 0.992, precision 0.833), and the eval job went from advisory to blocking.

Two rules keep it honest. If a change lowers a gated metric, stop and report, never relax the threshold. And if the model is right and the gold is wrong, fix the gold toward more truth: when spaCy correctly found salutation names I had not annotated, I added them (156 instead of 150) and took the higher bar.

A score scale is a contract

Switching the harness to run against the full backend, rather than the detection service alone, exposed a flaw a narrower test would have hidden. Address recall dropped from 1.0 to 0.0. The merge step resolves overlapping spans by score. spaCy entities came in at 0.85, my pattern recognizers at 0.70 to 0.75, so location fragments were beating whole addresses.

The lesson: once you resolve conflicts by score, the score scale is a contract. Pattern matches now score 0.9, statistical NER scores 0.85, and backend regex wins by source priority.

None of the final quality came from a general model getting smarter. It came from writing down how German legal documents actually read, and refusing to let the numbers drift.