Actuarial Validation & Filing Automation

Tokenizing Policyholder PII for Filing Pipelines

Tokenizing policyholder PII means replacing direct identifiers such as Social Security numbers and names with deterministic, keyed HMAC tokens before seriatim data enters a validation and filing pipeline, so records can be joined, checked, and submitted without exposing the underlying identity. This page shows the focused technique — a keyed tokenizer plus a secured re-identification map — and sits inside the wider Data Security & PII Boundaries for Filing Systems discipline within Regulatory Architecture & Compliance Mapping.

The Problem

A seriatim extract for a reserve valuation carries millions of policy records, and each one still holds the raw identifiers the source administration system happened to store: an SSN, a policyholder name, sometimes a date of birth. The actuarial work does not need any of them — reserves depend on face amount, issue age, mortality class, and duration, not on who the policyholder is — yet those identifiers ride along through every validation step, every intermediate file, and every log line, widening the blast radius of any leak and pulling the whole pipeline into scope for data-protection review. The naive fix, dropping the identifiers, breaks the one thing they are needed for: joining a record back to its policy across runs and reconciling it against the administration system. Tokenization threads that needle. It swaps each direct identifier for a stable, non-reversible token computed with a keyed hash, so two records for the same policy still collide on the same token and can be joined, but the token reveals nothing about the identity and cannot be reversed without the key. A separately secured map lets an authorised operator re-identify a specific policy when an examiner query demands it, and nothing else does.

A Minimal Working Example

The tokenizer below derives a per-field HMAC-SHA-256 token from a secret key, normalises inputs so trivial formatting differences do not fork the token, and records a secured re-identification entry keyed by token. It runs as-is on the standard library:

from __future__ import annotations

import hashlib
import hmac
import os
import re
from dataclasses import dataclass

# The tokenization key is a secret, loaded from the environment or a key vault.
# It NEVER travels with the tokenized data or the filing package.
TOKEN_KEY: bytes = os.environ.get("PII_TOKEN_KEY", "dev-only-key").encode()

DIRECT_IDENTIFIERS = ("ssn", "policyholder_name")


def _normalise(field: str, value: str) -> str:
    value = value.strip().lower()
    if field == "ssn":
        value = re.sub(r"\D", "", value)  # strip dashes/spaces: 123-45-6789 -> 123456789
    else:
        value = re.sub(r"\s+", " ", value)
    return f"{field}:{value}"


def tokenize(field: str, value: str, key: bytes = TOKEN_KEY) -> str:
    message = _normalise(field, value).encode()
    return hmac.new(key, message, hashlib.sha256).hexdigest()


@dataclass(frozen=True)
class ReidEntry:
    """A secured mapping row: token back to the original identifier."""

    field: str
    token: str
    original_value: str


def tokenize_record(record: dict[str, str]) -> tuple[dict[str, str], list[ReidEntry]]:
    tokenized = dict(record)
    reid_map: list[ReidEntry] = []
    for field in DIRECT_IDENTIFIERS:
        if field not in record:
            continue
        token = tokenize(field, record[field])
        tokenized[field] = token
        reid_map.append(ReidEntry(field=field, token=token, original_value=record[field]))
    return tokenized, reid_map


if __name__ == "__main__":
    seriatim = [
        {"policy_id": "P-0001", "ssn": "123-45-6789",
         "policyholder_name": "Jane A. Doe", "face_amount": "250000"},
        {"policy_id": "P-0002", "ssn": "123 45 6789",
         "policyholder_name": "jane a doe", "face_amount": "100000"},
    ]
    for row in seriatim:
        safe, reid = tokenize_record(row)
        print(safe["policy_id"], safe["ssn"][:12], "...")
    # Both rows above share one SSN in different formats -> identical token.
PII tokenization boundary: raw identifiers to a filing-safe record plus a secured re-identification map A raw seriatim record enters an HMAC tokenizer keyed by a secret in a vault; the tokenizer emits a filing-safe tokenized record for the pipeline and a separately secured re-identification map. Raw record SSN + name + face amount Secret key held in vault HMAC tokenizer normalise + keyed hash deterministic Tokenized record tokens replace PII joinable + safe Re-identification map secured, restricted token to identity Validation + filing pipeline no raw PII in scope
The tokenizer keeps identifiers out of the pipeline while a separately secured map preserves controlled re-identification.

How It Works, Block by Block

The key lives apart from the data. TOKEN_KEY is loaded from the environment or a key vault and is the only secret in the scheme; it never travels with the tokenized records or the filing package. This is what makes the tokens non-reversible in practice — an attacker holding a leaked tokenized extract cannot recover an SSN without also stealing the key, and the key is held under separate, tighter controls than the bulk data. Rotating or revoking access to the key is the lever that governs re-identification for the whole dataset.

Normalisation makes the token stable, not brittle. _normalise lowercases, trims, strips non-digits from an SSN, and collapses internal whitespace in a name before hashing. Without this step, 123-45-6789 and 123 45 6789 — the same SSN in two administration systems — would produce two different tokens and the same policy would fail to join across runs. Prefixing the value with the field name (ssn: or policyholder_name:) domain-separates the tokens so an SSN and a name that happened to share text can never collide into one token.

HMAC, not a bare hash, is the point. hmac.new(key, message, hashlib.sha256) keys the digest with the secret, so the token depends on both the identifier and the key. A plain sha256(ssn) would be trivially reversible: an attacker precomputes the hash of every possible nine-digit SSN — under a billion — and inverts the whole column in seconds. Keying the hash defeats that brute force, because the attacker would have to guess the secret as well, which is why HMAC is the standard construction for this job.

Determinism is what preserves joins. Because the same input and key always yield the same token, the tokenized ssn column is still a valid join key: two records for the same policy collide on one token and reconcile against each other and against the administration system, exactly as the demo’s two differently formatted rows do. The re-identification map is written keyed by token, so an authorised operator answering an examiner query can walk from a token in the filing back to the original identifier — and no unauthorised consumer of the tokenized data can.

Edge Cases and Production Hardening

Deterministic tokens leak equality and frequency. The trade-off for joinability is that anyone with the tokenized data can see which records share a policy and how often each token appears, even without reversing it — a distribution that can enable inference on small or skewed populations. Where a downstream consumer needs a value that must not even reveal equality, use random tokenization for that field instead and store the mapping, accepting that random tokens cannot be recomputed and so cannot be used as a stable join key:

import secrets

def random_token(value: str, vault: dict[str, str]) -> str:
    token = secrets.token_hex(16)   # unpredictable, not derivable from the value
    vault[token] = value            # the ONLY way back is this stored mapping
    return token

Choose deterministic tokens for identifiers you must join on, and random tokens for fields you only ever need to reveal under authorisation.

Key rotation silently forks every token. Because the token is a function of the key, rotating TOKEN_KEY changes every token, and a record tokenized under the old key will no longer join to one tokenized under the new key. Handle rotation as a versioned migration: tag each token with the key version that produced it, keep the retired key available long enough to re-tokenize historical extracts, and never mix key versions inside one join without translating through the map. Schedule rotation windows against the filing calendar so a key never rotates mid-submission — the Automating NAIC Filing Deadline Alerts in Python technique is a natural place to hang that guard.

Join stability breaks when normalisation is incomplete. Any identifier variant the normaliser does not fold — a trailing suffix like “Jr”, a middle initial present in one system and absent in another, a leading zero dropped from an SSN — forks the token and silently splits one policy into two. Define the normalisation contract once, test it against real dirty inputs before it touches production, and record which fields feed the token so the re-identification map and every downstream join agree on the same canonical form. Emit the token generation itself into the tamper-evident trail described in Building Secure Audit Logs for Regulatory Submissions so any re-identification is accountable.

Compliance Note

Tokenization is a concrete expression of the data-minimisation and confidentiality expectations that govern policyholder data. Data-protection regimes — from the safeguarding provisions of the NAIC Insurance Data Security Model Law to broad privacy frameworks — expect that personal identifiers are not processed or retained beyond what a task requires, and an actuarial validation task requires no direct identifiers at all. Keeping raw SSNs and names out of the pipeline shrinks the systems in scope for that review to the single, tightly controlled key store and re-identification map. It also supports the actuary’s professional confidentiality duty under ASOP No. 23 (Data Quality) and the profession’s Code of Professional Conduct, which expect appropriate care with the data underlying an actuarial work product. The re-identification map should be held to the same data security and PII boundary controls as the key: access-logged, encrypted, and disclosed only in response to an authorised regulatory request.

Frequently Asked Questions

Why use HMAC instead of a plain SHA-256 hash for tokenizing an SSN?

A plain hash of an SSN is trivially reversible because the identifier space is tiny — an attacker can hash every possible nine-digit number and invert the whole column. HMAC keys the hash with a secret, so the token depends on both the SSN and a key the attacker does not have, which defeats that brute-force reversal while still producing a stable, deterministic token.

When should tokenization be deterministic versus random?

Use deterministic tokens for any identifier you need to join on across runs or reconcile against a source system, because the same input always yields the same token. Use random tokens for fields you only ever need to reveal under authorisation and that should not expose even equality between records, accepting that random tokens cannot be recomputed and so cannot serve as a join key.

What happens to existing tokens when the tokenization key is rotated?

Rotating the key changes every token, so records tokenized under the old key no longer join to those under the new key. Treat rotation as a versioned migration: tag each token with its key version, retain the retired key long enough to re-tokenize historical data, and avoid rotating mid-submission by aligning rotation windows with the filing calendar.

How does tokenized seriatim data still support policy-level joins?

Because deterministic tokenization is a pure function of the identifier and the key, two records for the same policy collide on the same token even if the raw identifier was formatted differently, provided normalisation folds those formatting differences first. The tokenized identifier therefore remains a valid join key, while a separately secured re-identification map preserves controlled reversal for authorised queries.

Up a level: Data Security & PII Boundaries for Filing Systems · Regulatory Architecture & Compliance Mapping

Regulatory references: NAIC Insurance Data Security Model Law (#668), and Actuarial Standards Board ASOP No. 23 (Data Quality).