Step-by-Step OSFI Model Validation Checklist for Life Insurers

A Canadian life insurer cannot promote a reserving or capital model into a LICAT filing on the strength of a spreadsheet that claims it was reviewed — OSFI Guideline E-23 expects each model to clear a defined lifecycle gate, with rigour scaled to its risk rating, and to leave an attributable record of how it passed. This page turns that expectation into a runnable pre-filing checklist: six gates that map directly onto the E-23 lifecycle and refuse to mark a model filing-ready until every one returns pass. It is the concrete, code-first companion to the OSFI Model Risk Management Guidelines subsystem and sits inside the broader Regulatory Architecture & Compliance Mapping discipline that binds statutory text to production pipelines.

The Six Gates as a Runnable Checklist #

The mistake most manual checklists make is living in a document, where a gate is a box someone remembers to tick under deadline pressure. E-23 frames model governance as a lifecycle — rationale and design → data → development → validation → deployment → monitoring → decommission — and the supervisory expectation is that a high-rated model cannot reach deployment without independent validation on record. Encoding the checklist as executable gates makes that structurally true: the checklist itself decides filing_ready, and a model that has not cleared every gate is never promoted. The following snippet is the whole engine, runnable as-is with no framework dependencies:

from __future__ import annotations

from dataclasses import dataclass, field
from datetime import date, datetime, timezone
from enum import Enum
from typing import Callable


class GateStatus(str, Enum):
 PASS = "pass"
 FAIL = "fail"


@dataclass(frozen=True)
class ModelRecord:
 """One in-force model as it appears in the E-23 inventory."""

 model_id: str
 product_line: str
 risk_rating: str              # "high" | "medium" | "low", per E-23 materiality
 valuation_date: date
 last_validated: date | None
 independent_validator: str | None
 pii_scrubbed: bool
 reserve_adequacy_ratio: float
 audit_chain_sealed: bool


def within_revalidation_window(m: ModelRecord, today: date) -> bool:
 # Revalidation cadence scales with the model's risk rating.
 max_age_days = {"high": 365, "medium": 730, "low": 1095}[m.risk_rating]
 if m.last_validated is None:
     return False
 return (today - m.last_validated).days <= max_age_days


# Each gate maps one E-23 lifecycle expectation to a boolean check.
CHECKLIST: dict[str, Callable[[ModelRecord, date], bool]] = {
 "E-23 inventory record complete": lambda m, _: bool(m.model_id and m.product_line),
 "Independent validation performed": lambda m, _: (
     m.independent_validator is not None
     and m.independent_validator != m.model_id
 ),
 "Within risk-rated revalidation window": within_revalidation_window,
 "PII boundary enforced before run": lambda m, _: m.pii_scrubbed,
 "Reserve adequacy within tolerance": lambda m, _: m.reserve_adequacy_ratio >= 1.0,
 "Audit trail sealed": lambda m, _: m.audit_chain_sealed,
}


@dataclass
class ChecklistResult:
 model_id: str
 evaluated_at_utc: str
 results: dict[str, GateStatus] = field(default_factory=dict)

 @property
 def filing_ready(self) -> bool:
     return all(status is GateStatus.PASS for status in self.results.values())

 @property
 def failed_gates(self) -> list[str]:
     return [name for name, s in self.results.items() if s is GateStatus.FAIL]


def run_checklist(m: ModelRecord, today: date) -> ChecklistResult:
 result = ChecklistResult(
     model_id=m.model_id,
     evaluated_at_utc=datetime.now(timezone.utc).isoformat(),
 )
 for name, check in CHECKLIST.items():
     result.results[name] = GateStatus.PASS if check(m, today) else GateStatus.FAIL
 return result


if __name__ == "__main__":
 model = ModelRecord(
     model_id="LICAT-TERM-2024",
     product_line="term_life",
     risk_rating="high",
     valuation_date=date(2024, 12, 31),
     last_validated=date(2024, 6, 30),
     independent_validator="corporate_actuarial_mrm",
     pii_scrubbed=True,
     reserve_adequacy_ratio=1.03,
     audit_chain_sealed=True,
 )
 outcome = run_checklist(model, today=date(2024, 12, 31))
 print(outcome.filing_ready, outcome.failed_gates)

How the Checklist Engine Works, Gate by Gate #

ModelRecord is the E-23 inventory row, frozen. Every field is a governance attribute an examiner asks about: the risk rating that drives cadence, the identity of the independent validator, whether the run was scrubbed of policyholder identifiers, and whether the audit chain was sealed. Freezing the record means a gate cannot mutate the very state it is judging.

The independent-validation gate encodes E-23’s independence expectation as code. Returning fail when independent_validator is None — or when it equals the model_id, a proxy for the model’s own developer signing off — turns “who validated this?” from an unenforced convention into a blocking condition. A thorough review performed by the model owner still fails the gate, which is exactly the E-23 posture.

within_revalidation_window makes staleness a risk-rated function, not a fixed rule. A high-materiality LICAT projection is granted a 365-day window; lower-rated models age more slowly. A model whose last_validated has drifted past its rating’s window becomes an unquantified capital exposure, and the gate catches it before the filing does. This is the same proportionality principle the Assumption Validation & Rule Engine Design engine applies when it tightens tolerances on the assumptions feeding those models.

filing_ready is the circuit breaker. Because readiness is defined as every gate passing, adding a seventh gate later needs no change to the promotion logic — the aggregate is computed, not maintained by hand. failed_gates gives the compliance queue a precise remediation list rather than a bare “rejected.”

Step-by-Step Walkthrough #

Each numbered step below corresponds to one gate in the engine and the lifecycle stage it defends.

1. Establish regulatory architecture and compliance mapping. Inventory every in-force life model, categorise it by product line, valuation purpose, and capital sensitivity, and assign each a documented governance charter with owner, version control, and change-management thresholds. This is the E-23 inventory record complete gate: a model with no product line or owner cannot be reasoned about, so it fails closed. Codify the mapping as configuration-driven rules rather than hardcoded logic so revised supervisory guidance is a data change, not a code rewrite.

2. Enforce data security and PII boundaries before any run. Life models process mortality tables, health indicators, and policyholder financials; none of that may persist into the validation environment in raw form. The PII boundary enforced before run gate is satisfied only after seriatim inputs pass through the Data Security & PII Boundaries for Filing Systems layer — deterministic hashing of identifiers and masking of demographic fields prior to model execution:

import hashlib
import pandas as pd


def sanitize_pii(policy_frame: pd.DataFrame) -> pd.DataFrame:
    clean = policy_frame.copy()
    clean["policy_id_hash"] = clean["policy_id"].apply(
        lambda pid: hashlib.sha256(str(pid).encode()).hexdigest()
    )
    clean.drop(
        columns=["policy_id", "sin", "full_name"],
        inplace=True,
        errors="ignore",
    )
    return clean

3. Execute core actuarial logic and cross-jurisdictional calibration. Verify assumption reasonableness, scenario coverage, and output stability against the current economic environment. Many insurers run reinsurance treaties that intersect US statutory frameworks, so where a model feeds both regimes, reconcile its output against NAIC VM-20 Compliance Frameworks on a parallel track and flag deviations beyond a tolerance band (for example ±0.5% on net premium reserves). The reserve_adequacy_ratio feeding the fifth gate is produced here, from scenarios generated by Stochastic Scenario Generation Frameworks and reconciled through economic scenario mapping and yield-curve alignment so the LICAT discount path is defensible.

4. Construct the actuarial audit trail. OSFI expects complete reproducibility of every model run. Extend structured logging to bind each input parameter, code version, seed, and output artifact to a validation checkpoint, then seal the chain so the audit trail sealed gate can pass. The full tamper-evident ledger lives in Actuarial Audit Trail Architecture; a minimal checkpoint logger looks like this:

import json
import logging
from datetime import datetime, timezone


class AuditLogger:
    def __init__(self, run_id: str):
        self.run_id = run_id
        self.logger = logging.getLogger(f"validation_{run_id}")
        self.logger.setLevel(logging.INFO)
        handler = logging.FileHandler(f"audit_{run_id}.jsonl")
        handler.setFormatter(logging.Formatter("%(message)s"))
        self.logger.addHandler(handler)

    def log_checkpoint(self, stage: str, status: str, metadata: dict) -> None:
        entry = {
            "timestamp": datetime.now(timezone.utc).isoformat(),
            "run_id": self.run_id,
            "stage": stage,
            "status": status,
            "metadata": metadata,
        }
        self.logger.info(json.dumps(entry))

Audit logs must be immutable, cryptographically hashed, and retained long enough to satisfy OSFI record-keeping expectations.

5. Route failed regulatory syncs to a fallback path. Automated filing systems hit transient failures — network timeouts, gateway rate limits, schema rejections — and a hard failure must never corrupt validation state or duplicate a submission. Pair exponential backoff with a dead-letter queue for unrecoverable payloads, and escalate on a timer using the pattern in Automating NAIC Filing Deadline Alerts in Python:

import json
from datetime import datetime, timezone

import requests
from tenacity import retry, stop_after_attempt, wait_exponential


@retry(stop=stop_after_attempt(5), wait=wait_exponential(multiplier=1, min=2, max=30))
def submit_to_regulatory_gateway(payload: dict, endpoint: str) -> dict:
    response = requests.post(endpoint, json=payload, timeout=15)
    response.raise_for_status()
    return response.json()


def route_to_dlq(payload: dict, error: Exception, dlq_path: str = "dlq.jsonl") -> None:
    """Persist an unrecoverable payload for manual actuarial review."""
    entry = {
        "timestamp": datetime.now(timezone.utc).isoformat(),
        "run_id": payload.get("run_id"),
        "error": str(error),
        "payload": payload,
    }
    with open(dlq_path, "a") as handle:
        handle.write(json.dumps(entry) + "\n")

Preserve payload ordering, carry an idempotency key on every submission, and generate a formal deviation report if a sync failure persists beyond your escalation threshold.

6. Surface the results on a compliance dashboard. The checklist only creates value when its output is visible. Aggregate filing_ready, the failed_gates list, revalidation-window countdowns, and PII-boundary alerts into role-based views: assumption-drift charts for actuaries, submission-status trackers for compliance officers, pipeline latency for developers. This converts validation from a night-before-the-deadline scramble into a monitored, proactive control.

Edge Cases and Production Hardening #

A renamed inventory field silently disables a gate. If an upstream system renames pii_scrubbed to pii_masked but the ModelRecord is populated from a loose dictionary, the missing attribute defaults to falsy and the gate fails for the wrong reason — or worse, a permissive loader fills a default and it passes. Construct the record behind Pydantic schema enforcement so an inventory row with unexpected or missing keys is rejected at the boundary, not silently coerced.

Seed non-determinism makes the audit trail attest to a number the model no longer produces. If the projection drawing the reserve_adequacy_ratio uses a globally shared generator, a re-run yields a different ratio and the sealed checkpoint no longer reconstructs. Isolate the generator per reserve group and pin its seed, and record the seed actually used inside the audit checkpoint so any run replays bit-for-bit.

A borderline revalidation window flips between two filing dates. A high-rated model validated 364 days before one valuation date is 366 days stale at the next, and a checklist run pinned to date.today() rather than the valuation_date will disagree with itself across a quarter boundary. Always evaluate run_checklist against the filing’s valuation_date, not wall-clock time, so the gate decision is reproducible from the sealed record. Where drift is the concern, feed the same countdown into dynamic threshold tuning for assumption drift so a model approaching its window is flagged before it lapses.

Compliance Note #

This checklist is the mechanical evidence that OSFI Guideline E-23 (Model Risk Management) expects across its lifecycle: an inventory record, independent validation before deployment, risk-rated revalidation cadence, and a reproducible audit trail. The reserve_adequacy_ratio gate ties directly to the Life Insurance Capital Adequacy Test (LICAT), where a capital model that cannot demonstrate lifecycle governance is a supervisory finding regardless of the number it produces. For carriers filing on both sides of the border, run a single inventory whose records carry both the E-23 mapping and the NAIC VM-20 Compliance Frameworks mapping, so one validated model satisfies the independent-validation expectation of both regimes without a second, divergent pipeline — and document the whole gate set in line with the modelling standards (ASOP No. 56 on the US side, CIA guidance on the Canadian side) your appointed actuary signs against.

Related Guides #

• OSFI Model Risk Management Guidelines — the E-23 inventory, risk-rating engine, and lifecycle gates this checklist operationalises.
• Actuarial Audit Trail Architecture — the tamper-evident ledger the audit-trail gate depends on.
• Data Security & PII Boundaries for Filing Systems — the tokenisation layer that satisfies the PII gate.
• NAIC VM-20 Compliance Frameworks — the US-side reserve mapping for cross-jurisdictional carriers.
• Automating NAIC Filing Deadline Alerts in Python — escalating a failed sync with time to remediate.

Up a level: OSFI Model Risk Management Guidelines · Regulatory Architecture & Compliance Mapping

---

Regulatory references: OSFI Guideline E-23 (Model Risk Management) and the Life Insurance Capital Adequacy Test (LICAT); NAIC Valuation Manual VM-20; Actuarial Standards Board ASOP No. 56 (Modeling).