Actuarial Validation & Filing Automation

OSFI Model Risk Management Guidelines

OSFI Guideline E-23 sets the supervisory expectation that every model informing a Canadian insurer’s capital, reserving, or pricing decisions is governed across a defined lifecycle — from design and data through independent validation, deployment, and ongoing monitoring — with rigour that scales to each model’s risk rating. This page turns that principles-based expectation into a working subsystem: a code-enforced model inventory, a risk-rating engine, and lifecycle gates that refuse to let an unvalidated or stale model reach production. It sits within the broader Regulatory Architecture & Compliance Mapping discipline and shows the specific Python patterns that make E-23 compliance reproducible rather than aspirational.

The Compliance Problem OSFI E-23 Solves

The failure E-23 guards against is not a single wrong number — it is an ungoverned model reaching a capital calculation without anyone able to prove it was reviewed, by whom, against what data, and when. Under the revised E-23 lifecycle (rationale and design → data → development → validation → deployment → monitoring → decommission), OSFI expects a federally regulated insurer to maintain a complete model inventory, assign each model a risk rating driven by materiality and complexity, subject high-rated models to independent validation before use, and revalidate on a cadence proportionate to that rating. LICAT ties this directly to solvency: a capital model that feeds the Life Insurance Capital Adequacy Test but cannot demonstrate lifecycle governance is a supervisory finding waiting to happen.

The cost of getting this wrong is concrete and asymmetric. A model deployed with a validation sign-off that a spreadsheet claims exists but no immutable record confirms is indistinguishable, during an examination, from a model that was never validated at all. A high-materiality LICAT projection that quietly drifts past its revalidation window becomes an unquantified capital exposure. A validation performed by the model’s own developer violates the independence expectation no matter how thorough it was. Each of these is a control that a manual, document-driven process will eventually miss under filing-deadline pressure. The subsystem described here makes them structurally impossible: a model that has not cleared its gate cannot be promoted, and every gate decision is written to an append-only trail before promotion proceeds.

This is the Canadian counterpart to the US-focused controls in NAIC VM-20 Compliance Frameworks; insurers operating on both sides of the border run a single inventory that carries both regulatory mappings on each record.

Architecture of an E-23-Compliant Validation Subsystem

The subsystem separates three concerns that manual processes routinely conflate: the inventory (what models exist and their attributes), the rating (how much rigour each model warrants), and the gate (whether a model may move to the next lifecycle stage right now). The projection and reserving engines never make promotion decisions themselves — they consume only models the gate has already cleared. Every gate evaluation produces a structured decision that is fingerprinted and appended to the audit trail, so the question “was this model allowed into production, and why?” always has a reproducible answer.

OSFI E-23 lifecycle gate pipeline Left to right, four process stages chain in sequence — the model inventory, independent validation, PII-boundary enforcement, and the append-only audit-trail architecture — into a single gate decision. The gate splits two ways: a model that is not within tolerance is routed to the compliance queue for remediation, and a model that is within tolerance is promoted as filing-ready. Every stage is a hard gate, not a suggestion, and the routing decision is itself a logged event. no yes Model inventory Independent validation PII boundary enforcement Audit trail architecture Within tolerance? Compliance queue Filing-ready

Each stage in this flow is a gate, not a suggestion. A model that fails the independent-validation gate is routed to a compliance queue for remediation rather than silently proceeding, and the routing decision is itself a logged event. This mirrors the deterministic state-machine pattern used by the Assumption Validation & Rule Engine Design engine on the assumption side of the pipeline: inputs are classified, exceptions are routed, and nothing reaches the calculation layer without an attributable record of how it got there.

Prerequisites

Before implementing the gate engine, the following should be in place:

  • Python packages. pydantic>=2 for the model-inventory contract and validators, the standard library enum, datetime, hashlib, and json for lifecycle stages and deterministic fingerprinting, and pytest for the gate regression suite. No third-party dependency is required for the core logic — keeping the compliance-critical path dependency-light is itself a governance choice.
  • Actuarial data contracts. Each model record needs a stable model_id, its business use (e.g. a LICAT capital projection), a materiality and complexity score, a named owner, and a named independent reviewer distinct from the owner. These typed contracts build on the ingestion-side patterns in Schema Validation with Pydantic & Great Expectations.
  • Regulatory context. Familiarity with the E-23 model lifecycle stages and the LICAT capital framework they feed. Readers new to the end-to-end pipeline should start at the Actuarial Model Ingestion & Testing Workflows track, which produces the clean, versioned data this subsystem governs.
  • An audit trail. A hash-chained, append-only store described in Actuarial Audit Trail Architecture, plus the confidentiality controls in Data Security & PII Boundaries for Filing Systems for any record that carries policyholder-linked detail.

Core Implementation: The Model Inventory and Lifecycle Gate

The canonical pattern is a typed model-inventory record, a risk-rating function that scales rigour to materiality and complexity, and a gate that evaluates whether a requested lifecycle stage may open. The inventory contract enforces the independence expectation at construction time — a record whose reviewer equals its owner cannot even be instantiated.

from __future__ import annotations

from datetime import date, timedelta
from enum import Enum
from typing import List, Optional

from pydantic import BaseModel, Field, ValidationInfo, field_validator


class LifecycleStage(str, Enum):
    """Ordered E-23 model lifecycle stages."""
    RATIONALE = "rationale_and_design"
    DATA = "data"
    DEVELOPMENT = "development"
    VALIDATION = "validation"
    DEPLOYMENT = "deployment"
    MONITORING = "monitoring"
    DECOMMISSION = "decommission"


class RiskRating(str, Enum):
    LOW = "low"
    MEDIUM = "medium"
    HIGH = "high"


class ModelRecord(BaseModel):
    """One entry in the OSFI E-23 model inventory."""
    model_id: str
    model_name: str
    business_use: str                       # e.g. "LICAT capital projection"
    valuation_date: date
    materiality_score: float = Field(ge=0.0, le=1.0)
    complexity_score: float = Field(ge=0.0, le=1.0)
    owner: str
    independent_reviewer: str
    regulatory_mapping: List[str] = Field(min_length=1)   # e.g. ["OSFI E-23", "LICAT"]
    last_validated: Optional[date] = None
    approved_stages: List[LifecycleStage] = Field(default_factory=list)

    @field_validator("independent_reviewer")
    @classmethod
    def reviewer_is_independent(cls, v: str, info: ValidationInfo) -> str:
        owner = info.data.get("owner")
        if owner is not None and v.strip().lower() == owner.strip().lower():
            raise ValueError("Independent reviewer must differ from model owner (E-23 validation)")
        return v


# Lifecycle stages evaluated in their canonical order.
STAGE_ORDER: List[LifecycleStage] = list(LifecycleStage)


def rate_model_risk(record: ModelRecord) -> RiskRating:
    """Combine materiality and complexity into an E-23 risk rating.

    OSFI expects the depth of validation and the revalidation cadence to scale
    with the model's risk rating rather than applying one uniform standard.
    """
    score = 0.60 * record.materiality_score + 0.40 * record.complexity_score
    if score >= 0.66:
        return RiskRating.HIGH
    if score >= 0.33:
        return RiskRating.MEDIUM
    return RiskRating.LOW

With the inventory contract and rating in place, the gate itself is a pure function of a record, a requested stage, and the date it is evaluated. It refuses to promote a model past any unapproved prior stage and refuses to let anything reach deployment without a current independent validation.

# Revalidation cadence by risk rating — high-rated models are re-examined most often.
REVALIDATION_INTERVAL = {
    RiskRating.HIGH: timedelta(days=365),     # annual
    RiskRating.MEDIUM: timedelta(days=730),   # biennial
    RiskRating.LOW: timedelta(days=1095),     # triennial
}


def revalidation_due(record: ModelRecord, as_of: date) -> bool:
    """A model with no validation on file, or one past its cadence, is due."""
    if record.last_validated is None:
        return True
    rating = rate_model_risk(record)
    return as_of >= record.last_validated + REVALIDATION_INTERVAL[rating]


class GateDecision(BaseModel):
    model_id: str
    requested_stage: LifecycleStage
    risk_rating: RiskRating
    allowed: bool
    reasons: List[str] = Field(default_factory=list)


def evaluate_stage_gate(
    record: ModelRecord, requested_stage: LifecycleStage, as_of: date
) -> GateDecision:
    reasons: List[str] = []
    idx = STAGE_ORDER.index(requested_stage)

    # Every preceding lifecycle stage must be signed off before this one opens.
    for prior in STAGE_ORDER[:idx]:
        if prior not in record.approved_stages:
            reasons.append(f"Prior stage not approved: {prior.value}")

    # Deployment and everything after it require a current independent validation.
    if requested_stage in (LifecycleStage.DEPLOYMENT, LifecycleStage.MONITORING):
        if LifecycleStage.VALIDATION not in record.approved_stages:
            reasons.append("Independent validation not signed off")
        if revalidation_due(record, as_of):
            reasons.append("Revalidation overdue for current risk rating")

    return GateDecision(
        model_id=record.model_id,
        requested_stage=requested_stage,
        risk_rating=rate_model_risk(record),
        allowed=not reasons,
        reasons=reasons,
    )

The gate returns a structured GateDecision rather than a bare boolean so that every refusal carries its reasons. Those reasons are exactly what a compliance officer needs to route the model to the correct remediation step, and exactly what an examiner expects to see when asking why a model was — or was not — allowed into production.

Configuration and Tuning

Two families of parameter govern how strict the subsystem is, and both belong in version-controlled configuration rather than buried in code, so that a change to supervisory posture is itself an auditable event.

The rating weights and cutoffs decide how materiality and complexity combine into a risk rating. The default weighting leans on materiality (0.60) over complexity (0.40) because a simple model feeding a large capital figure carries more supervisory risk than a complex model feeding an immaterial one. The cutoffs at 0.33 and 0.66 divide the score into three bands. An insurer with a more conservative posture raises the HIGH cutoff downward so that more models attract the strictest cadence.

from dataclasses import dataclass


@dataclass(frozen=True)
class RatingPolicy:
    materiality_weight: float = 0.60
    complexity_weight: float = 0.40
    medium_cutoff: float = 0.33
    high_cutoff: float = 0.66

    def rate(self, materiality_score: float, complexity_score: float) -> RiskRating:
        score = (self.materiality_weight * materiality_score
                 + self.complexity_weight * complexity_score)
        if score >= self.high_cutoff:
            return RiskRating.HIGH
        if score >= self.medium_cutoff:
            return RiskRating.MEDIUM
        return RiskRating.LOW


# Production may tighten the high band so more models fall under annual revalidation.
PROD_RATING_POLICY = RatingPolicy(high_cutoff=0.60)

The revalidation cadence maps each rating to a maximum interval between independent validations. These intervals are the single most examined tuning knob: setting them too loose is the classic path to a stale high-materiality model, while setting them uniformly tight wastes independent-validation capacity on immaterial models. The cadence should be sourced from the insurer’s board-approved model risk policy and pinned in the same configuration file as the rating weights, so that a git history shows exactly when a cadence changed and under whose authority.

E-23 risk-rating matrix and revalidation cadence A square plot with materiality increasing left to right and complexity increasing bottom to top. Two parallel diagonal lines mark the score cutoffs at 0.33 and 0.66, where the score is 0.60 times materiality plus 0.40 times complexity. The bottom-left triangle is the LOW band with a triennial revalidation cadence, the central strip is the MEDIUM band with a biennial cadence, and the top-right triangle is the HIGH band with an annual cadence. Because materiality is weighted more heavily than complexity, the bands lean toward the materiality axis. A legend on the right restates each band's score range and cadence. 0 1 Materiality → 0 1 Complexity → LOW triennial MEDIUM biennial HIGH annual score = 0.60·materiality + 0.40·complexity LOW · triennial score < 0.33 MEDIUM · biennial 0.33 ≤ score < 0.66 HIGH · annual score ≥ 0.66

For high-volume inventories spanning thousands of models, the gate evaluation runs as a vectorized or asynchronous batch rather than one call at a time — the execution substrate is described in Async Batch Processing for Large Models.

Step-by-Step Walkthrough

The following sequence takes a single model from inventory entry to a gated deployment decision using the code above.

  1. Register the model. Construct a ModelRecord with its model_id, business use, materiality and complexity scores, owner, independent reviewer, and regulatory_mapping of at least ["OSFI E-23"] (plus "LICAT" for capital models). Construction fails immediately if the reviewer equals the owner, catching the independence violation at the earliest possible point.
  2. Rate the model. Call rate_model_risk (or RatingPolicy.rate for environment-specific cutoffs) to derive the risk rating that will govern its validation depth and revalidation cadence.
  3. Advance through the early lifecycle stages. As each of rationale-and-design, data, and development is signed off, append the corresponding LifecycleStage to approved_stages. The gate will block any attempt to skip a stage.
  4. Perform and record independent validation. Once the named independent reviewer completes validation, append LifecycleStage.VALIDATION and set last_validated to the sign-off date. This is the pivotal record: no capital model reaches deployment without it.
  5. Request the deployment gate. Call evaluate_stage_gate(record, LifecycleStage.DEPLOYMENT, as_of=today). Inspect GateDecision.allowed; if False, the reasons list names every blocking condition.
  6. Route on the decision. Promote the model only when the gate allows it. Otherwise, route the record and its reasons to the compliance queue for remediation.
  7. Persist the decision to the audit trail. Fingerprint the GateDecision and append it to the immutable log before the promotion takes effect, so the record of the decision cannot post-date the action it authorized.
  8. Monitor and revalidate on cadence. During the monitoring stage, re-run revalidation_due on the inventory so that any model crossing its interval is flagged before, not after, it becomes non-compliant.

Validation and Testing

A control subsystem is only trustworthy if its own behaviour is pinned by tests. The gate engine is pure and deterministic, which makes it straightforward to lock down with a regression suite that an examiner can read as executable evidence of the control.

import hashlib
import json
from datetime import date

import pytest


def _sample(**overrides) -> ModelRecord:
    base = dict(
        model_id="LICAT-CAP-001",
        model_name="LICAT capital projection",
        business_use="LICAT capital projection",
        valuation_date=date(2026, 12, 31),
        materiality_score=0.90,
        complexity_score=0.70,
        owner="a.actuary",
        independent_reviewer="b.reviewer",
        regulatory_mapping=["OSFI E-23", "LICAT"],
    )
    base.update(overrides)
    return ModelRecord(**base)


def test_high_materiality_rates_high():
    assert rate_model_risk(_sample()) is RiskRating.HIGH


def test_reviewer_independence_enforced():
    with pytest.raises(ValueError):
        _sample(independent_reviewer="a.actuary")


def test_deployment_blocked_without_validation():
    record = _sample(approved_stages=[
        LifecycleStage.RATIONALE, LifecycleStage.DATA, LifecycleStage.DEVELOPMENT,
    ])
    decision = evaluate_stage_gate(record, LifecycleStage.DEPLOYMENT, date(2027, 1, 15))
    assert decision.allowed is False
    assert "Independent validation not signed off" in decision.reasons


def test_stale_validation_blocks_deployment():
    record = _sample(
        last_validated=date(2025, 1, 1),                 # over a year old for a HIGH model
        approved_stages=list(LifecycleStage)[:4],        # through VALIDATION
    )
    decision = evaluate_stage_gate(record, LifecycleStage.DEPLOYMENT, date(2027, 1, 15))
    assert "Revalidation overdue for current risk rating" in decision.reasons


def test_clean_model_is_promotable_and_auditable():
    record = _sample(
        last_validated=date(2026, 12, 31),
        approved_stages=list(LifecycleStage)[:4],        # through VALIDATION
    )
    decision = evaluate_stage_gate(record, LifecycleStage.DEPLOYMENT, date(2027, 1, 2))
    assert decision.allowed is True

    # The decision fingerprint is deterministic and ties the audit record to this exact outcome.
    canonical = json.dumps(decision.model_dump(), sort_keys=True, default=str)
    checksum = hashlib.sha256(canonical.encode("utf-8")).hexdigest()
    assert checksum == hashlib.sha256(canonical.encode("utf-8")).hexdigest()

Beyond unit tests, two assertions belong in the pipeline itself. First, an audit-log assertion: every allowed promotion must have a matching GateDecision record in the append-only store whose checksum recomputes from the archived decision — a promotion with no matching record is a hard failure, not a warning. Second, an inventory-completeness assertion that runs on a schedule to confirm no deployed model is missing from the inventory or past its revalidation window, so drift is caught as a routine non-event. Distributional monitoring of model outputs over time — the statistical drift side of the monitoring stage — reuses the threshold machinery in Dynamic Threshold Tuning for Assumption Drift.

Failure Modes and Gotchas

The subsystem removes the obvious failures, but a production deployment still has sharp edges specific to E-23 automation.

  • Lifecycle-stage ordering conflicts. Reordering or renaming a LifecycleStage silently changes what STAGE_ORDER.index returns, so a gate that used to block a skipped stage may start permitting it. Treat the enum order as a versioned contract: pin it in tests and bump a schema version when it changes.
  • Independence enforced only at construction. The reviewer-independence check fires when a ModelRecord is built, but a later in-place mutation of owner bypasses it. Rebuild records through the validated constructor on any change rather than reassigning attributes.
  • Clock and time-zone skew on the cadence. revalidation_due compares against the as_of date the caller passes. A batch job that derives as_of from local wall-clock time near midnight can classify the same model as due or not-due depending on the runner’s time zone. Pass an explicit, UTC-derived valuation date rather than relying on date.today() inside the function.
  • Stale scores freezing the rating. Materiality and complexity are captured once at registration; if a block of business grows materially, a HIGH model can linger as MEDIUM until someone re-scores it. Recompute scores on a schedule and treat a score change that crosses a band boundary as an event that forces revalidation.
  • Confidential attributes leaking into the trail. Gate decisions and inventory records can carry names, business context, and model internals. Route them through the confidentiality controls in Data Security & PII Boundaries for Filing Systems before any record crosses a trust boundary, so the audit trail never becomes an unintended disclosure channel.

Worked end to end for a single model, this gate logic is expanded into a full operational sequence in the Step-by-Step OSFI Model Validation Checklist for Life Insurers, which maps each check above to the artifacts an examiner will request.

Up a level: Regulatory Architecture & Compliance Mapping — the parent guide covering filing automation, audit trails, and cross-jurisdictional compliance mapping.