Actuarial Validation & Filing Automation

SR 11-7 Model Risk Governance

SR 11-7 is the Federal Reserve and OCC supervisory guidance that treats every actuarial model as a source of risk in its own right, and it demands three things of a life insurer or bank that relies on those models: conceptually sound development, ongoing monitoring in production, and outcomes analysis that back-tests results against reality — all held together by an independent validation function empowered to deliver genuine effective challenge. This guide explains how each element of the SR 11-7 framework maps onto concrete machinery in an actuarial validation system, sitting within the broader Regulatory Architecture & Compliance Mapping discipline and relating directly to its sibling regimes, the OSFI Model Risk Management Guidelines and the NAIC Valuation Manual.

The SR 11-7 model-risk lifecycle: develop, validate, inventory, monitor, and analyse outcomes in a closed loop Five boxes left to right — Development for conceptual soundness, independent Validation delivering effective challenge, Inventory and risk-tiering, ongoing Monitoring, and Outcomes analysis. A feedback arrow runs from Outcomes analysis back to Development, closing the loop. outcomes analysis feeds back into development Development conceptual soundness Validation effective challenge Inventory register & risk-tier Monitor ongoing performance Outcomes back-testing analysis

The Problem

When the Federal Reserve issued SR 11-7 in April 2011 (jointly with OCC Bulletin 2011-12), it named a hazard that had been invisible on most balance sheets: the model itself. A reserve figure, a capital number, or a pricing rate is only as trustworthy as the model that produced it, and a model can be wrong in ways that never surface until a stress event exposes them — a fundamental error in the mathematics, or a correct model used for a purpose it was never designed to serve. SR 11-7 calls the potential for adverse consequences from decisions based on incorrect or misused model output “model risk,” and it insists that this risk be managed with the same rigour as credit or market risk.

For an actuarial organisation the stakes are acute. A principle-based reserve model under the NAIC Valuation Manual runs a million-policy in-force block across a thousand economic scenarios; an IFRS 17 measurement model rolls forward a contractual service margin across cohorts; a capital model feeds a solvency ratio a regulator will scrutinise. Each is exactly the kind of complex, high-materiality model SR 11-7 was written to govern. The guidance does not tell you which mortality table to select or how to discount a liability. It tells you that whatever model you build must be developed on a sound conceptual footing, validated by someone other than its author, catalogued in an inventory that survives staff turnover, monitored while it runs, and back-tested against emerging experience — and that the whole apparatus must be documented well enough that a supervisor, or your own board, can see it working. The engineering problem is turning that governance narrative into a system that enforces it rather than a binder that describes it.

Architecture

An actuarial validation system operationalises SR 11-7 as a closed lifecycle rather than a linear checklist, because the guidance’s own logic is circular: outcomes analysis on a live model produces findings that flow back into development, and the loop begins again. The diagram above traces that circuit. A model is developed against a conceptual-soundness standard; it passes through independent validation whose defining feature is effective challenge; it is entered into a risk-tiered inventory that determines how intensely it will be scrutinised thereafter; it is monitored in production; and its results are subjected to outcomes analysis whose findings re-enter development.

SR 11-7 frames the validation activity itself around three core elements, and a well-built system keeps them structurally distinct rather than collapsing them into one review. The first is an evaluation of conceptual soundness — a critical assessment of the model’s design, the quality of the theory and logic behind it, and the appropriateness of its assumptions and data for the intended use. The second is ongoing monitoring, which confirms the model continues to perform as intended once assumptions, portfolios, and market conditions drift away from the state in which it was built. The third is outcomes analysis, the comparison of model output to actual outcomes, of which back-testing is the most familiar form.

Around those three elements sit two governance structures the guidance treats as non-negotiable. One is the model inventory: a comprehensive, current record of every model in use, under development, or recently retired, carrying enough information for management to understand the firm’s aggregate model risk. The other is risk tiering — a materiality-and-complexity rating attached to each inventory entry that lets an organisation apply the most demanding validation to the models that can do the most damage, and a proportionate touch to the rest. The inventory is where all of this becomes tractable at scale, which is why building a model inventory for SR 11-7 is the first thing a validation programme should stand up. Both structures are underpinned by an actuarial audit trail architecture that records who validated what, when, and against which version of the assumptions.

Prerequisites

Before wiring SR 11-7 governance into a validation platform, expect the following to be in place:

  • Python 3.11+, for the dataclass and typing features used throughout the reference code, and pydantic v2 for typed inventory records.
  • A model registry — even a single append-only table — keyed by a stable model_id, so that inventory, tier, and validation history attach to a durable identity rather than a file path.
  • An independent validation function organisationally separate from model development. SR 11-7 is explicit that effective challenge depends on the challenger’s competence, influence, and incentives; a validation team that reports to the developers it reviews cannot supply it.
  • A running assumption-governance layer, because ongoing monitoring watches the same assumption drift that an Assumption Validation & Rule Engine Design system detects and thresholds. Monitoring metrics are only meaningful against a governed baseline.
  • A regulatory frame: familiarity with SR 11-7’s three-element validation structure, and awareness of how its sibling regimes — OSFI E-23 in Canada and the NAIC Valuation Manual’s model-governance expectations in the United States — express the same principles in different vocabulary.

Core Implementation

The heart of an SR 11-7 system is an orchestrator that runs the three validation elements against a model and returns a verdict that is either fit to rely on or carries blocking findings. The pattern below models each element as a discrete step, records the effective-challenge findings each raises, and derives an overall validation outcome. It is deliberately framework-light so the control flow is legible.

from __future__ import annotations

from dataclasses import dataclass, field
from datetime import date, timedelta
from enum import Enum


class Severity(Enum):
    OBSERVATION = "observation"   # note it, no action required
    LIMITATION = "limitation"     # bounds the model's use
    BLOCKING = "blocking"         # model may not be relied upon


@dataclass(frozen=True)
class Finding:
    """One effective-challenge finding raised during validation."""
    element: str          # conceptual_soundness | ongoing_monitoring | outcomes_analysis
    severity: Severity
    description: str


@dataclass
class ValidationResult:
    model_id: str
    findings: list[Finding] = field(default_factory=list)

    @property
    def approved(self) -> bool:
        # Effective challenge is meaningful only if a blocking finding stops reliance.
        return not any(f.severity is Severity.BLOCKING for f in self.findings)


def evaluate_conceptual_soundness(model, findings: list[Finding]) -> None:
    """SR 11-7 element 1: is the design theoretically and empirically sound?"""
    if not model.assumptions_documented:
        findings.append(Finding(
            "conceptual_soundness", Severity.BLOCKING,
            "Assumption set is undocumented; developmental evidence cannot be assessed.",
        ))
    if model.data_quality_score < 0.90:
        findings.append(Finding(
            "conceptual_soundness", Severity.LIMITATION,
            "Input data quality below threshold; restrict to in-sample cohorts.",
        ))


def evaluate_ongoing_monitoring(model, findings: list[Finding]) -> None:
    """SR 11-7 element 2: does the model still perform as intended in production?"""
    if model.assumption_drift_psi > 0.25:
        findings.append(Finding(
            "ongoing_monitoring", Severity.BLOCKING,
            "Population Stability Index breach: lapse assumption has drifted materially.",
        ))
    if model.used_outside_scope:
        findings.append(Finding(
            "ongoing_monitoring", Severity.LIMITATION,
            "Model applied beyond its documented intended use.",
        ))


def evaluate_outcomes_analysis(model, findings: list[Finding]) -> None:
    """SR 11-7 element 3: how does output compare to actual experience?"""
    actual_to_expected = model.actual_claims / model.expected_claims
    if not 0.90 <= actual_to_expected <= 1.10:
        findings.append(Finding(
            "outcomes_analysis", Severity.LIMITATION,
            f"Back-test A/E ratio {actual_to_expected:.2f} outside the 90-110% corridor.",
        ))


def validate_model(model) -> ValidationResult:
    """Run all three SR 11-7 validation elements and return the verdict."""
    result = ValidationResult(model_id=model.model_id)
    evaluate_conceptual_soundness(model, result.findings)
    evaluate_ongoing_monitoring(model, result.findings)
    evaluate_outcomes_analysis(model, result.findings)
    return result

Three design choices carry the SR 11-7 intent. Keeping the three elements as separate functions makes it impossible to quietly skip one — a validation that never called evaluate_outcomes_analysis produces a ValidationResult with no outcomes findings, which a completeness check downstream can flag. Modelling severity as an enum, with BLOCKING alone able to sink approval, encodes effective challenge as a mechanism rather than an aspiration: a validator who identifies a fatal flaw produces a finding that actually stops reliance on the model. And attaching every finding to its element and model makes the whole record legible to the audit trail, so a supervisor can see not just that a model was validated but what was challenged and how it was resolved.

Configuration and Tuning

The thresholds embedded above — the 0.25 Population Stability Index limit, the 90-110% actual-to-expected corridor, the 0.90 data-quality floor — should never be constants buried in code. They are the calibration surface of the whole programme, and SR 11-7 expects them to be set deliberately, reviewed, and applied proportionately to model risk. Externalise them as a tier-aware configuration so a high-materiality reserve model faces tighter tolerances and a more frequent validation cadence than a low-risk illustration tool.

from dataclasses import dataclass


@dataclass(frozen=True)
class TierPolicy:
    """Validation intensity calibrated to a model's risk tier."""
    revalidation_months: int      # maximum interval between full validations
    psi_limit: float              # ongoing-monitoring drift tolerance
    ae_corridor: tuple[float, float]  # acceptable actual-to-expected band


TIER_POLICIES: dict[str, TierPolicy] = {
    # High-materiality models: tightest tolerances, annual revalidation.
    "tier_1": TierPolicy(revalidation_months=12, psi_limit=0.10, ae_corridor=(0.95, 1.05)),
    "tier_2": TierPolicy(revalidation_months=24, psi_limit=0.25, ae_corridor=(0.90, 1.10)),
    # Low-risk models: proportionate touch, less frequent review.
    "tier_3": TierPolicy(revalidation_months=36, psi_limit=0.40, ae_corridor=(0.85, 1.15)),
}


def next_validation_due(last_validated: "date", tier: str) -> "date":
    from dateutil.relativedelta import relativedelta
    policy = TIER_POLICIES[tier]
    return last_validated + relativedelta(months=policy.revalidation_months)

Two tuning principles matter. First, tiers must be assigned on materiality and complexity together, not size alone — a small but highly leveraged capital model can outrank a large but simple valuation extract. Second, the cadence is a ceiling, not a target: a Tier 1 model whose ongoing monitoring trips a drift alarm should be revalidated immediately, regardless of when its scheduled review falls. The configuration sets the slow clock; monitoring provides the fast interrupt.

Step-by-Step

  1. Stand up the inventory first. Nothing else in SR 11-7 is measurable until every model has a durable identity and a record. Register each model with an owner, a documented purpose, and its upstream and downstream dependencies.
  2. Assign a risk tier. Score materiality and complexity, map the pair to a tier, and attach the corresponding TierPolicy. The tier decides how hard everything downstream works.
  3. Evaluate conceptual soundness at development. Before a model is relied upon, an independent reviewer assesses the theory, the assumption selection, and the fit of data to intended use, and documents the developmental evidence.
  4. Deliver effective challenge, and mean it. Validation findings must be able to block reliance. Ensure the validation function has the standing to escalate a BLOCKING finding over a developer’s objection.
  5. Turn on ongoing monitoring. Instrument the live model with drift metrics and scope checks so degradation surfaces between scheduled validations rather than at the next one.
  6. Run outcomes analysis on a cycle. Back-test model output against emerging experience — actual-to-expected claims, realised versus projected lapses — and record the ratios in the audit trail.
  7. Feed findings back into development. Route material outcomes findings to the model owner as change requests, closing the lifecycle loop and re-triggering conceptual-soundness review where warranted.
  8. Report aggregate model risk. Roll the inventory, tiers, and open findings into a management view so the board can see total exposure, not just individual model verdicts.

Validation and Testing

The governance layer is itself software, and it deserves the same testing discipline it imposes on actuarial models. Assert that a blocking finding actually blocks, that the three elements all run, and that tier policy binds the tolerances.

from datetime import date


class _StubModel:
    model_id = "RESERVE_PBR_V4"
    assumptions_documented = True
    data_quality_score = 0.97
    assumption_drift_psi = 0.31        # breaches the tier_2 limit
    used_outside_scope = False
    actual_claims = 1_020_000.0
    expected_claims = 1_000_000.0


def test_blocking_finding_denies_approval():
    result = validate_model(_StubModel())
    assert not result.approved                    # PSI breach is blocking
    assert any(f.element == "ongoing_monitoring" for f in result.findings)


def test_all_three_elements_execute():
    result = validate_model(_StubModel())
    elements = {f.element for f in result.findings}
    # A silent element is a coverage gap, not a clean pass — assert coverage explicitly.
    assert "outcomes_analysis" in elements


def test_tier_1_cadence_is_annual():
    due = next_validation_due(date(2026, 1, 1), "tier_1")
    assert due == date(2027, 1, 1)

Beyond unit tests, reconcile the inventory against reality on a schedule: every model the platform executes should resolve to exactly one active inventory record, and any execution with no matching record is a governance gap that SR 11-7 would treat as an uninventoried model. That reconciliation is the single most effective control against the inventory quietly falling out of date.

Failure Modes and Gotchas

  • Validation without effective challenge. The most common SR 11-7 failure is a validation function that reviews but cannot block — findings are logged, the model ships anyway, and the exercise is theatre. Effective challenge requires that a blocking finding has teeth; if it does not, the control does not exist.
  • A stale inventory. An inventory that is updated at audit time and ignored the rest of the year understates aggregate model risk precisely when it matters. Drive inventory currency from execution telemetry, not from a quarterly spreadsheet refresh.
  • Tier inflation and its opposite. Owners under-rate their own models to escape scrutiny, or an over-cautious programme tiers everything as high-risk and drowns the validation team. Both defeat proportionality; tier assignment should be set by the validation function, not the developer, and periodically re-examined.
  • Collapsing the three elements. Treating a one-time conceptual review as the whole of validation ignores ongoing monitoring and outcomes analysis, so a model that was sound at launch drifts undetected. The three elements are a continuing programme, not a launch gate.
  • Orphaned dependencies. When a model consumes the output of another model that was retired or renamed, the dependency silently breaks and the consuming model runs on stale inputs. Inventory records must capture upstream and downstream links so a retirement triggers an impact review.
  • Confusing model risk with model error. SR 11-7 governs correct models used incorrectly as much as incorrect models. A perfectly valid mortality model applied to a product it was never calibrated for is a model-risk event, and only scope monitoring — not accuracy testing — will catch it.

Frequently Asked Questions

What are the three core elements of SR 11-7 model validation?

An evaluation of conceptual soundness, ongoing monitoring, and outcomes analysis. The first assesses the model’s design, theory, assumptions, and data at development; the second confirms it keeps performing as intended in production as conditions drift; the third back-tests output against actual experience. A validation programme should keep all three running continuously rather than treating any one as a substitute for the others.

What does effective challenge mean under SR 11-7?

Effective challenge is critical analysis by objective, informed parties who can identify a model’s limitations and assumptions and compel appropriate change. SR 11-7 says its effectiveness depends on the challengers’ competence, their influence within the organisation, and their incentives — which is why the validation function must be independent of model development and empowered to block reliance on a model, not merely to file an opinion.

How does SR 11-7 relate to OSFI E-23 and the NAIC framework?

They are sibling regimes expressing the same principles for different jurisdictions. SR 11-7 is US supervisory guidance from the Federal Reserve and OCC; OSFI E-23 is the Canadian equivalent for federally regulated insurers and banks; and the NAIC Valuation Manual embeds model-governance expectations into US statutory reserving. A carrier operating across borders can run one inventory and one validation engine and map its findings onto each regime rather than maintaining separate pipelines.

Why is a model inventory the foundation of SR 11-7 compliance?

Because model risk cannot be managed in aggregate without a complete, current catalogue of what models exist, who owns them, what they are for, and how risky each is. The inventory is what makes tiering, validation scheduling, and dependency analysis possible at scale, and an uninventoried model in production is itself a finding under the guidance.

Up a level: Regulatory Architecture & Compliance Mapping

Regulatory references: Board of Governors of the Federal Reserve System SR 11-7 (Guidance on Model Risk Management) and OCC Bulletin 2011-12.