Actuarial Validation & Filing Automation

Building Great Expectations Checkpoints for Actuarial Data

A Great Expectations Checkpoint turns a loose promise about a policy extract into an enforced data contract: it binds an Expectation Suite to a batch and fails the run the moment policy_id goes null, a lapse_rate lands outside [0, 1], or the row count drifts from prior close. This guide builds one focused Checkpoint for a life-insurance policy extract, sitting inside the wider Schema Validation with Pydantic & Great Expectations discipline and complementing the record-level guarantees earned in Validating Actuarial Input Schemas with Pydantic.

The Problem

VM-20 reserves and SR 11-7 model governance both rest on an unspoken assumption: that the data feeding the projection engine is the data the actuary believes it to be. In practice a nightly policy extract arrives from an administration system that nobody on the actuarial team controls, and it degrades quietly — a join drops a cohort, a currency conversion silently truncates face_amount, an upstream release renames a column, or a batch reruns and doubles the row count. None of these raise an exception in pandas; the reserve simply comes out wrong and stays plausible. A Checkpoint is the automated gate that inspects the batch against a codified Expectation Suite before a single reserve is computed, and refuses to pass a breach downstream.

A Minimal Working Example

The example below defines an Expectation Suite for a policy extract, wraps it in a Checkpoint, and runs it against a pandas batch so that any contract breach flips checkpoint_result.success to False. It targets the fluent (1.x) Great Expectations API and is runnable as a script:

from __future__ import annotations

import datetime as dt

import great_expectations as gx
import pandas as pd

VALUATION_DATE = dt.date(2024, 12, 31)
EXPECTED_ROW_COUNT = 250_000
DRIFT_TOLERANCE = 0.02  # 2% band around the prior close row count


def build_policy_suite(context: gx.data_context.AbstractDataContext) -> None:
    suite = context.suites.add(gx.ExpectationSuite(name="policy_extract_contract"))

    # 1. Identity must be present and unique — the join key for every reserve.
    suite.add_expectation(
        gx.expectations.ExpectColumnValuesToNotBeNull(column="policy_id")
    )
    suite.add_expectation(
        gx.expectations.ExpectColumnValuesToBeUnique(column="policy_id")
    )
    # 2. Lapse rate is a probability: it must live in the closed unit interval.
    suite.add_expectation(
        gx.expectations.ExpectColumnValuesToBeBetween(
            column="lapse_rate", min_value=0.0, max_value=1.0
        )
    )
    # 3. Face amount funds the benefit — it must be strictly positive.
    suite.add_expectation(
        gx.expectations.ExpectColumnValuesToBeBetween(
            column="face_amount", min_value=0.01, strict_min=True
        )
    )
    # 4. No policy may be valued in the future relative to the close date.
    suite.add_expectation(
        gx.expectations.ExpectColumnValuesToBeBetween(
            column="valuation_date", max_value=VALUATION_DATE.isoformat()
        )
    )
    # 5. Row-count drift: a doubled or truncated batch is a loud failure.
    low = int(EXPECTED_ROW_COUNT * (1 - DRIFT_TOLERANCE))
    high = int(EXPECTED_ROW_COUNT * (1 + DRIFT_TOLERANCE))
    suite.add_expectation(
        gx.expectations.ExpectTableRowCountToBeBetween(min_value=low, max_value=high)
    )


def run_policy_checkpoint(extract: pd.DataFrame) -> gx.checkpoint.checkpoint.CheckpointResult:
    context = gx.get_context(mode="ephemeral")
    build_policy_suite(context)

    batch_def = (
        context.data_sources.add_pandas("policy_source")
        .add_dataframe_asset("nightly_extract")
        .add_batch_definition_whole_dataframe("close_batch")
    )
    validation = context.validation_definitions.add(
        gx.ValidationDefinition(
            data=batch_def,
            suite=context.suites.get("policy_extract_contract"),
            name="policy_extract_validation",
        )
    )
    checkpoint = context.checkpoints.add(
        gx.Checkpoint(
            name="policy_extract_checkpoint",
            validation_definitions=[validation],
            result_format="COMPLETE",
        )
    )
    result = checkpoint.run(batch_parameters={"dataframe": extract})
    if not result.success:
        raise ValueError("Policy extract failed its data contract; reserve run halted.")
    return result


if __name__ == "__main__":
    extract = pd.DataFrame(
        {
            "policy_id": [f"P{i:07d}" for i in range(EXPECTED_ROW_COUNT)],
            "lapse_rate": [0.06] * EXPECTED_ROW_COUNT,
            "face_amount": [250_000.0] * EXPECTED_ROW_COUNT,
            "valuation_date": ["2024-12-31"] * EXPECTED_ROW_COUNT,
        }
    )
    run_policy_checkpoint(extract)
    print("Policy extract passed its data contract.")
Great Expectations Checkpoint gate for a policy extract A nightly policy extract enters a Checkpoint that runs an Expectation Suite of five rules. On success the batch continues to the reserve engine; on failure the run halts and the validation result is logged to the audit trail. Nightly policy extract Checkpoint 5-rule suite null · range · drift success? Reserve engine batch passes onward Halt + audit trail validation result logged
A Checkpoint evaluates the Expectation Suite and routes the batch to the reserve engine only on a clean pass.

How It Works, Block by Block

The Expectation Suite is the contract, expressed as assertions. Each add_expectation call codifies one property the actuary is entitled to assume. ExpectColumnValuesToNotBeNull and ExpectColumnValuesToBeUnique on policy_id protect the join key that every seriatim reserve depends on — a null or duplicated identifier corrupts aggregation before any assumption is applied. The suite is a named, serializable object, so the same contract that runs tonight can be diffed, reviewed, and version-controlled like any other model artifact.

Range expectations encode domain truths, not preferences. lapse_rate between 0.0 and 1.0 is not a tuning choice; it is what makes the value a probability. face_amount uses strict_min=True against 0.01 so a zero or negative benefit — the signature of a truncated currency field or a placeholder record — fails rather than quietly deflating the reserve. The valuation_date expectation caps values at the close date, catching the classic administration-system defect where a future-dated record leaks into the extract and pulls the projection horizon forward.

Row-count drift is a table-level expectation, not a column one. ExpectTableRowCountToBeBetween compares the batch size against a band around the prior close. A batch that reran and doubled, or a failed join that halved the population, produces a reserve that is internally consistent and completely wrong; the count band is the cheapest possible tripwire for it. The 2% tolerance is deliberately narrow — real month-over-month policy movement is small, so a wide band would let a material data loss slip through.

The Checkpoint is what makes the suite enforceable. A suite on its own only describes; the Checkpoint binds it to a ValidationDefinition (suite plus batch) and executes it, returning a CheckpointResult whose success flag is the single boolean the pipeline reads. Raising on not result.success is the crucial line: it converts a data-quality opinion into a hard stop, so the reserve run never begins on a breached batch. The COMPLETE result format keeps the failing rows in the result object, which is exactly what an examiner or a debugging actuary needs.

Edge Cases and Production Hardening

Silent schema drift on new columns slips past a column-scoped suite. Every expectation above names a column, so an upstream release that adds a rider_code column — or renames face_amount to face_value — passes the suite untouched, because the rules simply never look at the new shape. Pin the column set explicitly so an unexpected or missing column is itself a failure:

EXPECTED_COLUMNS = ["policy_id", "lapse_rate", "face_amount", "valuation_date"]

suite.add_expectation(
    gx.expectations.ExpectTableColumnsToMatchSet(column_set=EXPECTED_COLUMNS)
)

With this in place a renamed or extra column trips the Checkpoint instead of being validated by omission, which is the failure mode SR 11-7 has in mind when it calls for controls over model inputs.

Expectation and Pydantic guard different layers — do not collapse them. Validating Actuarial Input Schemas with Pydantic enforces the shape and type of a single record at the moment of construction — the right tool for API payloads and per-policy coercion. Great Expectations reasons about the batch as a statistical whole: uniqueness across 250,000 rows, distributional range, row-count drift against history. A null-check belongs in Pydantic when you hold one record and in a Checkpoint when you hold the population; duplicating a rule in both layers is acceptable defense in depth, but the division of labour is that Pydantic answers “is this record well-formed?” and the Checkpoint answers “is this batch trustworthy?”

Partitioned batches need per-partition validation, not one global pass. A quarter-end extract often arrives as one file per issue-year cohort or per legal entity. Validating the concatenation hides a defect isolated to a single partition — a lapse-rate spike confined to the 2019 cohort averages away across the whole population. Add a batch definition partitioned on a column and let the Checkpoint iterate:

batch_def = (
    context.data_sources.get("policy_source")
    .get_asset("nightly_extract")
    .add_batch_definition_partitioned(
        "by_cohort", column="cohort_id"
    )
)

Each partition then earns its own pass or fail, and the failing cohort_id is named in the result rather than diluted across the aggregate.

Compliance Note

A Checkpoint is the operational answer to a governance requirement that predates it. VM-20 Section 3 and the VM-31 PBR Actuarial Report expect the appointed actuary to be able to attest that the data underlying the reserve is complete, accurate, and appropriate; SR 11-7 is more explicit still, treating input data quality as a first-class component of model risk and calling for controls that detect and reject deficient inputs before they reach the model. A named Expectation Suite, run by a Checkpoint whose failing rows are retained under COMPLETE, is precisely such a control — and because the CheckpointResult is a structured object, it can be sealed into the Actuarial Audit Trail Architecture as dated evidence that the data contract was checked on the batch that produced the filed reserve. This closes the loop the broader Actuarial Model Ingestion & Testing Workflows discipline is built to close: no reserve leaves the pipeline on data that was never verified.

Frequently Asked Questions

What is the difference between an Expectation Suite and a Checkpoint?

An Expectation Suite is the named collection of assertions describing what a valid batch looks like, while a Checkpoint is the executable object that binds that suite to a specific batch, runs it, and returns a success flag. The suite describes the contract; the Checkpoint enforces it and is the thing you wire into a pipeline.

How do I make a failing Checkpoint stop my reserve run?

Read the boolean success attribute on the CheckpointResult returned by checkpoint.run and raise an exception when it is false, before any reserve computation begins. Because the run halts on the raised error, no downstream step ever executes against a batch that breached its data contract.

Why do I still need Pydantic if I have Great Expectations?

Pydantic validates the shape and type of one record at construction time, which suits API payloads and per-policy coercion, whereas Great Expectations reasons about the whole batch — uniqueness across every row, distributional range, and row-count drift against history. They guard different layers, so the record-level and batch-level checks are complementary rather than redundant.

How do I catch a new or renamed column that my expectations do not mention?

Add a table-level expectation such as ExpectTableColumnsToMatchSet with the exact set of columns you expect, so an added, missing, or renamed column fails the Checkpoint. Column-scoped value expectations never inspect columns they do not name, so without a column-set check a schema change passes silently.

Up a level: Schema Validation with Pydantic & Great Expectations · Actuarial Model Ingestion & Testing Workflows

Regulatory references: NAIC Valuation Manual VM-20 and VM-31; Federal Reserve SR 11-7 (Guidance on Model Risk Management).