Building a Python Rule Engine for Policy Lapse Assumptions

Building a Python rule engine for policy lapse assumptions means turning a lapse basis — an experience-study rate curve, a surrender-charge schedule, and a dynamic response to policyholder option value — into a strongly typed, version-pinned function that emits a decrement rate for every in-force policy and a fingerprint that ties that rate back to an approved memorandum. This page isolates the engine itself from the wider Policy Lapse & Surrender Assumption Engines subsystem and shows the minimal, runnable pattern: a frozen assumption set that validates fail-closed at the ingestion boundary, a vectorized dynamic-lapse projection over a whole cohort, and a canonical hash for the audit trail. It is the concrete implementation of the control plane described in Assumption Validation & Rule Engine Design, scoped to the single most leveraged behavioral input on a life or annuity balance sheet.

The Problem in One Paragraph #

Lapse and surrender behavior cannot be modeled as a static table because it is dynamic: policyholders lapse more when a universal life policy is out of the money in a rising-rate environment, and less when a variable annuity guarantee is deep in the money and worth holding. Under NAIC VM-21, variable annuity reserves must reflect that dynamic relationship between lapse and the in-the-moneyness of guarantees, and the resulting rates must sit inside prudent-estimate margins; VM-20 Section 9 requires the same lapse basis for life PBR to be anchored to a company experience study and documented in the actuarial memorandum. A lapse rate hard-coded inside a projection model satisfies none of this — it cannot show its source, its margin, or its version. The engine below expresses lapse as an explicit, unit-testable function of policy duration and option moneyness, validates the basis before any rate is applied, and hashes the exact snapshot used so the number in the reserve model reconciles to the number in the filing.

Minimal Working Example #

The whole engine fits in one file. A frozen LapseAssumptionSet carries the base-rate curve, the dynamic-lapse slope, and its regulatory reference; a validate gate fails closed on any boundary violation; and project_lapse_rates applies the dynamic overlay across an entire in-force cohort in a single vectorized pass. No projection scaffolding, no I/O — just the rule-engine spine.

from __future__ import annotations

import hashlib
import json
from dataclasses import dataclass
from datetime import date

import numpy as np


@dataclass(frozen=True, slots=True)
class LapseAssumptionSet:
    """A version-pinned, examiner-traceable lapse basis for one product cohort."""

    version: str
    valuation_date: date
    product: str
    base_lapse: np.ndarray          # ultimate lapse by policy duration, year 1..N
    itm_sensitivity: float          # dynamic-lapse slope on in-the-moneyness
    max_dynamic_multiplier: float   # prudent-estimate cap on the dynamic overlay
    reg_ref: str                    # e.g. "VM-21 dynamic policyholder behavior"

    def fingerprint(self) -> str:
        """Canonical SHA-256 of the basis — stable across processes and runs."""
        payload = json.dumps(
            {
                "version": self.version,
                "valuation_date": self.valuation_date.isoformat(),
                "product": self.product,
                "base_lapse": np.round(self.base_lapse, 6).tolist(),
                "itm_sensitivity": self.itm_sensitivity,
                "max_dynamic_multiplier": self.max_dynamic_multiplier,
                "reg_ref": self.reg_ref,
            },
            sort_keys=True,
            separators=(",", ":"),
        )
        return hashlib.sha256(payload.encode("utf-8")).hexdigest()


def validate(basis: LapseAssumptionSet) -> None:
    """Fail closed at the ingestion boundary — no rate is applied until this passes."""
    if basis.valuation_date > date.today():
        raise ValueError(f"future-dated basis {basis.version}")
    if np.any(basis.base_lapse < 0.0) or np.any(basis.base_lapse > 1.0):
        raise ValueError("base lapse outside [0, 1]")
    if not np.all(np.diff(basis.base_lapse) >= 0.0):
        raise ValueError("select-period base lapse must grade non-decreasing to ultimate")
    if not 1.0 <= basis.max_dynamic_multiplier <= 3.0:
        raise ValueError("dynamic cap outside prudent-estimate band")


def project_lapse_rates(
    duration: np.ndarray,           # policy year, 1-based
    account_value: np.ndarray,      # policyholder account value
    guaranteed_value: np.ndarray,   # guaranteed benefit base
    basis: LapseAssumptionSet,
) -> np.ndarray:
    """Dynamic lapse for one in-force cohort, vectorized over every policy."""
    validate(basis)

    dur_idx = np.clip(duration - 1, 0, basis.base_lapse.size - 1)
    base_rate = basis.base_lapse[dur_idx]

    # In-the-moneyness of the guarantee: > 1 means the option is valuable to hold.
    moneyness = np.divide(
        guaranteed_value,
        account_value,
        out=np.ones_like(account_value, dtype="float64"),
        where=account_value > 0.0,
    )
    # Deeper in-the-money guarantees suppress lapse; out-of-the-money lifts it.
    overlay = 1.0 - basis.itm_sensitivity * (moneyness - 1.0)
    overlay = np.clip(
        overlay,
        1.0 / basis.max_dynamic_multiplier,
        basis.max_dynamic_multiplier,
    )

    lapse_rate = np.clip(base_rate * overlay, 0.0, 1.0)
    return lapse_rate.astype(np.float32)


if __name__ == "__main__":
    basis = LapseAssumptionSet(
        version="ul-dynamic-2026Q2",
        valuation_date=date(2026, 6, 30),
        product="variable_annuity_gmwb",
        base_lapse=np.array([0.02, 0.03, 0.04, 0.05, 0.07, 0.10], dtype="float64"),
        itm_sensitivity=0.35,
        max_dynamic_multiplier=2.0,
        reg_ref="VM-21 dynamic policyholder behavior",
    )
    duration = np.array([1, 3, 5, 5])
    account_value = np.array([120_000.0, 95_000.0, 80_000.0, 0.0])
    guaranteed_value = np.array([100_000.0, 100_000.0, 130_000.0, 90_000.0])

    rates = project_lapse_rates(duration, account_value, guaranteed_value, basis)
    print(basis.fingerprint())
    print(rates)

Run it as-is and it validates the basis, prints a reproducible SHA-256 fingerprint, and returns a float32 lapse rate for each of the four sample policies — the deep in-the-money guarantee lapses well below its base rate, the out-of-the-money one above it.

Block-by-Block Explanation #

LapseAssumptionSet — immutability is traceability. The class is frozen=True so a basis cannot be mutated after construction: once a projection run records its fingerprint, no downstream code can silently shift a rate out from under the audit trail. slots=True drops the per-instance __dict__, which matters when thousands of product cohorts are held in memory during a valuation. Every field is metadata an examiner will ask for — version, valuation_date, product, and an explicit reg_ref mapping the basis to the standard it satisfies.

fingerprint — canonical serialization or nothing. The hash is computed over json.dumps with sort_keys=True and tight separators, and the rate array is rounded and converted with tolist() before hashing. This is the load-bearing detail: Python’s built-in hash() is salted per interpreter session and will differ run to run, and an un-sorted dict or a raw NumPy buffer serializes non-deterministically across platforms. Only a canonicalized payload produces a fingerprint that re-derives identically, which is what lets the Actuarial Audit Trail Architecture reconcile a filed reserve to its source assumption years later.

validate — a fail-closed gate, not a warning. The function raises rather than returning a flag, so a bad basis halts the run instead of quietly producing a number. It rejects future-dated bases, rates outside $[0, 1]$, and a select-period curve that does not grade non-decreasing toward its ultimate — the shape a real lapse table takes as surrender charges wear off. This mirrors the ingestion-boundary discipline of Schema Validation with Pydantic & Great Expectations: validate once, at the edge, before any arithmetic touches the cohort.

project_lapse_rates — the dynamic overlay, vectorized. np.clip(duration - 1, ...) maps each policy’s 1-based year onto the rate curve and pins anything beyond the table to the ultimate rate, so a 30-year policy against a 6-year curve never raises IndexError. The in-the-moneyness ratio drives a linear overlay applied per policy:

$$\text{lapse}_i = \text{base}(d_i)\cdot\operatorname{clip}\!\Big(1 - s\,(m_i - 1),\ \tfrac{1}{c},\ c\Big),\qquad m_i=\frac{\text{GV}_i}{\text{AV}_i}$$

where $s$ is itm_sensitivity, $c$ is max_dynamic_multiplier, and $m_i$ is the guarantee’s moneyness. The whole cohort is processed in one broadcast pass — the vectorization concern shared with Pandas & NumPy for Actuarial Data Pipelines — and the clip band is exactly where the prudent-estimate margin lives: no dynamic effect may move the rate beyond $c\times$ or $1/c\times$ its base, so the overlay can never manufacture an unapproved extreme.

Edge Cases and Production Hardening #

Three failure modes account for nearly every lapse-engine bug that reaches production. Each has a concrete fix already wired into the example.

1. Zero or negative account value. A surrendered or fully-drawn policy has account_value == 0, and a naive guaranteed_value / account_value yields inf or nan that then propagates through the whole reserve. The np.divide(..., out=np.ones_like(...), where=account_value > 0.0) guard returns a neutral moneyness of 1.0 for those policies so the overlay collapses to the base rate instead of exploding. Never divide raw; always supply the where/out pair and decide explicitly what a zero denominator means for the assumption.

2. Duration beyond the table. In-force blocks contain policies older than the longest experience-study duration. Indexing base_lapse[duration - 1] directly would raise IndexError on year 30 against a 6-year curve, or worse, wrap a negative index and silently pull the wrong rate. np.clip(duration - 1, 0, base_lapse.size - 1) pins every over-long duration to the ultimate rate — the actuarially correct grading — and makes the lookup total over any cohort.

3. Hash instability from dtype and serialization. The projection returns float32 to halve memory on tens of millions of policies, but a fingerprint computed over float32 versus float64 buffers, or over an un-sorted dict, diverges across platforms and breaks the reconciliation. The fix is the canonical JSON in fingerprint — sorted keys, fixed separators, rounded rates via tolist() — and never hash(), whose per-session salt makes it useless for compliance. Assert in a unit test that two independently constructed copies of the same basis produce the same digest; that one assertion catches the entire class of serialization drift.

Compliance Note #

A dynamic lapse engine only earns its place in a filing when it is reproducible and margin-aware. Under NAIC VM-21, the dynamic relationship this overlay encodes between lapse and guarantee moneyness is prescribed for variable annuity reserves, and the max_dynamic_multiplier clip is where the prudent-estimate margin required by ASOP 52 is made explicit and auditable rather than buried in a spreadsheet. For life PBR, VM-20 Section 9 requires the base_lapse curve to be anchored to a company experience study and graded to an industry basis where credibility is thin under ASOP 25. The validate gate and the canonical fingerprint together satisfy the ongoing-monitoring and change-control expectations of SR 11-7: every rate applied to an in-force cohort is traceable to a versioned, validated snapshot, and the digest persisted into Building Secure Audit Logs for Regulatory Submissions is what makes the run examiner-ready.

Related Guides #

• Policy Lapse & Surrender Assumption Engines — the full subsystem this engine plugs into, from experience study to filing sync
• Economic Scenario Mapping & Yield Curve Alignment — how the credited-versus-market spread that drives dynamic lapse is calibrated
• Dynamic Threshold Tuning for Assumption Drift — catching a lapse basis that has drifted from emerging experience
• Mortality & Morbidity Rate Validation — cross-validating lapse against the other decrements in the same projection
• NAIC VM-20 Compliance Frameworks — the Section 9 rules the base-lapse curve must satisfy

Up one level: Policy Lapse & Surrender Assumption Engines · Assumption Validation & Rule Engine Design