Actuarial Validation & Filing Automation

Calibrating Dynamic Lapse Formulas in Python

Calibrating a dynamic lapse formula means fitting the parameters that bend a base lapse rate up or down as a policy moves in or out of the money, so the surrender assumption responds to the rate environment the way real policyholders do rather than sitting at a static best estimate. This page shows the focused technique — a least-squares fit of an in-the-moneyness response curve to observed surrender experience — and sits inside the wider Policy Lapse & Surrender Assumption Engines work under Assumption Validation & Rule Engine Design.

The Problem

A static lapse rate is a fiction the moment interest rates move. When a policy’s credited rate falls well below what a policyholder could earn elsewhere, the contract is in the money to surrender and lapses spike; when the contract is out of the money, lapses fall and the block becomes stickier than any flat assumption predicts. A dynamic lapse formula captures this by multiplying a base lapse rate by a response function of an in-the-moneyness signal — typically the differential between a competitor or market rate and the credited rate. But the shape of that response is not something to eyeball: the sensitivity, the point of inflection, and the ceiling on how much lapses can accelerate all have to be fitted to observed surrender data, and fitted in a way that stays well-behaved where you have no experience. The technique below fits a smooth, bounded response curve to observed surrenders by least squares, enforces the caps that keep the multiplier sane, and refuses to trust the curve past the range the data actually covers.

A Minimal Working Example

The pattern is: define a bounded response multiplier driven by an in-the-moneyness signal, express the dynamic lapse rate as base times multiplier, and fit the response parameters to observed surrender rates with scipy.optimize.least_squares. It runs as-is:

from __future__ import annotations

import numpy as np
from scipy.optimize import least_squares

# Observed surrender experience: moneyness = market_rate - credited_rate.
# Positive moneyness means the contract is in the money to surrender.
moneyness = np.array([-0.020, -0.010, -0.005, 0.0, 0.005, 0.010, 0.020, 0.030])
observed_lapse = np.array([0.030, 0.038, 0.045, 0.060, 0.085, 0.110, 0.150, 0.165])

BASE_LAPSE = 0.060          # anchor lapse at zero moneyness
MULT_FLOOR, MULT_CEIL = 0.4, 3.0  # response multiplier is bounded


def response_multiplier(m: np.ndarray, alpha: float, beta: float, gamma: float) -> np.ndarray:
    """Smooth arctan response, centered at gamma, scaled by alpha and beta."""
    raw = 1.0 + alpha * np.arctan(beta * (m - gamma))
    return np.clip(raw, MULT_FLOOR, MULT_CEIL)


def dynamic_lapse(m: np.ndarray, alpha: float, beta: float, gamma: float) -> np.ndarray:
    return BASE_LAPSE * response_multiplier(m, alpha, beta, gamma)


def residuals(params: np.ndarray) -> np.ndarray:
    alpha, beta, gamma = params
    return dynamic_lapse(moneyness, alpha, beta, gamma) - observed_lapse


if __name__ == "__main__":
    initial = np.array([1.0, 50.0, 0.0])           # alpha, beta, gamma
    lower = np.array([0.0, 1.0, -0.02])
    upper = np.array([3.0, 300.0, 0.02])
    fit = least_squares(residuals, initial, bounds=(lower, upper), method="trf")
    alpha, beta, gamma = fit.x
    rmse = float(np.sqrt(np.mean(fit.fun ** 2)))
    print(f"alpha={alpha:.3f} beta={beta:.1f} gamma={gamma:.4f} rmse={rmse:.5f}")
    print("fitted:", np.round(dynamic_lapse(moneyness, alpha, beta, gamma), 4))
Calibrating a dynamic lapse formula from observed surrenders to a bounded assumption Five stages left to right: observed surrender data, define arctan response function, bounded least-squares fit, enforce caps and floors, and a calibrated dynamic lapse assumption. Observed surrender data by moneyness Response fn arctan(m) Least-squares fit bounded params Enforce caps & floors clip multiplier Calibrated lapse assumption for the engine
Calibration flow: observed surrenders to a bounded, fitted dynamic lapse assumption.

The fitted response is the multiplier

l(m)=lbase⋅clip⁡ ⁣(1+α arctan⁡(β (m−γ)),  cfloor,  cceil),l(m) = l_{\text{base}}\cdot\operatorname{clip}\!\Big(1 + \alpha\,\arctan\big(\beta\,(m-\gamma)\big),\; c_{\text{floor}},\; c_{\text{ceil}}\Big),

where mm is the in-the-moneyness signal and α,β,γ\alpha,\beta,\gamma are the parameters the least-squares fit recovers from experience.

How It Works, Block by Block

moneyness and observed_lapse are the experience the fit answers to. The moneyness signal is the differential between the market or competitor rate and the credited rate, so a positive value means the contract is in the money to surrender and the observed lapse should climb. Pairing each moneyness point with its observed surrender rate gives the fit a target to reproduce; the whole calibration is the search for a response curve that passes as close as possible to these points.

response_multiplier is a smooth, bounded arctan response. The arctan shape is chosen deliberately: it is monotone, it flattens toward horizontal asymptotes at both extremes, and it has a single inflection point, which matches the economic intuition that lapse acceleration is steepest around a threshold and then saturates. The parameter alpha scales how far the multiplier can move from one, beta sets how sharply it turns through the threshold, and gamma shifts the threshold along the moneyness axis. The np.clip wrapping the raw response is not cosmetic — it is the cap and floor that keep the multiplier from producing an absurd lapse rate no matter what the parameters or the input do.

dynamic_lapse composes the base rate with the response. Keeping the base lapse as a fixed anchor and letting the fitted multiplier bend it means the calibration only has to explain the shape of the response, not re-estimate the overall level. This separation keeps the fitted parameters interpretable: alpha reads directly as the maximum proportional acceleration, and the base rate stays governed by the ordinary experience study.

residuals and least_squares do the fitting under bounds. The residual function returns the vector of fitted-minus-observed differences, and least_squares with the trust-region-reflective method drives their sum of squares to a minimum while respecting the lower/upper bounds on each parameter. The bounds are doing real work: they keep beta positive so the response never inverts, and they hold gamma inside the observed moneyness range so the inflection cannot wander off into a region the data never saw. The reported RMSE is the plain-language summary of how well the calibrated curve reproduces experience.

Edge Cases and Production Hardening

alpha and beta are only jointly identifiable near the threshold. Around the inflection point the arctan is close to linear, and there alpha * beta behaves like a single slope — so a fit can trade a larger alpha against a smaller beta and land on nearly the same curve with wildly different individual parameters. When your moneyness data clusters near zero and never reaches the flat tails, pin down the identifiability by reporting the fitted curve and its confidence band rather than the raw parameters, and hold alpha at a governance-set value while fitting only beta and gamma:

def residuals_fixed_alpha(params: np.ndarray, alpha_fixed: float) -> np.ndarray:
    beta, gamma = params
    return dynamic_lapse(moneyness, alpha_fixed, beta, gamma) - observed_lapse

fit2 = least_squares(
    residuals_fixed_alpha, x0=np.array([50.0, 0.0]),
    args=(1.2,), bounds=([1.0, -0.02], [300.0, 0.02]), method="trf",
)

An unclipped multiplier can drive lapse above one or below zero. Without the cap and floor, a steep fit combined with an extreme moneyness input can push the multiplier — and therefore the lapse rate — outside the interval a probability may occupy, which corrupts every downstream reserve. Enforce the bound at evaluation time, not only at fit time, so that a production input more extreme than anything in the experience data is still clamped to a sane multiplier before it reaches the Building a Python Rule Engine for Policy Lapse Assumptions that applies it seriatim.

Extrapolation beyond observed moneyness is where the curve lies to you. The fit is only informed inside the range of moneyness you supplied; past the largest observed differential the arctan simply approaches its asymptote, which may or may not reflect real behaviour in a rate shock far outside history. Clamp the input moneyness to the calibrated range before evaluating, flag any clamped policy for review, and monitor the live experience for drift so a shifting distribution is caught early — the divergence measures compared in PSI vs KL Divergence for Lapse Drift Detection are the natural trigger for a recalibration.

Compliance Note

Under VM-20 Section 9, which governs the assumptions used in principle-based reserves, lapse and other policyholder behaviour must reflect the dynamic relationship between the surrender decision and economic conditions rather than a single static rate, with margins added where experience is limited or the behaviour is not fully credible. A fitted response curve is the mechanical expression of that requirement: it makes the dependence on in-the-moneyness explicit, bounded, and reproducible. The bounds on the parameters and the caps on the multiplier are also how the required margin and the credibility limits are enforced in code, so the calibrated assumption can be reconstructed and defended in the actuarial memorandum. Keeping the base lapse, the fitted parameters, and the moneyness range together as a versioned record is what lets a reviewer trace the dynamic assumption back to the experience that justified it.

Frequently Asked Questions

Why use an arctan response instead of a plain linear multiplier?

Because policyholder behaviour saturates: lapse acceleration is steepest around an in-the-moneyness threshold and then levels off as the contract becomes deeply in or out of the money. The arctan is monotone with two horizontal asymptotes and a single inflection point, which reproduces that saturating shape, whereas a linear multiplier would keep climbing without bound and misstate lapses at the extremes.

What does the moneyness signal actually represent?

It is the differential between an external market or competitor rate and the contract’s credited rate. A positive value means the policyholder could do better elsewhere, so the contract is in the money to surrender and observed lapses should rise; a negative value means the contract is attractive to keep, so lapses fall below the base rate.

Why are the fit parameters bounded during calibration?

The bounds keep the response well-behaved: they hold the steepness parameter positive so the curve never inverts, and they keep the inflection point inside the observed moneyness range so it cannot drift into a region the data never covered. They also encode the credibility and margin limits, so the fitted curve stays defensible rather than chasing noise.

How do I know when to recalibrate a dynamic lapse formula?

Monitor live surrender experience against the fitted curve and watch for distribution drift in the moneyness signal. When a drift measure crosses its threshold, or when new experience falls persistently outside the fitted band, the calibration no longer reflects behaviour and should be refit on the updated data.

Up a level: Policy Lapse & Surrender Assumption Engines · Assumption Validation & Rule Engine Design

Regulatory references: NAIC Valuation Manual VM-20 Section 9, assumptions including dynamic policyholder behaviour; Actuarial Standards Board ASOP No. 52 (PBR for Life Products).