5 Overlapping Generations Models with DEQNs

In Chapters Chapter 2--Chapter 3 all agents were infinitely lived. We now extend the DEQN framework to overlapping generations (OLG) models Diamond, 1965, where $A$ finitely-lived cohorts coexist in every period. OLG models introduce lifecycle savings, intergenerational transfers, age-dependent heterogeneity, and inequality constraints on portfolio choices, phenomena that are central to fiscal policy analysis, pension reform, and demographic modeling. We proceed in two stages. We first solve a deliberately small 6-agent OLG that admits a closed-form solution Krueger & Kubler, 2004, which gives a clean ground truth against which to validate the neural-network solver. We then scale up to the 56-agent research benchmark of Azinovic et al. (2022), where the no-short-sale-of-capital constraint $k'^h\ge 0$ binds on a non-trivial slice of the ergodic set; that constraint introduces a kink, the main new computational challenge of the benchmark, and we handle it by combining softplus output activations (for non-negativity) with squared product residuals for the orthogonality conditions in the loss. The model also carries a collateral constraint $k'^h+\kappa\,b'^h\ge 0$ that the current notebook parameterization of $\hat q^h$ keeps slack on the learned ergodic set; we develop both constraints below so that the architecture is in place when a future calibration makes the collateral side bind.

5.1Why Overlapping Generations?¶

In the Brock--Mirman and IRBC models of Chapters Chapter 2--Chapter 3, all agents are infinitely lived. Picture instead a photograph of the economy taken at a single instant: it contains a twenty-something just entering the workforce with no savings, a forty-something at peak earnings putting money aside, and a retiree drawing down a lifetime of accumulated wealth, all making decisions in the same period and all linked through the prices that their collective saving determines. The infinitely-lived-agent assumption collapses this picture and rules out several economically important phenomena:

• Lifecycle savings. Agents accumulate wealth when young, draw it down in old age.

• Intergenerational transfers. Pensions, social security, and bequests cannot be studied without age structure.

• Age-dependent heterogeneity. Labor endowments, risk preferences, and portfolio composition vary systematically over the lifecycle.

An OLG economy consists of $A$ cohorts that coexist in each period: a new cohort of age 1 is born, the oldest cohort of age $A$ dies, and everyone else ages by one period. Crucially, the number of agent types is finite, so the cross-sectional distribution has only $A$ entries and the state space remains finite-dimensional, in contrast to the continuum-of-agents models treated in Chapter Chapter 6. The mechanism that ties the three phenomena above together is consumption smoothing over a hump-shaped earnings path (Figure Figure 5.1): because labor income rises and then falls over the lifecycle while agents prefer a steady consumption stream, they accumulate assets in their high-earning years and run them down afterwards, and the equilibrium interest rate is whatever clears the resulting demand for savings against the economy’s capital stock.

We develop the OLG framework in two stages. Section Section 5.2 works through the 6-agent model with a closed-form solution, maps it to a DEQN (Section 5.3), and validates the trained network against the analytical savings rates; Section 5.4 then explains how binding borrowing and collateral constraints are encoded, and Section 5.5 solves the 56-agent research benchmark with exactly the same training loop.

5.2The 6-Agent Analytic OLG Model¶

Krueger & Kubler (2004) proposed a deliberately simple OLG model with a closed-form solution, making it an ideal validation benchmark for the DEQN approach. We develop it here as the first of the two OLG instances of this chapter; Section 5.3 maps it to a DEQN and validates the trained network against the closed form derived below.

We instantiate the OLG environment with $A=6$ overlapping cohorts, indexed by age $h \in \{1,\ldots,6\}$. Time is discrete and infinite. The model equations below are written for general $A$ and specialized to $A=6$ in the calibration that follows.

5.2.1Household problem.¶

An agent of age $h$ at time $t$ maximizes expected lifetime utility:

subject to the period budget constraint

where $k_t^h$ denotes capital holdings, $r_t$ is the gross return on capital, $w_t$ is the wage, $\ell^h$ is an age-dependent labor endowment, and $\mathrm{inc}_t^h$ is total income.

5.2.2Boundary conditions.¶

• Newborns have no initial wealth: $k_t^1 = 0$.

• The oldest cohort consumes everything: $k_{t+1}^{A+1} = 0$.

• Borrowing is not permitted: $k_{t+1}^{h+1} \geq 0$ for all $h$.

5.2.3Euler equations.¶

The first-order conditions yield $A-1$ Euler equations (for ages $h=1,\ldots,A-1$):

5.2.4Firm problem and market clearing.¶

A representative firm operates a Cobb--Douglas technology with value added $F_t = \eta_t K_t^\alpha L_t^{1-\alpha}$, where $\eta_t$ is a TFP shock and $L_t = \sum_{h=1}^A \ell^h$; the gross resource available to households is $Y_t = F_t + (1-\delta_t)K_t = r_t K_t + w_t L_t$ (it is $Y_t$, not $F_t$, that the notebook passes as an engineered feature). Competitive factor markets imply:

where $\delta_t$ is the depreciation rate (potentially stochastic). Market clearing requires that aggregate capital at $t+1$ is the sum of holdings across cohorts:

with $k_{t+1}^{1}=0$ as a newborn boundary condition (cohort 1 enters life with no assets), and where $k_{t+1}^{h}$ for $h=2,\dots,A$ is the savings of cohort $h-1$ at date $t$ (which becomes the date-$(t+1)$ holdings of the cohort once it has aged by one period).

5.2.5Calibration.¶

The model has $A=6$ agents with log utility ($\gamma = 1$), Cobb--Douglas production ($\alpha = 0.3$), and discount factor $\beta = 0.7$. Only agent 1 works ($\ell = (1,0,0,0,0,0)$); this stripped-down labor profile is what gives the closed form below, not a realistic lifecycle assumption, and the 56-agent benchmark of Section 5.5 restores a hump-shaped endowment. Four exogenous shock states combine TFP $\eta \in \{0.95,1.05\}$ and depreciation $\delta \in \{0.5,0.9\}$, with i.i.d. transitions ($\pi_{ss'} = 0.25$).

5.2.6Analytical solution.¶

With log utility and i.i.d. shocks, the optimal savings rate has a closed form. Define the age-dependent savings rate:

The optimal policy is then $k'^h = \beta_h \cdot \mathrm{inc}^h$: each agent saves a fixed fraction of total income, regardless of the shock. Two features of the calibration drive this clean form. First, under log utility the income and substitution effects of a return shock exactly cancel, so the savings rate is invariant to $(r_t, w_t)$. Second, because the shocks are i.i.d. there is nothing about the future to forecast, so the rate does not depend on the current shock either; only the horizon matters. The fraction $\beta_h$ therefore declines with age: cohort $h$ has only $A-h$ remaining periods over which to spread its future income, so the marginal incentive to carry resources forward weakens as $h$ grows. For $A=6$, $\beta=0.7$, Table Table 5.1 reports the resulting savings rates.

Young agents save more (more periods ahead); old agents save less; Figure Figure 5.2 plots the same numbers across $h$. This vector is the validation target: at convergence, the trained network’s average sigmoid output should reproduce $\beta_h$ cohort by cohort.

5.3Mapping the Analytic OLG to a DEQN¶

The mapping follows the same “states $\to$ network $\to$ loss” structure as Brock--Mirman (Chapter Chapter 2). We now write each block explicitly for the 6-agent analytic model just set up; this is exactly what slides II.7--II.9 of lectures/lecture_08_olg_models_deqns/slides/lecture_08_olg_models_deqns.tex render in pictures. The 56-agent benchmark of Section 5.5 extends the same template with two extra policy blocks (multipliers, bond price) and an additional market-clearing residual; we write that version out there.

5.3.1State $\x_t$ entering the network.¶

What does the network actually need to know? The informational state of the analytic model is just the pair

the current shock index plus the cross-sectional capital distribution. This is the minimal vector that pins down the equilibrium, and it is what slide II.8 displays in the FREE signature. Everything else, the aggregate capital $K_t=\sum_h k_t^h$, the prices $(r_t,w_t)$, output $Y_t$, each cohort’s income, the row of next-period transition probabilities, is a deterministic function of $(z_t,\bm{k}_t)$. The network could in principle re-derive all of it from the raw pair, but there is no reason to make it: we hand the network those quantities pre-computed, which is a pure change of input coordinates that leaves the equilibrium map untouched and frees the network’s capacity for the one genuinely hard thing it has to learn, the savings policy. Concretely the notebook feeds an extended state of dimension $16+4A$,

with $\mathbf{1}\{z_t\}$ the 4-state one-hot of the current shock, $K_t=\sum_h k_t^h$, $L_t=\sum_h \ell^h$, $(r_t,w_t)$ from (5.4), $Y_t$ the gross resource $\eta_t K_t^\alpha L_t^{1-\alpha} + (1-\delta_t)K_t$, and the per-agent blocks $\mathrm{fw}_t^h = r_t k_t^h$ (capital income), $\mathrm{linc}_t^h = w_t\,\ell^h$ (labor income), $\mathrm{inc}_t^h = \mathrm{fw}_t^h + \mathrm{linc}_t^h$ (total income). Since the map $(z_t,\bm{k}_t)\mapsto\x_t$ is deterministic, (5.7) and (5.8) carry exactly the same information. For $A=6$ this is $16+4\cdot 6 = 40$ inputs (the notebook constant FEATURE_DIM).

5.3.2Policies approximated by the network.¶

A single multilayer perceptron with a sigmoid savings-fraction output head approximates the equilibrium policy as a function of the state. (Throughout this OLG chapter we use $\theta$ for the network parameters rather than the $\rho$ of Chapters Chapter 2--Chapter 3; both refer to the same object, and the switch follows the convention of the public OLG reference implementation.)

where the network output $\hat\beta^h(\x_t)\in(0,1)$ is cohort $h$’s savings rate and $\hat a^h(\x_t)$ its savings level (slide II.9, output column). This parameterization mirrors the closed-form solution’s structure (each cohort saves a fixed fraction of income, Eq. (5.6)). Cohort $A$ saves nothing by terminal boundary, so the network has $A-1$ outputs rather than $A$. Three by-construction guarantees follow:

• Non-negativity of savings $\hat a^h \ge 0$ holds at every iteration, so the borrowing constraint (5.12) is satisfied without an explicit Lagrange multiplier (in this calibration $\lambda^h \equiv 0$ on the ergodic set; see Section 5.4 for the multiplier-based variant used in the 56-agent benchmark).

• Non-negativity of consumption $\hat c^h = (1-\hat\beta^h)\,\mathrm{inc}_t^h \ge 0$ also holds by construction, so the soft penalty on $\max(-\hat c^h,0)$ in the loss (next paragraph) is a dead backstop.

• Capital-market clearing $K_{t+1} = \sum_{h=2}^{A} k_{t+1}^h$ also holds by construction, since aggregate next-period capital is read off as the sum of the network’s savings outputs together with the newborn boundary $k_{t+1}^1 = 0$.

5.3.3Equilibrium residual.¶

Each cohort $h\in\{1,\ldots,A-1\}$ contributes one relative Euler-equation residual, built from three quantities. First, the implied current consumption, read off from the budget (5.2) as $\hat c^h(\x_t) := \mathrm{inc}_t^h - \hat a^h(\x_t)$. Second, the next-state map $\Phi$, which combines the current policy with a fresh shock $z_{t+1}$ to produce next period’s extended state $\hat\x_{t,+}=\Phi(\x_t,z_{t+1};\theta)$ (the construction of $\Phi$ is spelled out in the next paragraph). Third, the implied next-period consumption $\hat c^{h+1}(\hat\x_{t,+})$ of the cohort that has just aged from $h$ to $h+1$. The relative Euler-equation residual is then

with $u(c)=\ln c$ in the analytic model so $(u')^{-1}(y) = 1/y$. Equation (5.10) is the unit-free residual of the standard Euler equation (5.3): a value of 10^-3 means cohort $h$’s implied consumption is mispriced by $0.1\%$ relative to the conditional certainty equivalent. This is the residual displayed in slide II.7.

5.3.4Sampling the conditional expectation.¶

The expectation in (5.10) is over the next-period shock $z_{t+1}$. Because the analytic-model shock has only four states with i.i.d. transition $\pi_{ss'} = 1/4$, the expectation is computed exactly (no Monte Carlo) by summing over the four next-period shocks: $\E{\,r(\hat\x_{t,+})\,u'(\hat c^{h+1})\,} = \tfrac14\,\sum_{s'=1}^{4} r(\Phi(\x_t,s';\theta))\,u'\!\bigl(\hat c^{h+1}(\Phi(\x_t,s';\theta))\bigr).$ For each candidate $s'$ the next-state map $\Phi$ ages the cross-section by one period, sets the newborn to $k^1=0$, evaluates the firm prices (5.4) at $K_{t+1}$, and produces the next-period extended state $\hat\x_{t,+}$ on which the network is evaluated again to obtain $\hat a^h(\hat\x_{t,+})$ and hence $\hat c^{h+1}(\hat\x_{t,+})$. When the shock has more states or is continuous, the same construction is replaced by a sample of $z_{t+1}\sim\pi(\cdot|z_t)$ inside the mini-batch (see Section 5.5).

5.3.5The DEQN loss for the analytic OLG.¶

Given a mini-batch $D_{\mathrm{train}}\subset\{\x_j\}_{j=1}^{N}$ sampled from the ergodic set of the current policy, the loss is the mean-squared relative Euler residual averaged across cohorts and states:

(matching slide II.7). Two small barrier-style additive penalties on rescaled negative-consumption and negative-aggregate-capital hinges are summed in alongside (5.11) to keep training numerically robust away from convergence; in the notebook they carry the weight PENALTY_WEIGHT $=10$ and act on terms such as $\max(-\hat c^h,0)/(1+|\hat c^h|)$ rather than the raw squared hinge. With the sigmoid savings-fraction head described above these hinges are in fact identically zero (savings stay in $[0,\mathrm{inc}^h]$, so $\hat c^h\ge 0$ and $K_{t+1}\ge 0$ always), so the penalties are pure backstops and do not bias the solution.^[1]

5.3.6DEQN architecture and training.¶

The network takes a 40-dimensional input (the extended state (5.8), $16 + 4 \times 6$) and outputs 5 savings rates $\hat\beta^h$ via a $40 \to 100 \to 50 \to 5$ architecture with ReLU hidden layers and a sigmoid savings-fraction output ($\approx 9{,}400$ parameters). Training uses the episode-based procedure from Chapter Chapter 2: the current network generates a capital path (episode), equilibrium residuals are computed and used for SGD updates, and a new episode is simulated periodically. The companion notebook exposes a RUN_MODE switch with three calibrated budgets: "smoke" ($\sim$25 training segments, $\sim$30 s on CPU; a code-path sanity check, well short of convergence), "teaching" ($\sim$500 segments, $\sim$5 min on CPU; savings rates match the closed form to a few parts in 10⁴ and mean relative Euler errors are already $\sim 10^{-3}$ on the simulated cloud, though larger off-trajectory), and "production" ($\sim$10,000 segments with longer trajectories, several hours on CPU; mean Euler errors $\sim 10^{-3}$ or below, matching Table 3 of Azinovic et al. (2022)). Adam is used throughout (learning rate $\sim 3\times 10^{-4}$ in the short presets, 10^-5 in the production preset); the analogous decay to 10^-6 used by the 56-agent benchmark (Section 5.5) is not needed at the analytic model’s scale.

5.4Inequality Constraints and KKT Complementarity¶

The 6-agent calibration above is deliberately frictionless: the no-short-sale-of-capital constraint never binds on its ergodic set, so we could solve it with a plain sigmoid-savings head and no multipliers. Realistic OLG economies are not so kind. The 56-agent benchmark of the next section carries a no-short-sale-of-capital constraint that binds on a non-trivial slice of states and a collateral constraint that the current notebook parameterization keeps slack on the learned ergodic set; binding inequality constraints in general bring in Karush--Kuhn--Tucker (KKT) complementarity, with its characteristic non-smooth orthogonality condition. This section sets out how the DEQN framework encodes that complementarity; the next section puts it to work.

The no-short-sale-of-capital constraint $k'^h \geq 0$ introduces a complementarity condition via the Karush--Kuhn--Tucker (KKT) system:

where $\lambda^h$ is the KKT multiplier on the constraint. In a generic non-linear program, the orthogonality condition $k'^h \cdot \lambda^h = 0$ is non-smooth at the origin and cannot be differentiated through naively.

The DEQN setup of Azinovic et al. (2022) sidesteps the kink by splitting enforcement across the architecture and the loss:

• Hard side (architecture). The savings $k'^h$ and the multiplier $\lambda^h$ are both produced by the network through a softplus activation, so the inequalities $k'^h \geq 0$ and $\lambda^h \geq 0$ hold by construction at every iteration.

• Soft side (loss). With non-negativity already guaranteed, the orthogonality $k'^h \cdot \lambda^h = 0$ is enforced by adding the squared product residual $(k'^h \lambda^h)^2$ to the loss.

This product form is what the public reference implementation accompanying Azinovic et al. (2022) uses, and what we adopt in the 56-agent benchmark of Section 5.5 (Notebook lecture_08_10_OLG_Benchmark_DEQN_persistent.ipynb). As noted above, in the 6-agent analytic calibration of Section 5.2 the no-short-sale-of-capital constraint is non-binding everywhere on the ergodic set, so $\lambda^h \equiv 0$ and the multipliers (and the KKT residual) drop out of both the network output and the loss; that is why the mapping there was the simpler $\mathcal{N}_\theta: \mathbb{R}^{16+4A} \to \mathbb{R}^{A-1}$ of Section 5.3 above, with no multiplier outputs. The smoother Fischer--Burmeister (FB) reformulation, $\Phi(a,b) = a + b - \sqrt{a^2 + b^2}$, is an alternative used in the IRBC notebook of Chapter Chapter 3 for the investment-irreversibility constraint.

5.4.1When to choose product form vs. Fischer--Burmeister.¶

The product form $(k'^h \lambda^h)^2$ is simpler, gradient-cheaper, and sufficient whenever the constraint is rarely active on the ergodic set, since the optimizer just needs to verify slackness in expectation. The Fischer--Burmeister residual $\Phi(a,b)^2$ keeps gradient information on both sides of the active set: when the constraint is frequently binding (e.g. the IRBC irreversibility constraint on a non-trivial fraction of states), product-form gradients vanish whenever the constraint is locally inactive, which can stall training; FB does not have this pathology. As a rule of thumb: product form for occasionally-binding KKT, FB for frequently-binding KKT. In the OLG benchmark of Section 5.5 the no-short-sale-of-capital constraint binds on a thin slice of the ergodic set, so the product form was sufficient; the IRBC application of the previous chapter binds more often and benefits from FB.

5.4.2The two OLG models we solve, side by side.¶

We have now built and solved the first of the two OLG instances that anchor the rest of the manuscript: the 6-agent analytic model used to validate the DEQN against a closed form (Sections Section 5.2--Section 5.3). The second is the 56-agent benchmark of Azinovic et al. (2022), developed in the next section. Table Table 5.2 summarizes the structural and computational gap between them before we turn to it.

5.5The 56-Agent Benchmark¶

Table Table 5.2 above previewed the gap; we now develop the second model in full. The benchmark of Azinovic et al. (2022) scales the OLG framework to $A = 56$ agents (ages 25--80) with several realistic features:

• CRRA utility with $\gamma = 2$ (replacing log utility).

• Two assets: capital $k$ and one-period bonds $b$, with bond price $p$ determined in equilibrium.

• Hump-shaped labor endowment $e^h$ peaking in the early 50s.

• No-short-sale of capital: $k'^h \geq 0$ (the constraint historically labelled the “borrowing constraint” in this literature; we keep the more precise name to free “borrowing” for the bond side).

• Collateral constraint: $k'^h + \kappa\, b'^h \geq 0$, where $\kappa = 1/(1-\delta_{\max})$.

• Capital adjustment costs: $\Psi^h = \frac{\zeta}{2}(k'^h - r\cdot k^h)^2$.

• Persistent shocks: a 4-state Markov chain for TFP $\times$ depreciation (contrast with i.i.d. in the analytic model).

5.5.1Lifecycle labor endowments.¶

The labor endowment profile $e^h$ follows Brumm et al. (2017). In the implementation used here, $e^h$ is a quadratic in age that rises from 0.60 at age 25, peaks at $\approx 1.36$ around age 53, then decays linearly between ages $\sim 62$ and $\sim 70$ to a flat post-retirement floor of $\approx 0.64$. Table Table 5.3 lists the values produced by the notebook formula at a few representative ages.

This hump-shaped profile ensures realistic savings heterogeneity: young agents with low labor income and no initial wealth are borrowing-constrained, mid-career agents with high earnings accumulate both capital and bonds, and older agents gradually decumulate toward the end of life.

5.5.2Persistent aggregate shocks.¶

The 4-state Markov chain combines TFP $\eta$ and depreciation $\delta$ into the pairs $(\eta,\delta) \in \{(0.978, 0.08),\, (1.022, 0.08),\, (0.978, 0.11),\, (1.022, 0.11)\}$. The transition matrix is persistent (diagonal entries $\sim$0.63--0.88), in contrast to the i.i.d. shocks in the analytic model. This persistence creates richer dynamics in capital accumulation: a sequence of bad TFP draws can push young agents deep into their borrowing constraint, producing endogenous amplification that a single-period shock would not generate.

5.5.3Budget constraint.¶

Each agent of age $h$ faces:

The collateral constraint $k'^h + \kappa\,b'^h \geq 0$ acts as a margin requirement: it limits bond borrowing ($b'^h < 0$) relative to capital holdings. Since $\kappa = 1/(1-\delta_{\max})$, the constraint tightens when depreciation is high, precisely when agents are most likely to seek insurance through borrowing.

5.5.4State $\x_t$ entering the network.¶

The informational state of the benchmark is the triple $(z_t,\,\bm{k}_t,\,\bm{b}_t) \in \{1,\ldots,4\}\times\R^A\times\R^A$, where $\bm{k}_t = (k_t^1,\ldots,k_t^A)$ and $\bm{b}_t = (b_t^1,\ldots,b_t^A)$ are the cross-sectional capital and bond distributions, so the minimal state has dimension $1+2A = 113$. As in the analytic case, the notebook feeds the network an extended state of the same $16+4A$ form -- twelve aggregate scalars (shock index and its one-hot, $\eta_t$, $\delta_t$, $K_t$, $L_t$, $r_t$, $w_t$, and the gross resource $Y_t = \eta_t K_t^\alpha L_t^{1-\alpha} + (1-\delta_t)K_t$), four per-agent blocks ($k_t^h$, financial income $r_t k_t^h + b_t^h$, labor income $w_t e^h$, and cash $r_t k_t^h + b_t^h + w_t e^h$ -- the bond holdings $b_t^h$ are recoverable from financial income and are not passed as a separate block, and the bond price $\hat p_t$ is an output, not an input), and the row of next-period transition probabilities $\pi(z_t,\cdot)$ (used by the conditional-expectation block of the loss); concretely $240 = 12 + 4\times 56 + 4$ (the notebook constant FEATURE_DIM). This is the analogue of slide III.8.

5.5.5Policies approximated by the network.¶

A single network $\mathcal{N}_\theta$ with softplus output produces a $4(A-1)+1$-dimensional vector that is sliced into five economic blocks (slide III.9):

where $\hat k'^h$ are capital savings, $\hat\lambda_b^h$ the no-short-sale-of-capital multipliers, $\hat q^h \equiv \hat k'^h + \kappa\,\hat b'^h$ the collateral requirement (from which bond holdings are recovered as $\hat b'^h = (\hat q^h - \hat k'^h)/\kappa$), $\hat\mu^h$ the collateral-constraint multipliers, and $\hat p$ the equilibrium bond price. Each raw output is mapped to an admissible value: softplus for the multipliers, and a bounded-exponential map around a baseline for the positive levels. Concretely, writing $z^k_h$, $z^q_h$, and $z^p$ for the raw network outputs, the heads are

so the four non-negativity inequalities $\hat k'^h\ge 0$, $\hat\lambda_b^h\ge 0$, $\hat q^h\ge 0$, $\hat\mu^h\ge 0$ hold by construction, leaving the orthogonality conditions of the KKT systems to be enforced softly in the loss (next paragraph).^[2] The production network uses $1000\times 1000$ hidden units ($\sim\!1.5$M parameters); the teaching version uses $128\times 128$.

5.5.6Equilibrium residuals.¶

Each cohort $h\in\{1,\ldots,A-1\}$ contributes four residuals, one per equilibrium condition (slide III.6). To keep the displayed form compact, introduce numerator/denominator shorthands for the two Euler conditions:

Here $\mathcal{D}^h_k$ is the marginal-adjustment-cost wedge from $\Psi^h = \tfrac{\zeta}{2}(k'^h - r_t k^h)^2$: the capital Euler equation in envelope form reads $u'(c_t^h)\,\mathcal{D}^h_k = \beta\,\E{r_{t+1}\,\mathcal{D}^{h+1}_k\,u'(c_{t+1}^{h+1})} + \lambda_b^h + \mu^h$, so the same wedge appears next period on the marginal return to capital (this is the factor adj_factor_next in the notebook). With $\zeta = 0$ it collapses to the textbook Euler equation. The bond Euler reduces to the textbook stochastic-discount-factor form $\hat p = \beta\,\mathbb{E}[u'(c')]/u'(c)$ only when the collateral constraint is slack ($\hat\mu^h = 0$); whenever $\hat\mu^h > 0$, the bond price carries an additional shadow-value term $\kappa\hat\mu^h/u'(\hat c^h)$ that captures the value of relaxing the collateral constraint. The four per-cohort residuals are then

On top of these per-agent residuals the bond market must clear: bonds are in zero net supply, so the residual is the cross-sectional sum of bond holdings against the target $\bar B = 0$,

Capital-market clearing $K_{t+1} = \sum_{h=2}^{A} k_{t+1}^h$ is once again satisfied by construction and does not appear as a residual. The conditional expectation in the two Euler equations is computed exactly as in (5.10): by summing over the four next-period shocks weighted by the persistent-Markov transition probabilities $\pi(z_t,\cdot)$.

5.5.7The DEQN loss for the 56-agent benchmark.¶

Stack the four per-cohort residuals into one squared-cohort term $R^h(\x)^2 := (e_{\mathrm{REE},k}^h)^2 + (e_{\mathrm{REE},b}^h)^2 + (e_{\mathrm{KKT},b}^h)^2 + (e_{\mathrm{KKT},c}^h)^2,$ then add the bond-market-clearing residual. The mini-batch loss is

(matching slide III.6). With $A=56$ this is $4\times 55 + 1 = 221$ squared residuals per training state. Each residual enters with weight one: no adaptive loss balancing (cf. Chapter Chapter 4) is applied because the relative-Euler convention (5.10) already homogenizes the per-cohort Euler scales, and the product-form KKT residuals are unit-free under the softplus head; ReLoBRaLo or GradNorm would be the natural next step if a future calibration broke this homogeneity. Comparison with (5.11): the analytic case is the special instance of (5.19) in which the no-short-sale-of-capital constraint never binds (so $\lambda_b^h\equiv 0$), there are no bonds (so all $b$- and collateral-related blocks drop out), and $4(A-1)+1$ collapses to $A-1$. The two losses are the same template instantiated at different complexity. Table Table 5.4 unpacks the residual blocks.

5.5.8Training and results.¶

Production training uses 60,000 episodes at lr $= 10^{-5}$ followed by 140,000 episodes at lr $= 10^{-6}$, with runtime of several hours on GPU. The teaching version ($\sim$200 segments, 128-128 hidden units) runs in a few minutes on CPU and is meant to show the mechanics and qualitative lifecycle patterns, not final accuracy. The loss trajectory typically exhibits oscillations, caused by re-simulation of the capital path at each episode, but the overall trend is steadily downward.

5.5.9Lifecycle diagnostics.¶

The trained model produces economically plausible lifecycle patterns. Capital savings $k'^h$ follow a hump shape that mirrors the labor income profile: young agents save little (borrowing constraint binds), mid-career agents accumulate rapidly, and older agents decumulate. Bond holdings $b'^h$ are initially negative (young agents borrow against future income) and increase with age as agents shift from illiquid capital to liquid bonds. Bond prices vary across shock states, with higher prices in high-TFP states reflecting stronger demand for savings. In the teaching run the Euler residuals are still large enough to treat the output as diagnostic; in production runs the mean Euler equation errors are of order 10^-4--10^-3 for both capital and bond equations (matching Table 3 of Azinovic et al. (2022)), corresponding to a $\sim$0.01%--0.1% deviation in consumption. Market clearing residuals are comparably small. Convergence is also confirmed by the policy-drift check on the fixed anchor cloud: the run is treated as time-invariant once policy_drift_rms and policy_drift_max fall below their prescribed tolerances.

5.6Further Reading¶

• Azinovic et al. (2022), the IER paper that established the 56-agent benchmark.

• Auerbach & Kotlikoff (1987), the classical OLG reference.

• Azinovic-Yang & Žemlička (2024), market-clearing output layer in OLG with rare disasters.

5.7Exercises¶

Worked solutions and guidance for these exercises appear in Appendix Appendix F.