F Solutions and Guidance for Exercises

This appendix provides worked solutions for analytical end-of-chapter exercises and guidance for coding exercises. Coding exercises (those that ask the reader to modify a notebook or measure a numerical quantity) are not treated as fixed numerical answers; their outputs depend on hardware, seeds, calibration choices, and notebook versions, and should be reported from the corresponding companion notebook listed in the Execution Map. For exercises that mix analytical work with a small numerical check, the analytical part is solved and the numerical component is described as a verification task.

Exercises are referenced by their stable label ex:ch$N$:$M$, where $N$ is the chapter number and $M$ is the position within that chapter’s exercise list. Each solution opens with a back-pointer to the exercise statement so the reader can scan the question first.

F.0.1Chapter Chapter 1: Introduction to Machine Learning and Deep Learning¶

F.0.1.1Exercise 1.1: Backprop on a 2-layer net.¶

Let $z = w_1 x + b_1$, $a = \mathrm{ReLU}(z)$, $\hat y = w_2 a$, $\ell = (\hat y - y)^2$. Reverse-mode chain rule:

Plugging in $x=2$, $y=1$, $w_1 = w_2 = b_1 = 0.5$: $z = 1.5$, $a = 1.5$, $\hat y = 0.75$, $\ell = 0.0625$. Hence $\partial\ell/\partial w_2 = 2(-0.25)(1.5) = -0.75$ and $\partial\ell/\partial w_1 = 2(-0.25)(0.5)(1)(2) = -0.5$. A two-line PyTorch script (torch.autograd.grad) returns the same numbers to machine precision; this is the simplest non-trivial sanity check that the chain-rule derivation matches what AD computes.

F.0.1.2Exercise 1.2: MSE vs. MLE.¶

For Gaussian errors $\varepsilon_i \sim \mathcal{N}(0, \sigma^2)$, the log-likelihood is

Maximizing over $\beta$ (the only $\beta$-dependent term) is identical to minimizing $\sum_i (y_i - \beta x_i)^2$, i.e. OLS. For Laplace errors with scale $b$, $p(\varepsilon) \propto \exp(-|\varepsilon|/b)$, so the log-likelihood is proportional to $-\sum_i |y_i - \beta x_i|$ and the MLE solves a least-absolute-deviations regression (median regression). Squaring penalizes outliers quadratically, which is suboptimal under Laplace tails because a single large residual dominates the loss; the median estimator is robust precisely because $|\cdot|$ grows linearly.

F.0.1.3Exercise 1.3: Activation choice for a PINN.¶

A ReLU network $\hat y(x) = \sum_k w_k\,\mathrm{ReLU}(a_k x + b_k) + c$ is piecewise-linear: between consecutive kink points $x = -b_k/a_k$ it is affine in $x$, so $\hat y''(x) = 0$ on every open subinterval. At a kink, the second distributional derivative is a Dirac measure, not a classical pointwise value. A strong-form PDE residual such as the Black--Scholes operator

must hold pointwise (a.e.) for almost every $S$; with a piecewise-linear $V$, the term $V_{SS}$ vanishes a.e. in the classical sense, and the residual reduces to $\partial_t V + r S V_S - rV$, which has no nontrivial solution that respects the actual boundary conditions of an option-pricing problem. Concrete example: $\hat y(x) = \mathrm{ReLU}(x) + \mathrm{ReLU}(-x) = |x|$ has $\hat y''(x) = 0$ for every $x \neq 0$ in the classical sense. Smooth activations (tanh, Swish, GELU, softplus) are $C^\infty$ and produce well-defined pointwise second derivatives, which is why the PINN literature recommends them whenever the PDE is of order $\ge 2$.

F.0.1.4Exercise 1.4: Adam vs. AdamW.¶

Write the gradient at step $t$ as $g_t = \nabla_\theta \ell(\theta_t)$. With $L_2$ regularization added to the loss, $\ell^{L_2}(\theta) = \ell(\theta) + \tfrac{\lambda}{2}\|\theta\|^2$, so the effective gradient becomes $\tilde g_t = g_t + \lambda \theta_t$. Adam’s update applied to $\tilde g_t$ is

The key observation is that the weight-decay contribution $\lambda \theta_t$ enters $m_t$ and $v_t$ through the same EWMA averaging as the data gradient, so the effective shrinkage applied to $\theta$ is $\eta\lambda \theta_t$ scaled by $1/(\sqrt{\hat v_t}+\varepsilon)$, which differs across coordinates. Parameters with large historical gradients receive small effective decay; parameters with small gradients receive large decay. AdamW decouples the two:

Now the multiplicative shrinkage $(1 - \eta\lambda)$ acts uniformly on all parameters, independent of the adaptive denominator. Numerically the two updates differ by a factor of $1/(\sqrt{\hat v_t}+\varepsilon)$ on the decay term; AdamW preserves the textbook intuition “weight decay shrinks weights at the same rate everywhere.”

F.0.1.5Exercise 1.5: RNN forward pass by hand.¶

With $W_h = 0.5\,I_2$, $W_x = (1,0)^\top$, $h_0 = (0,0)^\top$:

Outputs: $\hat y_t = W_y h_t$ gives $\hat y_1 \approx 0.7616$, $\hat y_2 \approx 0.3637$, $\hat y_3 \approx 0.8275$ (only the first hidden coordinate is excited, so the second column of $W_y$ is irrelevant).

For the gradient, write $h_t = \tanh(z_t)$ with $z_t = W_h h_{t-1} + W_x x_t$. Then

where $\partial h_t/\partial z_t = \mathrm{diag}(1-\tanh^2(z_t))$. Numerically, $\tanh'(1)\approx 0.4200$, $\tanh'(0.3808)\approx 0.8678$, $\tanh'(1.1818)\approx 0.3155$, so

The decay rate is the product of two distinct effects: (i) the spectral radius of $W_h$ (here 0.5), which contributes one factor of $W_h$ per recurrent step ($T-1$ multiplicative copies in the chain above), and (ii) the saturation of $\tanh'(\cdot) \in (0,1]$, which contributes a second multiplicative factor per step. Setting (ii) aside, with $\|W_h\|_2 = 0.5 < 1$ the gradient magnitude already decays at least as fast as $0.5^{T-1}$, i.e. exponentially in the sequence length. This is the canonical vanishing-gradient pathology of vanilla RNNs (Section 1.11). LSTM and GRU gates address (i) by allowing the recurrent Jacobian’s eigenvalues to stay close to one without making training unstable; effect (ii) is intrinsic to bounded activations and is mitigated by skip connections (and architectural choices that keep activations away from saturation regimes), not by gating.

F.0.1.6Exercise 1.6: Attention by hand.¶

With $q_i = k_i = v_i = x_i$ and $x = (0,1,0.5)$, the score matrix $S_{ij} = q_i k_j$ is

Softmaxing each row and using $e^0 = 1$, $e^{0.25} \approx 1.284$, $e^{0.5}\approx 1.649$, $e^1 \approx 2.718$:

Each attention vector $a_i$ is on the simplex (entries non-negative, summing to one), so $o_i = \sum_j a_{ij} v_j$ is a convex combination of the values, lying in $[0, 1]$. Token 2, the largest input, attends most strongly to itself ($a_{22}\approx 0.51$); token 3 also attends most to token 2 because the inner products $q_3 k_2 = 0.5$ exceed the self-score $q_3 k_3 = 0.25$. Token 1 has zero query magnitude, so its row is uniform: with no signal to discriminate on, attention defaults to a uniform average over the values.

F.0.1.7Exercise 1.7: TensorBoard optimizer comparison.¶

Coding exercise. Expected qualitative behavior on a small classification task: SGD with momentum is often slower to reduce training loss but can generalize competitively when the learning-rate schedule is well tuned; Adam often reaches a low training loss fastest, but its validation curve can diverge from the training curve sooner than for SGD or AdamW; AdamW often sits between the two. The actual crossover point is a notebook output and should be reported from the run.

F.0.2Chapter Chapter 2: Deep Equilibrium Nets¶

F.0.2.1Exercise 2.1: Closed-form Brock--Mirman.¶

Conjecture $V(K, z) = A\log K + B\log z + C$. The Bellman equation

yields the FOC $1/(zK^\alpha - K') = \beta A/K'$, which gives the constant savings rate

Substituting the conjecture back and matching the $\log K$ coefficient produces $A = \alpha + \beta A\alpha$, hence $A = \alpha/(1-\alpha\beta)$. Plugging this $A$ into $s^\star$:

The DEQN parameterizes $s_t = \sigma\!\bigl(\mathcal{N}_\rho(K_t, z_t)\bigr)$. Once converged, the average sigmoid output across the ergodic set should equal $\alpha\beta$ (typical calibrations $\alpha=0.36$, $\beta=0.96$ give $s^\star \approx 0.346$). Any systematic deviation indicates either insufficient training or a quadrature / sampling bias.

F.0.2.2Exercise 2.2: Hard vs. soft constraints.¶

With a softplus head on consumption alone, $C_t = \mathrm{softplus}(\mathcal{N}_\rho(K_t, z_t)) > 0$ is guaranteed, but next-period capital is then defined as the residual $K_{t+1} = z_t K_t^\alpha - C_t$, which is unconstrained in sign. At random initialization the network output is approximately $\mathcal{N}(0, 1)$, so $C_t$ is approximately $\mathrm{softplus}(0) \approx 0.69$ on average but can be much larger.

Concrete failure: take $K = 1$, $z = 1$, $\alpha = 0.36$, so $zK^\alpha = 1$. If the network produces an output of, say, 5, then $C = \mathrm{softplus}(5) \approx 5.007$ and $K_{t+1} = 1 - 5.007 = -4.007 < 0$. The next iterate $K_{t+1}^{\alpha-1}$ is then complex, and the loss explodes.

The sigmoid-savings parameterization $s_t = \sigma(\mathcal{N}_\rho) \in (0,1)$ avoids this entirely: $K_{t+1} = s_t z_t K_t^\alpha > 0$ and $C_t = (1-s_t) z_t K_t^\alpha > 0$ both hold by construction, regardless of the network’s raw output. This is the simplest example of the broader principle that architectural feasibility encodings dominate loss-based ones whenever the constraint can be written as a closed-form algebraic identity.

F.0.2.3Exercise 2.3: Path averaging vs. conditional expectation.¶

On the simulated path, the path-averaged residual is $\bar r_T(\theta) = T^{-1}\sum_{t=1}^T r(\theta, x_t)$ with $\{x_t\}$ generated by the model dynamics. Under ergodicity, Birkhoff’s theorem gives

where $\mu$ is the stationary distribution. The conditional-expectation residual at a fixed point $x_t$, $\E[r(\theta, x_{t+1}) \mid x_t]$, is a different object: it integrates only over the conditional shock distribution, holding the current state fixed.

Their connection: if one sweeps the conditional residual over $x_t \sim \mu$ and averages, the result coincides with $\mathbb{E}_\mu[r(\theta, x)]$ by the law of iterated expectations. In other words, path averaging combines sampling over states (the outer expectation) and the implicit Monte Carlo integration over shocks (one shock per simulated step). Conditional expectation evaluates the inner integral exactly via quadrature but still requires a way to draw states $x_t$.

Bias--variance trade-off at finite $T_{\text{sim}}$: the path-averaged loss has variance $O(1/T_{\text{sim}})$ from finite-sample noise but no bias. An exact-quadrature residual has zero stochastic noise on the inner integral but its outer-state coverage depends on the sampling scheme; with mini-batch SGD over the ergodic set, the variance comes from the random batch, not the integration. In the chapter the deterministic quadrature is preferred whenever the shock dimension is low ($d \lesssim 5$), and pathwise residuals dominate once $d$ is high enough that explicit quadrature becomes expensive.

F.0.2.4Exercise 2.4: Brock--Mirman with Gauss--Hermite.¶

The Euler equation in Brock--Mirman with $\delta = 1$, log utility, and AR(1) productivity $\ln z' = \varrho\ln z + \sigma_z\varepsilon'$, $\varepsilon' \sim \mathcal{N}(0,1)$, is

Replace the expectation by a Gauss--Hermite rule. After the change of variables $\varepsilon' = \sqrt{2}\,\xi$ that absorbs the normalization $1/\sqrt{\pi}$, the $Q$-point GH quadrature is

with classical nodes $\xi_q$ and weights $w_q$ that satisfy $\sum_q w_q = \sqrt{\pi}$. Table Table F.1 lists the five-point rule used in this exercise.

Substituting, the residual at a state $(K_t, z_t)$ becomes

where $z_q' = \exp(\varrho\ln z_t + \sigma_z\sqrt{2}\,\xi_q)$ and $C_{t+1}^q$ is the consumption obtained by feeding $(K_{t+1}, z_q')$ into the network. Numerically, comparing this 5-point sum with a high-draw Monte Carlo estimate on the same residual is a useful diagnostic: agreement should be close for smooth integrands and small shock variance, but the realized discrepancy is an output of the check rather than a fixed benchmark.

F.0.2.5Exercise 2.5: Monomial rule by hand (Stroud-3 at $d=4$).¶

Equation (2.18) places nodes at $\bm{x}_k^\pm = \pm\sqrt{d}\,\bm{e}_k$ for $k = 1, \dots, d$, with equal weights $1/(2d) = 1/8$. At $d=4$, the eight nodes are $\pm 2 \bm{e}_k$ for $k=1,2,3,4$.

First moment. $\E[\varepsilon_i']_{\text{rule}} = (1/8)\sum_k [(\sqrt{d}\,\bm{e}_k)_i + (-\sqrt{d}\,\bm{e}_k)_i] = 0$ by ±-cancellation.

Second moment. $(\varepsilon_i')^2$ is non-zero only at $\pm\sqrt{d}\,\bm{e}_i$, where it equals $d$. Two such nodes contribute $2d$, weighted by $1/(2d)$, giving exactly 1.

Cross moment. $\varepsilon_i'\varepsilon_j'$ for $i \neq j$ is zero at every node because each node has only one nonzero coordinate. So the rule returns 0, the true value.

Third moment. $(\varepsilon_i')^3$ at $\pm\sqrt{d}\bm{e}_i$ equals $\pm d^{3/2}$; the two values cancel. Returns 0, exact.

Fourth moment. $(\varepsilon_i')^4$ at $\pm\sqrt{d}\bm{e}_i$ equals $d^2$, both signs. Two nodes contribute $2d^2$, weighted by $1/(2d)$, gives $d$. At $d=4$, the rule returns 4, while the true value $\E[\varepsilon_i'^{\,4}] = 3$.

Linear bias growth. In general the rule reports $d$ for the fourth moment, so the relative error $(d - 3)/3$ is linear in $d$: $33\%$ at $d=4$, $67\%$ at $d=6$, doubles to 1 ($100\%$) at $d=9$.

When does this matter? The Euler residual is a smooth function of the next-period shock $\varepsilon'$. Taylor-expanding around the conditional mean, the leading bias term is the Hessian of the residual with respect to $\varepsilon'$, which probes the second moment, exact under Stroud-3. Fourth-moment bias enters only at the next order, scaled by the integrand’s fourth derivative. For moderate CRRA curvature and thin-tailed shocks this term can be small relative to classroom residual tolerances, but it is a diagnostic to check rather than a universal bound. The bias becomes material when (i) the integrand has heavy fourth-order content (e.g., very risk-averse preferences or fat-tailed shocks), or (ii) the shock dimension is large enough that the relative error $(d-3)/3$ exceeds the residual tolerance one targets. This is the threshold at which the monomial rule should be replaced by Stroud-5 ($2d^2 + 1$ nodes) or QMC.

F.0.2.6Exercise 2.6: Loss-kernel selection.¶

Definitions for reference. MSE: $\frac{1}{N}\sum_i r_i^2$. MAE: $\frac{1}{N}\sum_i |r_i|$. Huber($\delta$): quadratic for $|r| \le \delta$, linear above. Pinball loss at $\tau$: $L_\tau(r) = \max(\tau r,\, (\tau - 1)r)$, whose minimizer is the $\tau$-quantile of $r$. CVaR at $\alpha$: the expected value of $r$ conditional on $r$ exceeding its $\alpha$-quantile. Log-cosh: $\sum_i \log\cosh(r_i)$, smooth and quadratic near zero, linear in tails.

(a) Huber loss: “smooth, quadratic near zero, linear in tails” is the literal definition. MAE is also linear-in-tails but is not differentiable at $r=0$, so its gradient flips discontinuously and the optimizer stalls; log-cosh is smooth and shares the asymptotic shape with Huber but has no tunable threshold. Huber($\delta$) gives the cleanest control of where the regime change happens.

(b) CVaR at $\alpha = 0.99$. The CVaR loss optimizes the conditional mean above the 99th-percentile threshold, which is exactly what the regulator audits. The pinball loss at $\tau = 0.99$ would only target the 99th-percentile residual itself, not the conditional average above it; if the residuals have a fat right tail, the pinball-trained policy can have arbitrarily large worst-case $1\%$ residuals. CVaR is the right primitive for “worst-case mean” control.

(c) MAE (or equivalently pinball at $\tau = 0.5$). By construction MAE’s first-order condition is solved at the median, not the mean: $\partial/\partial r |r| = \mathrm{sign}(r)$, so the gradient contribution is $\pm 1$ per residual, regardless of magnitude. This is exactly what the desideratum asks for, no single tail residual dominates the gradient. Huber would also down-weight tails but still tracks the mean below the threshold; the user explicitly wants median-targeting.

F.0.2.7Exercise 2.7 and Exercise 2.8 (statements: p. , p. ).¶

These are coding exercises (notebook lecture_03_02_Brock_Mirman_Uncertainty_DEQN.ipynb); reference outputs and timing curves are in the companion repository. Qualitative anchors: in Ex. Exercise 2.7, the per-epoch wall time of tensor-product Gauss--Hermite ($Q^d$ nodes) should grow exponentially in $d$ while Stroud-3 ($2d$ nodes) grows linearly, with a crossover that is visible already at $d=4$--5 on a single GPU; the relative Euler error should track the integration accuracy of each rule, with Stroud-3 inheriting the fourth-moment bias of Section 2.6.3. In Ex. Exercise 2.8, swapping Swish for $\tanh$ typically slows time-to-converge by tens of percent on a smooth problem like Brock--Mirman because $\tanh$ saturates faster, but final accuracy is comparable; convergence should still hold under the same hyperparameters.

F.0.3Chapter Chapter 3: The International Real Business Cycle Model¶

F.0.3.1Exercise 3.1: Fischer--Burmeister.¶

For the forward direction, suppose $a \ge 0$, $b \ge 0$, $ab = 0$. Without loss of generality $b = 0$; then $\Phi(a, 0) = a + 0 - \sqrt{a^2 + 0} = a - |a| = 0$ since $a \ge 0$. By symmetry the same holds when $a = 0$.

For the reverse direction, suppose $\Phi(a,b) = 0$, i.e. $a + b = \sqrt{a^2 + b^2}$. The right-hand side is non-negative, so $a + b \ge 0$. Squaring: $(a+b)^2 = a^2 + b^2 \Rightarrow 2ab = 0 \Rightarrow ab = 0$. Combined with $a + b \ge 0$ and $ab = 0$, the only possibility is one of $a, b$ being zero and the other non-negative, i.e. $a, b \ge 0$ and $ab = 0$. $\square$

The level set $\Phi(a, b) = 0$ is the union of the non-negative $a$-axis and the non-negative $b$-axis (an L-shape in the $(a,b)$ plane). The smoothed variant $\Phi_\varepsilon(a,b) = a + b - \sqrt{a^2 + b^2 + \varepsilon^2}$ rounds the corner at the origin: at $(a,b) = (0,0)$ one finds $\Phi_\varepsilon(0,0) = -\varepsilon \neq 0$, while far from the origin $\sqrt{a^2 + b^2 + \varepsilon^2} \approx \sqrt{a^2 + b^2}$ and the smoothed level set converges to the unsmoothed L-shape. In a numerical setting the smoothing eliminates the gradient kink at the origin, which is convenient for AD but distorts the strict KKT zero set by an $O(\varepsilon)$ amount.

(c) Gradient direction. At an interior point of the open positive quadrant, $\nabla\Phi(a,b) = (1 - a/\sqrt{a^2+b^2},\, 1 - b/\sqrt{a^2+b^2})$. At $(a,b) = (1,1)$ this evaluates to $(1-1/\sqrt 2,\, 1-1/\sqrt 2) \approx (0.293, 0.293)$, both components strictly positive. The raw gradient $\nabla\Phi$ therefore points northeast, away from the L-shaped zero set, so the bare residual $\Phi$ on its own is not the right object for SGD to descend on. What does descend toward the zero set is the squared loss $\Phi^2$: by the chain rule $\nabla(\Phi^2) = 2\Phi\,\nabla\Phi$, and inside the open positive quadrant $\Phi(a,b) > 0$, so $-\nabla(\Phi^2) = -2\Phi\,\nabla\Phi$ has both components strictly negative at $(1,1)$ and points southwest, i.e. back toward the closer feasible axis. This is the operative observation for training: SGD on $\Phi^2$ is what pulls infeasible iterates back to the L; the raw gradient $\nabla\Phi$ on its own would push them away.

(d) Sign convention. Replacing $\Phi$ by $-\Phi$ leaves the squared loss untouched: $(-\Phi)^2 = \Phi^2$, so the gradient field of the loss and the SGD trajectory are unchanged. In particular, the sign convention $a + b - \sqrt{a^2+b^2}$ versus $\sqrt{a^2+b^2} - a - b$ is irrelevant once the residual is squared, and the operative quantity for SGD is $-\nabla(\Phi^2)$ rather than $-\nabla\Phi$. The L-shaped zero set is invariant under sign flip; only the value of $\Phi$ at off-zero points changes (it flips sign), and squaring removes that distinction.

F.0.3.2Exercise 3.2: State-space scaling.¶

For an IRBC with $N$ symmetric countries, the state vector contains $(K^j, z^j)_{j=1}^N$ for a total of $2N$ components. The network outputs the next-period capital vector $(k^{j\prime})_{j=1}^N$, the irreversibility multipliers $(\mu^j)_{j=1}^N$, and the resource-constraint shadow price $\lambda$, for $2N + 1$ outputs in total. Country-level consumption is recovered algebraically from $\lambda$ via the consumption-sharing FOC and is therefore not a separate output. The loss has $N$ Euler residuals, $N$ Fischer--Burmeister residuals from the irreversibility complementarity, and 1 aggregate-resource-constraint residual, $2N + 1$ components total, matching the $2N + 1$ outputs. Each Euler residual is an expectation over a $(N+1)$-dimensional shock vector (one country-specific innovation plus one aggregate, as in equation (3.4)).

Tensor-product Gauss--Hermite at $Q=3$ costs $3^{N+1}$ evaluations per residual. Setting $3^{N+1} > 10^4$ gives $N + 1 > \log_3(10^4) \approx 8.38$, so $N \ge 8$ already exceeds the threshold, and $N = 9$ overshoots by a factor of nearly 6 (cost $3^{10} = 59\,049$).

The Stroud-3 rule has $2(N+1)$ nodes per residual. Setting $2(N+1) < 100$ gives $N \le 48$, comfortable for any IRBC dimension actually used in practice. At $N = 50$ the monomial cost is 102 nodes; the tensor cost is $3^{51} \approx 2.15 \times 10^{24}$ evaluations. This four-order-of-magnitude gap at $N = 10$ and twenty-order-of-magnitude gap at $N = 50$ is the operational reason DEQNs use Stroud-3 by default once $N \gtrsim 5$.

F.0.3.3Exercise 3.3: Two-phase training.¶

At a randomly initialized network, the policy outputs $k^{j\prime}$ are not coordinated with the resource constraint. Suppose the network produces $k^{j\prime}$ values that, summed and combined with country-$j$ consumption, exceed total output: $\sum_j (k^{j\prime} + c^j + \Gamma^j) > \sum_j Y^j$. The implied state on the next simulated step has $k^{j\prime} < (1-\delta) k^j$ for some country (irreversibility violated), or $c^j < 0$ (negative consumption), or both. Concretely, take $N = 2$, $A_\mathrm{tfp} = 1$, $\delta = 0.025$, $k^1 = k^2 = 1$, and a random network output $(k^{1\prime}, k^{2\prime}) = (0.5, 0.5)$ (instead of the symmetric $(1, 1)$ steady state). Capital has dropped by $50\%$ in one step, so $I^j = k^{j\prime} - (1-\delta)k^j = 0.5 - 0.975 = -0.475 < 0$, violating irreversibility. Even before the irreversibility check fires, the implied consumption $c^j = Y^j - I^j - \Gamma^j$ becomes huge, and in the next step the marginal utility $u'(c)$ is essentially zero, so the Euler residual gradient is uninformative.

Phase 1 (uniform sampling on a wide box of states, with Euler residuals computed against any feasible policy guess) gives the optimizer signal to bring outputs into the feasible region before any simulation is attempted. Once the policy is in a feasible neighbourhood, Phase 2 (simulation-based sampling on the ergodic set) refines accuracy. Without Phase 1, the simulation in Phase 2 starts in regions where the policy is grossly infeasible, the loss explodes, and gradient descent diverges.

F.0.3.4Exercise 3.4: Adjustment-cost partials and Tobin’s Q.¶

Write $g^j \equiv k^{j\prime}/k^j - 1$. Then $\Gamma^j = (\kappa/2)\,k^j (g^j)^2$. The partials are

matching equation (3.6).

At the steady state, $k^{j\prime} = k^j$ so $g^j = 0$. Both partials vanish: $\partial\Gamma^j/\partial k^{j\prime} = 0$ and $\partial\Gamma^j/\partial k^j = 0$. The adjustment cost itself $\Gamma^j(k^j, k^j) = 0$, and its first-order presence in the resource constraint and the Euler equation drops out. The deterministic steady state of the IRBC with adjustment costs is therefore identical to the frictionless steady state, $k_\mathrm{ss}$ being pinned down by $\beta(1 - \delta + \zeta A_\mathrm{tfp} k^{\zeta-1}) = 1$.

The expression $\partial\Gamma^j/\partial k^{j\prime} = \kappa g^j$ is the marginal cost of investing one more unit and is the IRBC’s analogue of Tobin’s marginal Q: large positive $g^j$ (rapid expansion) creates a large investment wedge, raising the effective per-unit cost of capital next period. Higher $\kappa$ flattens the response of investment to a productivity shock: a large adjustment cost makes the planner spread the response of $k^{j\prime}$ across several periods rather than absorbing the shock in one big move, slowing convergence to the new steady state. In the limit $\kappa \to \infty$, capital adjusts only infinitesimally each period and the dynamics become arbitrarily slow.

F.0.3.5Exercise 3.5: Complete-markets risk sharing.¶

The consumption-sharing condition (3.12) reads $c_t^j = (\lambda_t/\tau^j)^{-\gamma_j}$. Therefore

(i) With homogeneous IES $\gamma_i = \gamma_j = \gamma$, the $\lambda_t$ exponent vanishes and the ratio collapses to a time-invariant constant:

Since $\log(c_t^i/c_t^j)$ is constant, $\Delta\log c_t^i = \Delta\log c_t^j$, the perfect risk-sharing prediction of Backus et al. (1992): cross-country consumption growth is perfectly correlated (correlation $= 1$).

(ii) With heterogeneous IES $\gamma_i \neq \gamma_j$, the exponent $\gamma_j - \gamma_i$ is non-zero and $\lambda_t$ enters the ratio. As shocks move the planner’s shadow price, the consumption ratio fluctuates: low-IES countries’ consumption is less sensitive to $\lambda_t$ than high-IES countries’. But the log growth rate is still

Thus, for positive $\gamma_i,\gamma_j$, any pair of country consumption-growth rates is a positive scalar multiple of the same aggregate shock $\Delta\log\lambda_t$. The correlation remains one; heterogeneous IES changes relative consumption-growth volatility, not the correlation, in this complete-markets planner allocation.

(iii) The empirical consumption-correlation puzzle is that real-world cross-country consumption growth correlations sit well below the near-perfect correlations implied by the complete-markets benchmark. Heterogeneous IES alone does not break the common-shadow-price structure; closing the gap with the data requires incomplete markets, wedges, nontraded goods, preference nonseparabilities, or other frictions that break full insurance.

F.0.3.6Exercise 3.6 and Exercise 3.7 (statements: p. , p. ).¶

These are notebook exercises (lecture_05_05_IRBC_Exercise.ipynb); reference solutions are in the notebook itself, behind the “attempt first” dividers. Qualitative anchors: in Ex. Exercise 3.6, the closed-form steady state $k_\mathrm{ss} = \bigl[(1/\beta - 1 + \delta)/(\zeta A_\mathrm{tfp})\bigr]^{1/(\zeta-1)}$ falls in all three scenarios --- a higher $\delta$ raises the required net return $1/\beta - 1 + \delta$, a lower $\beta$ raises $1/\beta$, and a lower $\zeta$ shrinks the multiplicative term while (since $\zeta - 1 < 0$) steepening diminishing returns --- so $k_\mathrm{ss}$ is decreasing in $\delta$, increasing in $\beta$, and increasing in $\zeta$ around the baseline, with $c_\mathrm{ss} = A_\mathrm{tfp} k_\mathrm{ss}^\zeta - \delta k_\mathrm{ss}$ following by substitution. In Ex. Exercise 3.7, inverse-loss weighting $w_i \propto 1/\ell_i$ equalizes the per-component contributions $w_i \ell_i$, so the smallest-magnitude residuals (here the Fischer--Burmeister terms) receive the largest weight; the speed-up is largest when a component is small because it is hard to fit, and the scheme can hurt when a component is small because it is already satisfied by construction (e.g., a hard-coded resource constraint), in which case up-weighting it merely amplifies noise.

F.0.4Chapter Chapter 4: Neural Architecture Search and Loss Normalization¶

F.0.4.1Exercise 4.1: Random vs. grid.¶

With two hyperparameters and only one “important” axis, a $3\times 3$ grid uses 9 candidates but only 3 distinct values along the important axis. If the near-optimal interval has length fraction $p$ and its location relative to the grid is unknown, the grid hit probability is approximately $\min\{3p,1\}$ when $p$ is small. Random search at 9 evaluations samples 9 independent values along the important axis, so its hit probability is

For $p=0.05$, the grid probability is approximately 0.15, while random search gives $1-0.95^9\approx 0.37$.

The general principle: with $n$ evaluations and only $r \ll d$ important axes, random search effectively gives $n$ independent draws on those $r$ axes (since the irrelevant axes don’t matter), while grid search wastes most of its budget on the irrelevant axes’ marginals. This is the projection argument of Bergstra & Bengio (2012): when the loss landscape is anisotropic, random search dominates grid search.

F.0.4.2Exercise 4.2: Bayesian optimization toy problem.¶

This is a coding exercise. A grid with step size 0.01 over $[-1,2]$ has 301 points, so the qualitative benchmark is that BO should require far fewer objective evaluations when the GP posterior and acquisition function identify the promising region early. The exact evaluation count is a notebook output and depends on the initial design, acquisition optimizer, random seed, and stopping rule.

F.0.4.3Exercise 4.3: Hyperband budget allocation.¶

Hyperband with $R = 81$, $\eta = 3$ runs a ladder of brackets indexed by $s = s_{\max}, s_{\max}-1, \dots, 0$, where $s_{\max} = \lfloor\log_\eta R\rfloor = 4$. Each bracket starts with $n_s = \lceil (s_{\max}+1)\,\eta^s / (s+1)\rceil$ candidates trained for $r_s = R / \eta^s$ resource each, then runs Successive Halving with reduction factor $\eta$. Table Table F.2 works out the resulting schedule.

Within each bracket, Successive Halving reduces candidates by $\eta$ at each rung and increases the resource per surviving candidate by $\eta$. Therefore the total cost must sum all rungs, not only the first rung. With the floor/ceil schedule above, total Hyperband budget is

resource units, just below the loose worst-case bound $(s_{\max}+1)^2 R = 25 \cdot 81 = 2025$.

A naive “train all $n_0 = 27$ candidates to full $R = 81$” costs $27 \cdot 81 = 2187$ resource units. Hyperband is only moderately cheaper in total resource here, but it screens a much larger initial pool: across the five brackets, the total number of first-rung candidates is $81 + 34 + 15 + 8 + 5 = 143$, vs. 27 for the naive scheme. Its advantage is adaptive allocation: many more candidates are sampled, but only a small subset receives large training budgets.

F.0.4.4Exercise 4.4: Loss balancing.¶

Let $\ell_i^{(t)}$ have magnitude $L_i \in \{10^0, 10^{-2}, 10^{-4}\}$ and per-component gradient norm $\|\nabla\ell_i\|$. If we crudely model gradient norm as scaling linearly with loss magnitude (true for, e.g., quadratic losses far from the optimum), then $\|\nabla \ell_i\| \propto L_i$. Equalising gradient contributions $\lambda_i \|\nabla\ell_i\|$ requires $\lambda_i \propto 1/L_i$, i.e. $(\lambda_1, \lambda_2, \lambda_3) \propto (1, 10^2, 10^4)$, which after normalisation becomes

The scheme breaks down when the gradients are correlated: $\langle \nabla\ell_i, \nabla\ell_j\rangle \neq 0$ means that scaling up $\lambda_3$ to “boost” $\ell_3$ also moves $\theta$ along the $\nabla\ell_1$ direction, changing $\ell_1$. The “equal contribution” targeted by the fixed weights is no longer a fixed point: each parameter update changes the local gradient geometry, and weights tuned at one iteration become wrong at the next. Adaptive schemes (ReLoBRaLo, GradNorm) re-tune the $\lambda_i$ at every step, recovering the equalisation in a way that fixed weights cannot.

F.0.4.5Exercise 4.5: Pareto frontier geometry.¶

(i) Differentiate $\mathcal{L}(\theta;\lambda) = \lambda(\theta-a)^2 + (1-\lambda)(\theta-b)^2$ in $\theta$ and set to zero: $2\lambda(\theta - a) + 2(1-\lambda)(\theta - b) = 0$, hence