DEFINE-UK ecological stock-flow
DEFINE-UK
The replication gates, the baseline against official numbers, the multiplier, and the ceiling.
Partial replication; deltas only
The macro block matches Table 4 within every gate (the 2025 growth gate was set at that miss); 2025 growth (−0.30pp, gate ±0.31) and 2040 unemployment (−0.19pp, gate ±0.2) clear by only 0.01pp. No numeric v1.1 scenario results exist to check.
| Check | Against | Result | Verdict |
|---|---|---|---|
| Baseline macro block | Manual v1.1 Table 4, 2025/2030/2040; population and labour force exact to 0.01m; 2025 growth −0.30pp (gate ±0.31), 2040 unemployment −0.19pp (gate ±0.2) | inside every gate (the gates were set after the run); two by only 0.01pp | partial |
| Baseline emissions | Manual v1.1 Table 4: 393 vs 407 (2025), 343 vs 382 (2030), 249 vs 324 (2040) MtCO2e | −23.3% by 2040 | fail |
| Baseline vs outturns | ONS 2025 growth; unemployment 4.39% vs 4.88%; 2024 emissions 401.5 vs 371 (DESNZ); levels never served | 4.66% vs 1.31% | fail |
| Scenario design and consistency | Published switch sets for the 8 checked scenarios (10 tests); net lending sums to zero; 2 fossil-fuel-ban variants refused | 28 of 30 files close | partial |
| Scenario magnitudes | No published v1.1 results; FMM 2023 anchors are our own bands; the recorded “1% of GDP” is FMM’s investment size, and no published GDP response is confirmed | GDP peak +0.92%, pinned | weak |
| Baseline/policy parameter parity | Upstream run’s own parameter files: αPP 0.2748 vs 0.2762; the 2.25 multiplier is as low as 2.08 if the effect persists after 2026 | real GDP −0.16% through 2026 | fail |
| Green-investment multiplier | Published figures are short-horizon (OBR ≈1.0; Batini et al. 2021, 1.1–1.5), so not like-for-like; ours is below them at their horizons | 2.25 nominal, from the 2027 policy start to 2040 | weak |
| Clean-room reimplementation | Upstream run as oracle; accounting matrices pass (milestone 1); multiplier 1.52 vs the upstream’s full-run 2.30 | GDP up to 44% below upstream by 2040 | fail |
Evidence by dimension
| Dimension | Level | What would change it |
|---|---|---|
| Implementation fidelity | moderate | The clean-room model reproducing the upstream baseline (milestone 2, not passed) |
| Predictive validation | not applicable | None: not a forecaster |
| Identification | not applicable | None: deterministic scenario deltas |
| Policy counterfactuals | weak | Reopens if scenario tables are published |
| Uncertainty | weak | Sweep of the most influential parameters |
| Vintage reproducibility | weak | The clean-room build, with no R dependency |
Gates, calibration, the multiplier, emissions
The replication gates
Every row is enforced by tests against the cached pinned run (define-uk-model tests/; results in VALIDATION.md).
The gates, one by oneTolerance and status of each target
| What is checked | Tolerance | Status |
|---|---|---|
| Manual Table 4, macro block — growth, unemployment, population 16+, labour force, S1 baseline | population and labour force ±0.01m; growth ±0.31pp; unemployment ±0.2pp | PASS (2026-08-04) — but 2025 growth misses by −0.30pp (gate ±0.31, set after the run) and 2040 unemployment by −0.19pp (gate ±0.2): both clear by only 0.01pp |
| Manual Table 4, total emissions, S1 baseline | gated at the observed ratios so further drift fails | DIVERGENCE — pinned; see below |
| Scenario design — each of the 8 checked scenarios toggles exactly its published policy switches | exact flag set per scenario; HOUSING_SUB_RATE=0.4 pinned |
PASS (2026-08-05, 10 tests) |
| FMM-2023 anchors — green public investment GDP peak; baseline 2030 emissions | wide (pre-1.0 vintage): GDP peak 0.7–1.3%; emissions 330–352 MtCO2e | PINNED, not a replication — peak +0.92% vs our own 0.7–1.3% band (includes the αPP mismatch); the “1% of GDP” recorded from FMM 2023 is the size of the investment; no published GDP response is confirmed. Baseline 2030 emissions 342.8 vs “just under 350”; a band reaching 352 cannot test that |
| Published numeric scenario results for v1.1 | — | CLOSED at the achievable ceiling — none exist; see below |
| Clean-room reimplementation, milestone 1 — §2.2 matrices on the §5 initial values | manual's 4-significant-figure printing precision | PASS (2026-08-04) — 62 tests, 44 of them Table 1/2 row and column identities; 13 defects in the manual pinned as machine-readable MANUAL_GAPS records |
| Calibration vs external observations (ONS/DESNZ/OBR) | computed and regression-tested, not tolerance-gated | COMPUTED — gaps are material; see below |
The pinned run is upstream commit 846081a. Results are recorded in the adapter's
VALIDATION.md,
so drift fails loudly.
The baseline against official numbers
Recomputed from the cached pinned run against pinned external observations.
The comparison tableModel against ONS, DESNZ and OBR figures
| Quantity | Model | External | Verdict |
|---|---|---|---|
| Real GDP growth, 2025 | 4.66% y/y | 1.31% y/y | Far above outturn — never a forecast |
| Real GDP growth, mean 2025–40 | 2.4% y/y | ≈1.65% y/y | High vs official long-run |
| Unemployment, 2025 | 4.39% | 4.88% | Below outturn by ≈0.5pp |
| Population 16+, 2025 | 55.66m | ≈56m | Consistent |
| Labour force, 2025 | 35.97m | ≈34.5m | Slightly high |
| Total emissions, 2024 | 401.5 MtCO2e/yr | 371 MtCO2e/yr | Above the actuals it should start from |
| Emissions, 2030, vs NDC path | 342.8 MtCO2e/yr | ≈260 MtCO2e/yr | Short of the NDC by design — current-policies baseline |
- The recomputed rows are committed as baseline_vs_external.csv.
- The suite fails on any drift.
- The annual-mean convention puts 2025 growth at 4.66%.
- Table 4 prints 4.96% for the same year.
- That is the same −0.30pp anchoring gap as the Table 4 gate above.
external sources: ONS national accounts and MGSX (2026-07-26 vintages), ONS population and labour-force estimates, DESNZ 2024 provisional territorial emissions, OBR March 2026 EFO, and the UK NDC — attributed row by row in the committed CSV.
So the levels are not forecasts and are never served as such.
- The manual says the baseline “should not be seen as a prediction”.
- Near-term levels are not competitive with the OBR emulator or the BoE SVAR.
- Only scenario deltas are defensible, carrying the demand-led caveat.
The fiscal multiplier
2.25 nominal, from the 2027 policy start to 2040. It is back-loaded, so it sits below published short-horizon figures at their horizons.
| Measure | Horizon | Multiplier |
|---|---|---|
| Ours | One year, 2027 | 0.47 |
| Ours | To 2031 | 0.79 |
| Ours | To 2034 | 1.40 |
| Ours, headline | To 2040 | 2.25 |
| OBR, capital spending | Impact | ≈1.0 |
| Batini et al. (2021), renewable investment | A few years | 1.1–1.5 |
Definitions and caveatsThree ratios · why it is not like-for-like · the spending breakdown
Three ratios from one cached run, all nominal
- Headline: from the 2027 policy start, per £ of power-sector spending
- 2.25
- Full run from 2023, the model repository’s headline
- 2.30
- Full run, per £ of all non-wage spending (ΔSPEND_GVT)
- 1.78
How it is computed
- It is our own computation: cumulative nominal ΔGDP over cumulative ΔSPEND_GVTPS, the government power-sector spending the policy switch sets.
- From the 2027Q1 policy start to 2040 that is 209.9 less 4.8 of pre-policy change, over 91.2.
- The full-run 2.30 keeps the 4.8. It falls in 2023–26, before any policy switch acts, and comes from the parameter mismatch.
- 2.25 is the headline here because served scenario paths start at the 2027 policy start (define-uk-model #31).
- 1.78 was the headline until define-uk-model #16 was decided on 15 Nov 2026.
Why it is not like-for-like
- Horizon. Ours is cumulative to 2040; the published figures are impact or few-year multipliers (Batini et al. 2021 is IMF WP/21/87).
- Closure. The model is demand-led, so a cumulative ratio above 1 is what the closure implies. The channel behind the late gain is not traced through the equations.
- Parameter mismatch. It is included: 2.25 if its effect stops in 2027, as low as 2.08 if it persists, and 1.73–1.75 for the 1.78 ratio (limits).
- Prices. Cumulative real ΔGDP is 250.6 against 209.9 nominal. The model repository pins no real-terms multiplier, so none is quoted.
- The caveats travel with every served delta (
GPI_MULTIPLIER_CAVEATS, gated by test).
What the 1.78 ratio divides by
- ΔSPEND_GVT sums to 118.2: power-sector spending +91.2, other consumption +21.1, social benefits +8.0 and government investment −2.3.
- Government investment is green +17.0 offset by conventional −19.3. The parts sum to 118.0; the 0.2 is rounding in the printed outputs.
- Social benefits rise, so the bias against an impulse-based multiplier has no established direction. An earlier statement that 1.78 is upper-leaning is withdrawn.
- The headline divides by the policy instrument alone, which leaves those induced items out (define-uk-model #16).
Upstream’s own summary table
Multiplier_Summary.csvis unusable and never quoted: its multiplier columns are identical across all eight scenarios.- Its totals are real government spending (104.22; 104.1 by our sum) and real GDP (250.76; 250.6), so the ≈2.4 derived from them is a real ratio, not comparable with the nominal ones here.
The emissions divergence
The pinned upstream code runs below the manual’s published table.
S1 baseline total UK emissions, MtCO2e a year
- pinned run
- published Table 4
846081a) against the published Table 4. What is gated is the ratio of the two — 0.965 / 0.898 / 0.767, pinned to ±0.02 in the adapter's validation/reference_outputs.json, which is where the percentages above come from; the bar labels round the published levels.
Source: PolicyEngine validation data
Source details
Source: validation/figures/data/define_emissions_divergence.csv.
What the divergence isNot an extraction error · pinned at the observed ratios
The one open divergence: the pinned upstream code runs below the manual's published Table 4 emissions by −3.5% in 2025 and −10.2% in 2030, widening to −23.3% in 2040.
Not an extraction error: the components
(EMIS_NELEC + EMIS_ELEC) sum
exactly to the total and the baseline is identical across scenario
folders. A re-check of all 15 Table 4 rows finds the power and
energy block diverging while the inputs and macro block match:
consistent with, but not proof of, a different model vintage from
commit 846081a. Raised with the authors as
DEFINE_UK_1.1#1.
No numeric v1.1 scenario results are published. That is the ceiling
Why the ceilingNo published scenario figures to replicate
For DEFINE-UK v1.1 the manual stops at the baseline plus scenario design parameters, and the accompanying papers are paywalled with figure-only results. A published-figure replication of the scenarios is not possible today, and every delta this adapter serves carries that status. What is gated is design and self-consistency (the switch sets, and the delta paths against the cached pinned run), plus the two coarse FMM-2023 anchors above.
| Limit | Detail |
|---|---|
| Deltas only, never levels | The baseline sits far from official outturns, so levels are never served (baseline against official numbers). |
| Emissions divergence | The pinned upstream run sits below the manual’s published emissions table, widening to 2040; pinned, raised with the authors (the divergence). |
| Multiplier not comparable | Our cumulative green-investment multiplier is back-loaded: below published short-horizon figures at their horizons and above them only cumulated to 2040, so it is not like-for-like (the multiplier). |
| Baseline and policy runs use different αPP |
|
| Unlicensed upstream, never hosted | The authors’ R code is fetched at a pinned commit and run locally; hosted tools return run instructions, never results. |
| The clean-room model solves the whole manual but does not reproduce it |
|
| Two cached scenarios are not stock-flow consistent |
|
| Emissions cuts come mostly from outside the power sector | In green public investment, 23 of the 49 MtCO2e cut in 2030 and 46 of 54 in 2035 are non-electric emissions, while the electricity price falls sharply. That substitution response is untested, and no published v1.1 magnitude exists to check it. |
| Subsidy scenarios lower GDP, unexplained |
|
| No uncertainty | Deterministic deltas from one calibration, with no interval and no sensitivity sweep. |
What would reopen the gatePublished scenario tables, or accessible papers
The gate reopens if the authors publish scenario tables or the papers become accessible. Until then: experimental, partial replication, deltas only, and score a reform does not accept this model.