DEFINE-UK ecological stock-flow

DEFINE-UK

The replication gates, the baseline against official numbers, the multiplier, and the ceiling.

Partial replication; deltas only

The macro block matches Table 4 within every gate (the 2025 growth gate was set at that miss); 2025 growth (−0.30pp, gate ±0.31) and 2040 unemployment (−0.19pp, gate ±0.2) clear by only 0.01pp. No numeric v1.1 scenario results exist to check.

What was checked, against what, and the result
CheckAgainstResultVerdict
Baseline macro blockManual v1.1 Table 4, 2025/2030/2040; population and labour force exact to 0.01m; 2025 growth −0.30pp (gate ±0.31), 2040 unemployment −0.19pp (gate ±0.2)inside every gate (the gates were set after the run); two by only 0.01pppartial
Baseline emissionsManual v1.1 Table 4: 393 vs 407 (2025), 343 vs 382 (2030), 249 vs 324 (2040) MtCO2e−23.3% by 2040fail
Baseline vs outturnsONS 2025 growth; unemployment 4.39% vs 4.88%; 2024 emissions 401.5 vs 371 (DESNZ); levels never served4.66% vs 1.31%fail
Scenario design and consistencyPublished switch sets for the 8 checked scenarios (10 tests); net lending sums to zero; 2 fossil-fuel-ban variants refused28 of 30 files closepartial
Scenario magnitudesNo published v1.1 results; FMM 2023 anchors are our own bands; the recorded “1% of GDP” is FMM’s investment size, and no published GDP response is confirmedGDP peak +0.92%, pinnedweak
Baseline/policy parameter parityUpstream run’s own parameter files: αPP 0.2748 vs 0.2762; the 2.25 multiplier is as low as 2.08 if the effect persists after 2026real GDP −0.16% through 2026fail
Green-investment multiplierPublished figures are short-horizon (OBR ≈1.0; Batini et al. 2021, 1.1–1.5), so not like-for-like; ours is below them at their horizons2.25 nominal, from the 2027 policy start to 2040weak
Clean-room reimplementationUpstream run as oracle; accounting matrices pass (milestone 1); multiplier 1.52 vs the upstream’s full-run 2.30GDP up to 44% below upstream by 2040fail

Evidence by dimension

Evidence by dimension: the registry’s categorical judgement, and what would change it
DimensionLevelWhat would change it
Implementation fidelitymoderateThe clean-room model reproducing the upstream baseline (milestone 2, not passed)
Predictive validationnot applicableNone: not a forecaster
Identificationnot applicableNone: deterministic scenario deltas
Policy counterfactualsweakReopens if scenario tables are published
UncertaintyweakSweep of the most influential parameters
Vintage reproducibilityweakThe clean-room build, with no R dependency

Gates, calibration, the multiplier, emissions

1

The replication gates

Every row is enforced by tests against the cached pinned run (define-uk-model tests/; results in VALIDATION.md).

The gates, one by oneTolerance and status of each target
DEFINE-UK replication gates against the manual and the papers
What is checkedToleranceStatus
Manual Table 4, macro block — growth, unemployment, population 16+, labour force, S1 baseline population and labour force ±0.01m; growth ±0.31pp; unemployment ±0.2pp PASS (2026-08-04) — but 2025 growth misses by −0.30pp (gate ±0.31, set after the run) and 2040 unemployment by −0.19pp (gate ±0.2): both clear by only 0.01pp
Manual Table 4, total emissions, S1 baseline gated at the observed ratios so further drift fails DIVERGENCE — pinned; see below
Scenario design — each of the 8 checked scenarios toggles exactly its published policy switches exact flag set per scenario; HOUSING_SUB_RATE=0.4 pinned PASS (2026-08-05, 10 tests)
FMM-2023 anchors — green public investment GDP peak; baseline 2030 emissions wide (pre-1.0 vintage): GDP peak 0.7–1.3%; emissions 330–352 MtCO2e PINNED, not a replication — peak +0.92% vs our own 0.7–1.3% band (includes the αPP mismatch); the “1% of GDP” recorded from FMM 2023 is the size of the investment; no published GDP response is confirmed. Baseline 2030 emissions 342.8 vs “just under 350”; a band reaching 352 cannot test that
Published numeric scenario results for v1.1 — CLOSED at the achievable ceiling — none exist; see below
Clean-room reimplementation, milestone 1 — §2.2 matrices on the §5 initial values manual's 4-significant-figure printing precision PASS (2026-08-04) — 62 tests, 44 of them Table 1/2 row and column identities; 13 defects in the manual pinned as machine-readable MANUAL_GAPS records
Calibration vs external observations (ONS/DESNZ/OBR) computed and regression-tested, not tolerance-gated COMPUTED — gaps are material; see below
This adapter runs the authors' unmodified code at a pinned commit. DEFINE-UK belongs to George and Dafermos (SOAS), building on the DEFINE model of Dafermos, Nikolaidi and Galanis; nothing here is produced or endorsed by them, and no result is theirs.

The pinned run is upstream commit 846081a. Results are recorded in the adapter's VALIDATION.md, so drift fails loudly.

2

The baseline against official numbers

Recomputed from the cached pinned run against pinned external observations.

The comparison tableModel against ONS, DESNZ and OBR figures
DEFINE-UK baseline (cached pinned run) against external observations, from validation/baseline_vs_external.csv
QuantityModelExternalVerdict
Real GDP growth, 2025 4.66% y/y 1.31% y/y Far above outturn — never a forecast
Real GDP growth, mean 2025–40 2.4% y/y ≈1.65% y/y High vs official long-run
Unemployment, 2025 4.39% 4.88% Below outturn by ≈0.5pp
Population 16+, 2025 55.66m ≈56m Consistent
Labour force, 2025 35.97m ≈34.5m Slightly high
Total emissions, 2024 401.5 MtCO2e/yr 371 MtCO2e/yr Above the actuals it should start from
Emissions, 2030, vs NDC path 342.8 MtCO2e/yr ≈260 MtCO2e/yr Short of the NDC by design — current-policies baseline
  • The recomputed rows are committed as baseline_vs_external.csv.
  • The suite fails on any drift.
  • The annual-mean convention puts 2025 growth at 4.66%.
  • Table 4 prints 4.96% for the same year.
  • That is the same −0.30pp anchoring gap as the Table 4 gate above.

external sources: ONS national accounts and MGSX (2026-07-26 vintages), ONS population and labour-force estimates, DESNZ 2024 provisional territorial emissions, OBR March 2026 EFO, and the UK NDC — attributed row by row in the committed CSV.

So the levels are not forecasts and are never served as such.

  • The manual says the baseline “should not be seen as a prediction”.
  • Near-term levels are not competitive with the OBR emulator or the BoE SVAR.
  • Only scenario deltas are defensible, carrying the demand-led caveat.
3

The fiscal multiplier

2.25 nominal, from the 2027 policy start to 2040. It is back-loaded, so it sits below published short-horizon figures at their horizons.

Cumulative green-public-investment multiplier by horizon: nominal ΔGDP over the power-sector spending the policy sets (ΔSPEND_GVTPS), counted from the 2027Q1 policy start. The two published rows are short-horizon multipliers on other definitions: not like-for-like
MeasureHorizonMultiplier
OursOne year, 20270.47
OursTo 20310.79
OursTo 20341.40
Ours, headlineTo 20402.25
OBR, capital spendingImpact≈1.0
Batini et al. (2021), renewable investmentA few years1.1–1.5
Definitions and caveatsThree ratios · why it is not like-for-like · the spending breakdown

Three ratios from one cached run, all nominal

Headline: from the 2027 policy start, per £ of power-sector spending
2.25
Full run from 2023, the model repository’s headline
2.30
Full run, per £ of all non-wage spending (ΔSPEND_GVT)
1.78

How it is computed

  • It is our own computation: cumulative nominal ΔGDP over cumulative ΔSPEND_GVTPS, the government power-sector spending the policy switch sets.
  • From the 2027Q1 policy start to 2040 that is 209.9 less 4.8 of pre-policy change, over 91.2.
  • The full-run 2.30 keeps the 4.8. It falls in 2023–26, before any policy switch acts, and comes from the parameter mismatch.
  • 2.25 is the headline here because served scenario paths start at the 2027 policy start (define-uk-model #31).
  • 1.78 was the headline until define-uk-model #16 was decided on 15 Nov 2026.

Why it is not like-for-like

  • Horizon. Ours is cumulative to 2040; the published figures are impact or few-year multipliers (Batini et al. 2021 is IMF WP/21/87).
  • Closure. The model is demand-led, so a cumulative ratio above 1 is what the closure implies. The channel behind the late gain is not traced through the equations.
  • Parameter mismatch. It is included: 2.25 if its effect stops in 2027, as low as 2.08 if it persists, and 1.73–1.75 for the 1.78 ratio (limits).
  • Prices. Cumulative real ΔGDP is 250.6 against 209.9 nominal. The model repository pins no real-terms multiplier, so none is quoted.
  • The caveats travel with every served delta (GPI_MULTIPLIER_CAVEATS, gated by test).

What the 1.78 ratio divides by

  • ΔSPEND_GVT sums to 118.2: power-sector spending +91.2, other consumption +21.1, social benefits +8.0 and government investment −2.3.
  • Government investment is green +17.0 offset by conventional −19.3. The parts sum to 118.0; the 0.2 is rounding in the printed outputs.
  • Social benefits rise, so the bias against an impulse-based multiplier has no established direction. An earlier statement that 1.78 is upper-leaning is withdrawn.
  • The headline divides by the policy instrument alone, which leaves those induced items out (define-uk-model #16).

Upstream’s own summary table

  • Multiplier_Summary.csv is unusable and never quoted: its multiplier columns are identical across all eight scenarios.
  • Its totals are real government spending (104.22; 104.1 by our sum) and real GDP (250.76; 250.6), so the ≈2.4 derived from them is a real ratio, not comparable with the nominal ones here.
4

The emissions divergence

The pinned upstream code runs below the manual’s published table.

S1 baseline total UK emissions, MtCO2e a year

DEFINE-UK: S1 baseline total emissions — pinned run vs manual Table 4 (MtCO2e/yr) Grouped bar chart of S1 baseline total UK emissions in MtCO2e per year, the cached pinned run of upstream commit 846081a against the manual's published Table 4. 2025: pinned run 393, published table 407, a gated ratio of 0.965, a gap of -3.5 per cent; 2030: pinned run 343, published table 382, a gated ratio of 0.898, a gap of -10.2 per cent; 2040: pinned run 249, published table 324, a gated ratio of 0.767, a gap of -23.3 per cent. The pinned code runs below the published table and the gap widens with horizon; what is gated is the ratio of the two, pinned to plus or minus 0.02, so any further drift fails loudly. Bar heights are the levels in validation/figures/data/define_emissions_divergence.csv, rounded to whole MtCO2e; the percentages are the gated ratios themselves, which is why they differ slightly from the ratio of the rounded bars.
  • pinned run
  • published Table 4
S1 baseline total emissions: the cached pinned run (upstream 846081a) against the published Table 4. What is gated is the ratio of the two — 0.965 / 0.898 / 0.767, pinned to ±0.02 in the adapter's validation/reference_outputs.json, which is where the percentages above come from; the bar labels round the published levels.

Source: PolicyEngine validation data

Source details

Source: validation/figures/data/define_emissions_divergence.csv.

What the divergence isNot an extraction error · pinned at the observed ratios

The one open divergence: the pinned upstream code runs below the manual's published Table 4 emissions by −3.5% in 2025 and −10.2% in 2030, widening to −23.3% in 2040.

Not an extraction error: the components (EMIS_NELEC + EMIS_ELEC) sum exactly to the total and the baseline is identical across scenario folders. A re-check of all 15 Table 4 rows finds the power and energy block diverging while the inputs and macro block match: consistent with, but not proof of, a different model vintage from commit 846081a. Raised with the authors as DEFINE_UK_1.1#1.

No numeric v1.1 scenario results are published. That is the ceiling

Why the ceilingNo published scenario figures to replicate

For DEFINE-UK v1.1 the manual stops at the baseline plus scenario design parameters, and the accompanying papers are paywalled with figure-only results. A published-figure replication of the scenarios is not possible today, and every delta this adapter serves carries that status. What is gated is design and self-consistency (the switch sets, and the delta paths against the cached pinned run), plus the two coarse FMM-2023 anchors above.

Every known limit of the DEFINE-UK adapter, with the evidence that holds it
LimitDetail
Deltas only, never levelsThe baseline sits far from official outturns, so levels are never served (baseline against official numbers).
Emissions divergenceThe pinned upstream run sits below the manual’s published emissions table, widening to 2040; pinned, raised with the authors (the divergence).
Multiplier not comparableOur cumulative green-investment multiplier is back-loaded: below published short-horizon figures at their horizons and above them only cumulated to 2040, so it is not like-for-like (the multiplier).
Baseline and policy runs use different αPP
  • The upstream Baseline runs production’s own-input coefficient alpha_PPLR at 0.2748; every policy run uses 0.2762 (printed in each run’s Parameters-sorted_*.csv).
  • No policy switch acts before 2027Q1, so the 2023–26 deltas are this parameter change alone, not policy effects: real GDP settles about 0.16% below baseline through 2026.
  • After 2026 the two cannot be separated. The published numbers stand, and each range below runs from the parameter effect stopping in 2027 to it persisting at its 2026 level or share of baseline.
  • Headline multiplier: 2.25 → as low as about 2.08 (the 1.78 ratio → 1.73–1.75).
  • Green-investment GDP peak: +0.92% → +0.92–1.08%.
  • Housing-regulation real GDP in 2035: −2.3bn a quarter (−0.27% of baseline) → −0.96 to −2.3.
  • Fossil-fuel-ban real GDP in 2030: −3.85bn a quarter (−0.50%) → −2.65 to −3.85.
  • Emissions deltas change by under 0.25%.
  • Sized in define-uk-model #20; the fix is not yet decided.
Unlicensed upstream, never hostedThe authors’ R code is fetched at a pinned commit and run locally; hosted tools return run instructions, never results.
The clean-room model solves the whole manual but does not reproduce it
  • The clean-room Python registers every printed equation of the manual (§3.1–§3.4.7) and solves the whole model quarter by quarter (define-uk-model #19, merged).
  • It does not reproduce the upstream run: baseline GDP falls up to 44% below it by 2040, mostly through equity-revaluation equations that are not yet diagnosed, so milestone 2 is not passed (#17).
  • Its own green public investment multiplier is about 1.52, against 2.30 upstream on the same impulse over the full run. That figure is a diagnostic, not a served result: every served number comes from the cached upstream run.
  • The Taylor rule’s inflation input is inferred from the upstream outputs, because the manual prints no equation for it, and two power-sector profit rates are still read from the upstream run (#24).
  • Where the upstream outputs follow a form that differs from the manual as printed, the model records that form, found from the outputs alone (empirical.py; Code, step 4).
Two cached scenarios are not stock-flow consistent
  • Affected: the fossil-fuel ban with no forward guidance (2035Q3 and 2036Q1) and with unexpected enforcement (2033Q1).
  • In those quarters the power sector’s revenue is valued at a different electricity price from the one buyers pay, so the sectors’ net lending does not sum to zero.
  • The other 28 cached files close. The fault is in the upstream outputs.
  • The adapter refuses these two scenarios and attaches a stock-flow check to every result (define-uk-model #15, merged 6 Nov).
Emissions cuts come mostly from outside the power sectorIn green public investment, 23 of the 49 MtCO2e cut in 2030 and 46 of 54 in 2035 are non-electric emissions, while the electricity price falls sharply. That substitution response is untested, and no published v1.1 magnitude exists to check it.
Subsidy scenarios lower GDP, unexplained
  • Real GDP in 2028: green power subsidy −0.67%; power-sector subsidy −0.34%; both regulations plus subsidies −0.72%.
  • Cumulative real GDP over 2027–40, from the cached run: green power subsidy −47.3 (−0.10% of the cumulative baseline); housing regulation plus subsidy −149.9 (−0.32%).
  • Net of the parameter mismatch the falls remain.
  • How the subsidies are financed is not stated in the scenario descriptions, and the cause is not yet diagnosed.
No uncertaintyDeterministic deltas from one calibration, with no interval and no sensitivity sweep.
What would reopen the gatePublished scenario tables, or accessible papers

The gate reopens if the authors publish scenario tables or the papers become accessible. Until then: experimental, partial replication, deltas only, and score a reform does not accept this model.