Run in Colab Enterprise

Run in Colab Enterprise: click the badge to import this notebook, pick a runtime, and Run all. The Terraform-deployed templates already carry the SF_* run identity in their env, so there's no environment cell to fill in. Runs on the sf-main runtime template (Python 3.11). See docs/notebook_runtimes.md for the per-notebook template mapping and the headless acceptance harness.

06 · Coherent Hierarchical Forecasting (S Matrix & 7 FPP3 Reconciliation Methods)¶

Enterprise organizations require forecasts that add up coherently across product, geography, and organizational hierarchies (__total__ $\rightarrow$ region $\rightarrow$ region/category $\rightarrow$ ts_id). scale-forecasting implements full bottom-up aggregation and post-forecast projection $\tilde{y} = S G \hat{y}$ across all 7 Hyndman FPP3 reconciliation estimators (reconciliation.py):

At a glance¶

Dimension	Details
Learning outcomes	Build bottom-up summing matrices (S), execute all 7 FPP3 hierarchical reconciliation methods (bottom_up, top_down, middle_out, ols, wls_struct, wls_var, mint_shrink), and verify cross-level coherence.
Prerequisites	Deployed scale-forecasting Google Cloud project (SF_* env vars) + scale-forecasting[gcp].
Estimated time	~5–8 minutes.
Estimated run cost	Estimated ~\$0.10–\$0.40 USD (Vertex AI / Spark / BigQuery compute for the hierarchical panel + BigQuery Storage API).
Ongoing cost	Zero compute when idle (ephemeral job compute tears down automatically on completion).

Method	Formula / Weighting	When to Use
bottom_up	Sums bottom-level forecasts directly ($G = [0 \mid I]$)	High signal-to-noise bottom series; zero aggregate model bias
top_down	Disaggregates __total__ forecast by historical average proportions $p_i$	Noisy bottom series with stable macro proportions
middle_out	Anchors at an intermediate middle_level, sums up & disaggregates down	Strong regional/category signal
ols	Identity error covariance ($W = I$) ordinary least squares projection	Fast coherence without backtest residuals
wls_struct	Structural variance scaling ($W = \text{diag}(S \mathbf{1})$)	Balanced accuracy across hierarchy levels without OOF residuals
wls_var	Diagonal out-of-fold residual variance ($W = \text{diag}(\hat{\sigma}_i^2)$)	Heteroskedastic series with backtesting enabled
mint_shrink	Schafer-Strimmer shrinkage covariance of OOF residuals ($W = \hat{\Sigma}_{\text{shrink}}$)	Gold standard — exploits cross-series error correlations

Act 1 · Cloud Bootstrap & Deployment Resolution¶

On a managed cloud notebook (Colab Enterprise, Vertex AI Workbench) the bootstrap cell below clones or updates the repository and installs the locked dependency set (uv.lock) into an isolated .colab-deps prefix so import scale_forecasting resolves against the exact package versions every cloud worker uses. In a local clone with .venv, this cell is a fast no-op.

In [1]:

# Cloud bootstrap: clone + install the LOCKED dependency set (uv.lock) so
# `import scale_forecasting` resolves against the exact versions every other surface runs.
import importlib.util
import os
import shutil
import subprocess
import sys

REPO_URL = os.environ.get("SF_REPO_URL", "https://github.com/statmike/scale-forecasting.git")
REPO_DIR = os.environ.get("SF_REPO_DIR", "scale-forecasting")
EXTRAS = ['gcp']

if importlib.util.find_spec("scale_forecasting") is None:
    if not os.path.isdir(REPO_DIR):
        subprocess.run(["git", "clone", "--depth", "1", REPO_URL, REPO_DIR], check=True)
    uv = shutil.which("uv")
    if uv is None:
        subprocess.run([sys.executable, "-m", "pip", "install", "-q", "uv"], check=True)
        uv = shutil.which("uv") or "uv"
    reqs = os.path.abspath(os.path.join(REPO_DIR, "colab-requirements.txt"))
    extra_flags = [f for e in EXTRAS for f in ("--extra", e)]
    subprocess.run(
        [uv, "export", "--frozen", "--no-emit-project", "--no-hashes", "--no-dev", *extra_flags, "-o", reqs],
        cwd=REPO_DIR,
        check=True,
    )
    target = os.path.abspath(os.path.join(REPO_DIR, ".colab-deps"))
    subprocess.run(
        [uv, "pip", "install", "--python", sys.executable, "--target", target, "-r", reqs],
        check=True,
    )
    sys.path.insert(0, target)
    sys.path.insert(0, os.path.join(REPO_DIR, "src"))
    print("installed scale_forecasting from", REPO_DIR)
else:
    print("scale_forecasting already importable — skipping bootstrap")

# Cloud bootstrap: clone + install the LOCKED dependency set (uv.lock) so # `import scale_forecasting` resolves against the exact versions every other surface runs. import importlib.util import os import shutil import subprocess import sys REPO_URL = os.environ.get("SF_REPO_URL", "https://github.com/statmike/scale-forecasting.git") REPO_DIR = os.environ.get("SF_REPO_DIR", "scale-forecasting") EXTRAS = ['gcp'] if importlib.util.find_spec("scale_forecasting") is None: if not os.path.isdir(REPO_DIR): subprocess.run(["git", "clone", "--depth", "1", REPO_URL, REPO_DIR], check=True) uv = shutil.which("uv") if uv is None: subprocess.run([sys.executable, "-m", "pip", "install", "-q", "uv"], check=True) uv = shutil.which("uv") or "uv" reqs = os.path.abspath(os.path.join(REPO_DIR, "colab-requirements.txt")) extra_flags = [f for e in EXTRAS for f in ("--extra", e)] subprocess.run( [uv, "export", "--frozen", "--no-emit-project", "--no-hashes", "--no-dev", *extra_flags, "-o", reqs], cwd=REPO_DIR, check=True, ) target = os.path.abspath(os.path.join(REPO_DIR, ".colab-deps")) subprocess.run( [uv, "pip", "install", "--python", sys.executable, "--target", target, "-r", reqs], check=True, ) sys.path.insert(0, target) sys.path.insert(0, os.path.join(REPO_DIR, "src")) print("installed scale_forecasting from", REPO_DIR) else: print("scale_forecasting already importable — skipping bootstrap")

scale_forecasting already importable — skipping bootstrap

Resolve the Live GCP Deployment (Settings.resolve())¶

Settings.resolve() reads the SF_* environment variables (SF_PROJECT_ID, SF_REGION, SF_DATASET_ID, SF_WAREHOUSE_URI, SF_CONNECTION, SF_COMPUTE_SA) pre-populated by Terraform in the sf-main runtime template (or your local shell).

In [2]:

from google.cloud import bigquery
from scale_forecasting.settings import Settings

settings = Settings.resolve()
client = bigquery.Client(project=settings.project_id)
DATASET = settings.dataset_ref
print(f"Deployment: {DATASET} | Region: {settings.region} | Bucket: {settings.warehouse_uri}")

from google.cloud import bigquery from scale_forecasting.settings import Settings settings = Settings.resolve() client = bigquery.Client(project=settings.project_id) DATASET = settings.dataset_ref print(f"Deployment: {DATASET} | Region: {settings.region} | Bucket: {settings.warehouse_uri}")

Deployment: statmike-scale-forecasting.scale_forecasting | Region: us-central1 | Bucket: gs://statmike-scale-forecasting-warehouse/warehouse

Act 2 · Configure Hierarchical Forecasting & Explain the Plan¶

We configure hierarchy={'enabled': True, 'levels': [['region'], ['region', 'category']], 'reconciliation_methods': ['bottom_up', 'wls_struct', 'mint_shrink']} so the engine automatically rolls up the panel before model fitting and reconciles both out-of-fold backtest predictions and future forecasts.

In [3]:

import time
from scale_forecasting import Forecaster, RunConfig

# === Parameters — edit me ===============================================
RUN_NAME = f"nb06 hierarchy mint {int(time.time())}"
SOURCE_TABLE = "source_series_covariates_native"  # includes region & category hierarchy columns
MODELS = ["theta", "holtwinters", "lightgbm"]
RECON_METHODS = ["bottom_up", "wls_struct", "mint_shrink"]
HORIZON = 14
SERIES_LIMIT = 20
# ========================================================================

cfg = RunConfig(
    run_name=RUN_NAME,
    python_runtime="vertex",
    data={"source_table": SOURCE_TABLE, "horizon": HORIZON, "series_limit": SERIES_LIMIT},
    models=MODELS,
    hierarchy={
        "enabled": True,
        "levels": [["region"], ["region", "category"]],
        "reconciliation_methods": RECON_METHODS,
    },
    backtest={"enabled": True, "n_folds": 2, "horizon": HORIZON, "step": HORIZON},
    ensemble={"enabled": True, "strategies": ["inverse_error", "nnls"]},
)

forecaster = Forecaster(cfg, settings=settings)
print("run_id:", forecaster.run_id)
display(forecaster.explain(validate_data=False))

import time from scale_forecasting import Forecaster, RunConfig # === Parameters — edit me =============================================== RUN_NAME = f"nb06 hierarchy mint {int(time.time())}" SOURCE_TABLE = "source_series_covariates_native" # includes region & category hierarchy columns MODELS = ["theta", "holtwinters", "lightgbm"] RECON_METHODS = ["bottom_up", "wls_struct", "mint_shrink"] HORIZON = 14 SERIES_LIMIT = 20 # ======================================================================== cfg = RunConfig( run_name=RUN_NAME, python_runtime="vertex", data={"source_table": SOURCE_TABLE, "horizon": HORIZON, "series_limit": SERIES_LIMIT}, models=MODELS, hierarchy={ "enabled": True, "levels": [["region"], ["region", "category"]], "reconciliation_methods": RECON_METHODS, }, backtest={"enabled": True, "n_folds": 2, "horizon": HORIZON, "step": HORIZON}, ensemble={"enabled": True, "strategies": ["inverse_error", "nnls"]}, ) forecaster = Forecaster(cfg, settings=settings) print("run_id:", forecaster.run_id) display(forecaster.explain(validate_data=False))

run_id: nb06-hierarchy-mint-1791145253-bbc069a2c659

	run_id	family	job_key	runtime	machine_type	workers	hardware	gpu_type	models	training_modes	n_series	n_folds	expected_cells	covariates	depends_on
0	nb06-hierarchy-mint-1791145253-bbc069a2c659	statistical	sf-nb06-hierarchy-mint-1791145253-bbc069a2c659...	vertex	n2-standard-8	1	cpu	None	theta, holtwinters	theta:local, holtwinters:local	20	2	40	none
1	nb06-hierarchy-mint-1791145253-bbc069a2c659	ml	sf-nb06-hierarchy-mint-1791145253-bbc069a2c659...	vertex	n2-standard-8	1	cpu	None	lightgbm	lightgbm:local	20	2	20	none
2	nb06-hierarchy-mint-1791145253-bbc069a2c659	ensemble	sf-nb06-hierarchy-mint-1791145253-bbc069a2c659...	bigquery	sql	1	sql	None	ensemble_inverse_error, ensemble_nnls	barrier	20	2	40	none	sf-nb06-hierarchy-mint-1791145253-bbc069a2c659...

Act 3 · Launch & Monitor Live (forecaster.run_live())¶

During execution, the worker engine augments the 20 bottom series with their aggregate parent nodes (__total__, region=*, region=*/category=*), fits each model, reconciles every model's forecasts across bottom_up, wls_struct, and mint_shrink, and writes the coherent forecasts to BigQuery.

In [4]:

result = forecaster.run_live(poll_s=15.0, probe=True)
print(f"Completed {result.run_id}: status={result.status}, runtime={result.runtime_seconds:.1f}s")

result = forecaster.run_live(poll_s=15.0, probe=True) print(f"Completed {result.run_id}: status={result.status}, runtime={result.runtime_seconds:.1f}s")

Completed nb06-hierarchy-mint-1791145253-bbc069a2c659: status=COMPLETED, runtime=427.4s

Act 4 · Inspect Hierarchy Level Rollups (hierarchy_df & plot_hierarchy)¶

forecaster.hierarchy_df() and forecaster.plot_hierarchy() aggregate predicted totals across each level of the tree (level_0_total, level_1, level_2, bottom_or_leaf) to verify that every level sums to the exact same grand total.

In [5]:

import matplotlib.pyplot as plt
from scale_forecasting import plot_leaderboard

display(forecaster.leaderboard_df())
plot_leaderboard(forecaster.review_run())
plt.show()

h_df = forecaster.hierarchy_df()
display(h_df)

fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(13, 4.5))
forecaster.plot_hierarchy(model_type="theta", ax=ax1, title="Before Reconciliation")
forecaster.plot_hierarchy(model_type="theta_mint_shrink", ax=ax2, title="After MinT Shrinkage")
fig.tight_layout()
plt.show()

import matplotlib.pyplot as plt from scale_forecasting import plot_leaderboard display(forecaster.leaderboard_df()) plot_leaderboard(forecaster.review_run()) plt.show() h_df = forecaster.hierarchy_df() display(h_df) fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(13, 4.5)) forecaster.plot_hierarchy(model_type="theta", ax=ax1, title="Before Reconciliation") forecaster.plot_hierarchy(model_type="theta_mint_shrink", ax=ax2, title="After MinT Shrinkage") fig.tight_layout() plt.show()

	rank	model_type	family	is_ensemble	ensemble_id	compute_engine	n_series	decision_metric	score	pooled_wape	...	mean_mae	mean_rmse	mean_coverage	mean_interval_score	mean_fit_seconds	median_fit_seconds	no_artifact_rate	n_predictions	lift_vs_best_base	lift_pct
0	1	holtwinters_mint_shrink	unknown	False	None	vertex	37	wape	0.290296	0.101042	...	19.777619	29.661252	0.686293	112.278367	1.656038	1.460099	1.0	518	NaN	NaN
1	2	holtwinters_bottom_up	unknown	False	None	vertex	37	wape	0.300555	0.103318	...	21.299447	31.538323	0.622587	118.562089	1.656038	1.460099	1.0	518	NaN	NaN
2	3	holtwinters	statistical	False	None	vertex	37	wape	0.302522	0.100471	...	21.196424	28.973053	0.666023	108.865936	1.656038	1.460099	1.0	518	NaN	NaN
3	4	lightgbm_mint_shrink	unknown	False	None	vertex	37	wape	0.305898	0.092156	...	17.987551	26.310532	0.277992	137.620348	1.015369	0.821361	1.0	518	NaN	NaN
4	5	ensemble_inverse_error	ensemble	True	34432fdde9fa	ensemble	37	wape	0.306834	0.095035	...	19.474818	27.197892	NaN	NaN	NaN	NaN	1.0	518	-0.016538	-0.056969
5	6	holtwinters_wls_struct	unknown	False	None	vertex	37	wape	0.312112	0.097846	...	20.821935	31.072354	0.596525	128.089127	1.656038	1.460099	1.0	518	NaN	NaN
6	7	theta_mint_shrink	unknown	False	None	vertex	37	wape	0.322315	0.109499	...	21.620279	31.275288	0.989382	234.936310	1.899232	1.767336	1.0	518	NaN	NaN
7	8	ensemble_nnls	ensemble	True	34432fdde9fa	ensemble	37	wape	0.325074	0.097184	...	20.193194	27.936101	NaN	NaN	NaN	NaN	0.0	518	-0.034778	-0.119801
8	9	lightgbm_bottom_up	unknown	False	None	vertex	37	wape	0.331109	0.099090	...	19.987410	28.746893	0.302124	144.084096	1.015369	0.821361	1.0	518	NaN	NaN
9	10	theta_bottom_up	unknown	False	None	vertex	37	wape	0.332475	0.110100	...	22.993494	33.070271	0.991313	248.964410	1.899232	1.767336	1.0	518	NaN	NaN
10	11	lightgbm	ml	False	None	vertex	37	wape	0.342224	0.103643	...	21.298468	29.281147	0.302124	152.355357	1.015369	0.821361	1.0	518	NaN	NaN
11	12	theta_wls_struct	unknown	False	None	vertex	37	wape	0.344228	0.105001	...	22.186364	32.193439	0.977799	203.129876	1.899232	1.767336	1.0	518	NaN	NaN
12	13	theta	statistical	False	None	vertex	37	wape	0.344728	0.106097	...	22.334919	30.012708	0.992278	293.876694	1.899232	1.767336	1.0	518	NaN	NaN
13	14	lightgbm_wls_struct	unknown	False	None	vertex	37	wape	0.430991	0.096712	...	19.718478	28.989593	0.281853	155.450679	1.015369	0.821361	1.0	518	NaN	NaN

14 rows × 25 columns

	model_type	level	n_series	n_points	mean_yhat	sum_yhat	max_coherence_residual
0	ensemble_inverse_error	total	1	14	2041.509304	28581.130250	1.720098e+02
1	ensemble_inverse_error	aggregate	16	224	254.698593	57052.484804	1.720098e+02
2	ensemble_inverse_error	bottom	20	280	98.526903	27587.532735	1.720098e+02
3	ensemble_nnls	total	1	14	2033.259359	28465.631027	1.556478e+02
4	ensemble_nnls	aggregate	16	224	257.158405	57603.482652	1.556478e+02
5	ensemble_nnls	bottom	20	280	100.847734	28237.365411	1.556478e+02
6	holtwinters	total	1	14	2055.259926	28773.638965	1.677141e+02
7	holtwinters	aggregate	16	224	251.861556	56416.988558	1.677141e+02
8	holtwinters	bottom	20	280	95.761727	26813.283446	1.677141e+02
9	holtwinters_bottom_up	total	1	14	1915.234532	26813.283446	2.273737e-13
10	holtwinters_bottom_up	aggregate	16	224	239.404316	53626.566892	2.273737e-13
11	holtwinters_bottom_up	bottom	20	280	95.761727	26813.283446	2.273737e-13
12	holtwinters_mint_shrink	total	1	14	1978.191835	27694.685693	4.547474e-13
13	holtwinters_mint_shrink	aggregate	16	224	247.273979	55389.371386	4.547474e-13
14	holtwinters_mint_shrink	bottom	20	280	98.909592	27694.685693	4.547474e-13
15	holtwinters_wls_struct	total	1	14	2000.069839	28000.977742	4.547474e-13
16	holtwinters_wls_struct	aggregate	16	224	250.008730	56001.955484	4.547474e-13
17	holtwinters_wls_struct	bottom	20	280	100.003492	28000.977742	4.547474e-13
18	lightgbm	total	1	14	2020.503921	28287.054896	2.735230e+02
19	lightgbm	aggregate	16	224	260.400024	58329.605338	2.735230e+02
20	lightgbm	bottom	20	280	103.702852	29036.798615	2.735230e+02
21	lightgbm_bottom_up	total	1	14	2074.057044	29036.798615	4.547474e-13
22	lightgbm_bottom_up	aggregate	16	224	259.257130	58073.597229	4.547474e-13
23	lightgbm_bottom_up	bottom	20	280	103.702852	29036.798615	4.547474e-13
24	lightgbm_mint_shrink	total	1	14	2057.980161	28811.722248	4.547474e-13
25	lightgbm_mint_shrink	aggregate	16	224	257.247520	57623.444495	4.547474e-13
26	lightgbm_mint_shrink	bottom	20	280	102.899008	28811.722248	4.547474e-13
27	lightgbm_wls_struct	total	1	14	2065.240337	28913.364712	9.094947e-13
28	lightgbm_wls_struct	aggregate	16	224	258.155042	57826.729424	9.094947e-13
29	lightgbm_wls_struct	bottom	20	280	103.262017	28913.364712	9.094947e-13
30	theta	total	1	14	2049.633483	28694.868758	1.528919e+02
31	theta	aggregate	16	224	252.896365	56648.785670	1.528919e+02
32	theta	bottom	20	280	97.237932	27226.620901	1.528919e+02
33	theta_bottom_up	total	1	14	1944.758636	27226.620901	4.547474e-13
34	theta_bottom_up	aggregate	16	224	243.094829	54453.241802	4.547474e-13
35	theta_bottom_up	bottom	20	280	97.237932	27226.620901	4.547474e-13
36	theta_mint_shrink	total	1	14	2015.972547	28223.615657	4.547474e-13
37	theta_mint_shrink	aggregate	16	224	251.996568	56447.231314	4.547474e-13
38	theta_mint_shrink	bottom	20	280	100.798627	28223.615657	4.547474e-13
39	theta_wls_struct	total	1	14	2010.183488	28142.568832	9.094947e-13
40	theta_wls_struct	aggregate	16	224	251.272936	56285.137664	9.094947e-13
41	theta_wls_struct	bottom	20	280	100.509174	28142.568832	9.094947e-13

---

Where next¶

• Next in Track 3 (Accuracy & Post-Processing): 07_hpo_backtesting_and_ensembles.ipynb — combine Optuna hyperparameter tuning, 4 backtest schemes, and in-run, post-run, and cross-run ensembling.
• Documentation: Configuration reference (hierarchy) · Backtesting guide · Cost estimates & controls