Run in Colab Enterprise

Run in Colab Enterprise: click the badge to import this notebook, pick a runtime, and Run all. The Terraform-deployed templates already carry the SF_* run identity in their env, so there's no environment cell to fill in. Runs on the sf-main runtime template (Python 3.11) — or fully locally with just a clone; this notebook needs no cloud at all. See docs/notebook_runtimes.md for the per-notebook template mapping and the headless acceptance harness.

00 · Interactive Model Playground: Zero-GCP Sandbox for Models, Metrics, Regimes & SDK Helpers¶

Welcome to the Interactive Model Playground (00_model_playground.ipynb) for scale-forecasting. This notebook is your zero-cloud entry point: explore all 37 forecasting models across 5 model families and 21 evaluation metrics, inspect deterministic synthetic time-series archetypes, compare local vs. global vs. hybrid training regimes, test 3-tier exogenous covariates, Two-Tier Feature Attributions, and all 8 FPP3 hierarchical reconciliation methods, and preview the explanatory SDK helpers — all 100% offline in local memory with zero Google Cloud setup.

At a glance¶

Dimension	Details
Learning outcomes	Discover all 37 models & 21 metrics, generate synthetic archetypes, fit local/global/hybrid regimes, reconcile a 3-level hierarchy across 8 FPP3 methods, and inspect Two-Tier feature attributions.
Prerequisites	Python 3.11 + scale-forecasting (no Google Cloud project, credentials, or billing required).
Estimated time	~3–5 minutes end-to-end on local CPU.
Estimated run cost	\$0.00 (runs 100% locally in memory; zero Google Cloud services billed).
Ongoing cost	None (\$0 when idle; no cloud resources created).

[!NOTE] Identical Execution Contract (worker.run_cell & worker.run_panel): Everything you run in this playground executes the exact same pure Python worker functions that run across Dataproc Spark executors, Vertex AI Ray actors, Vertex CustomJob worker pools, GKE Indexed Jobs, and GCE VMs at 100,000-series scale.

flowchart LR
    subgraph Local["Pure Offline Layer (Zero GCP Credentials Required)"]
        GEN["sample_data()
Synthetic Multi-Archetype Panel"] --> VAL["validate_panel()
Schema & Frequency Guard"]
        VAL --> CELL["worker.run_cell()
Local Per-Series Fit + Backtest"]
        VAL --> PANEL["worker.run_panel()
Global & Hybrid Cross-Series Fit"]
        CELL & PANEL --> CAL["calibration.py
Residual Conformal Intervals"]
        CAL --> MET["metrics.compute_metrics()
21-Metric Evaluation Panel"]
        MET --> REC["reconciliation.py · ensembler.py
8 FPP3 Methods + 9 Ensembles"]
    end
    REC --> SDK["Explanatory SDK Helpers
explain() · attributions_df() · plot_attributions() · plot_forecasts_frame()"]

What You Will Explore in This Notebook¶

Section	Platform Surface	Key Functions & Classes
Act 2 · Catalog Discovery	All 37 Models (5 families) & 21 Metrics (16 point + 5 interval)	model_catalog(), metric_catalog(), available_models()
Act 3 · Synthetic Archetypes	Deterministic daily/weekly/monthly generator with covariates & hierarchy	sample_data(with_exog=True, with_hierarchy=True)
Act 4 · Fit, Backtest & Intervals	Rolling-origin CV, conformal prediction bands, and forecast plotting	run_model(), summarize(), build_predictions_frame(), plot_forecasts_frame()
Act 5 · Training Regimes	local (per-series) vs. global (cross-series) vs. hybrid (local trend + global residual)	worker.run_cell(), worker.run_panel()
Act 6 · Hierarchy & Reconciliation	Bottom-up rollup (S matrix) & all 8 FPP3 reconciliation estimators	build_hierarchy(), reconcile_forecasts(), verify_coherence(), plot_hierarchy_frame()
Act 7 · Multi-Model Bake-Off, Attributions & SDK	Head-to-head comparison + Two-Tier Feature Attributions + inverse_error & nnls ensembles + Forecaster.explain()	bakeoff(), build_attributions_frame(), plot_attributions(), Forecaster.explain(validate_data=False)

Act 1 · Cloud Bootstrap & Deployment Resolution¶

On a managed cloud notebook (Colab Enterprise, Vertex AI Workbench) the bootstrap cell below clones or updates the repository and installs the locked dependency set (uv.lock) into an isolated .colab-deps prefix so import scale_forecasting resolves against the exact package versions every cloud worker uses. In a local clone with .venv, this cell is a fast no-op.

In [1]:

# Cloud bootstrap: clone + install the LOCKED dependency set (uv.lock) so
# `import scale_forecasting` resolves against the exact versions every other surface runs.
import importlib.util
import os
import shutil
import subprocess
import sys

REPO_URL = os.environ.get("SF_REPO_URL", "https://github.com/statmike/scale-forecasting.git")
REPO_DIR = os.environ.get("SF_REPO_DIR", "scale-forecasting")
EXTRAS = []

if importlib.util.find_spec("scale_forecasting") is None:
    if not os.path.isdir(REPO_DIR):
        subprocess.run(["git", "clone", "--depth", "1", REPO_URL, REPO_DIR], check=True)
    uv = shutil.which("uv")
    if uv is None:
        subprocess.run([sys.executable, "-m", "pip", "install", "-q", "uv"], check=True)
        uv = shutil.which("uv") or "uv"
    reqs = os.path.abspath(os.path.join(REPO_DIR, "colab-requirements.txt"))
    extra_flags = [f for e in EXTRAS for f in ("--extra", e)]
    subprocess.run(
        [uv, "export", "--frozen", "--no-emit-project", "--no-hashes", "--no-dev", *extra_flags, "-o", reqs],
        cwd=REPO_DIR,
        check=True,
    )
    target = os.path.abspath(os.path.join(REPO_DIR, ".colab-deps"))
    subprocess.run(
        [uv, "pip", "install", "--python", sys.executable, "--target", target, "-r", reqs],
        check=True,
    )
    sys.path.insert(0, target)
    sys.path.insert(0, os.path.join(REPO_DIR, "src"))
    print("installed scale_forecasting from", REPO_DIR)
else:
    print("scale_forecasting already importable — skipping bootstrap")

# Cloud bootstrap: clone + install the LOCKED dependency set (uv.lock) so # `import scale_forecasting` resolves against the exact versions every other surface runs. import importlib.util import os import shutil import subprocess import sys REPO_URL = os.environ.get("SF_REPO_URL", "https://github.com/statmike/scale-forecasting.git") REPO_DIR = os.environ.get("SF_REPO_DIR", "scale-forecasting") EXTRAS = [] if importlib.util.find_spec("scale_forecasting") is None: if not os.path.isdir(REPO_DIR): subprocess.run(["git", "clone", "--depth", "1", REPO_URL, REPO_DIR], check=True) uv = shutil.which("uv") if uv is None: subprocess.run([sys.executable, "-m", "pip", "install", "-q", "uv"], check=True) uv = shutil.which("uv") or "uv" reqs = os.path.abspath(os.path.join(REPO_DIR, "colab-requirements.txt")) extra_flags = [f for e in EXTRAS for f in ("--extra", e)] subprocess.run( [uv, "export", "--frozen", "--no-emit-project", "--no-hashes", "--no-dev", *extra_flags, "-o", reqs], cwd=REPO_DIR, check=True, ) target = os.path.abspath(os.path.join(REPO_DIR, ".colab-deps")) subprocess.run( [uv, "pip", "install", "--python", sys.executable, "--target", target, "-r", reqs], check=True, ) sys.path.insert(0, target) sys.path.insert(0, os.path.join(REPO_DIR, "src")) print("installed scale_forecasting from", REPO_DIR) else: print("scale_forecasting already importable — skipping bootstrap")

scale_forecasting already importable — skipping bootstrap

Act 2 · Discover the 37-Model & 21-Metric Catalogs¶

model_catalog() inspects the live model factory (src/scale_forecasting/models/) and returns one row per registered model with its family (statistical, ml, deep_learning, automl, native), runtime compatibility (python_local, vertex_automl, bigquery_sql), GPU capability, 3-tier covariate support (future_covariates, past_covariates, static_covariates), explainability (supports_explainability), and global_mode / hybrid_mode flags.

metric_catalog() inspects the metric factory (src/scale_forecasting/metrics/) across all 21 evaluation metrics (16 point accuracy metrics + 5 prediction interval metrics).

In [2]:

import pandas as pd
from scale_forecasting import metric_catalog, model_catalog
from scale_forecasting.playground import available_models

pd.set_option("display.max_columns", 25)
pd.set_option("display.width", 140)

models_df = model_catalog()
print(f"Total registered models: {len(models_df)} | Locally available in this kernel: {len(available_models(installed_only=True))}")
print("Models by family:", models_df.groupby("family")["model"].count().to_dict())
display(models_df)

import pandas as pd from scale_forecasting import metric_catalog, model_catalog from scale_forecasting.playground import available_models pd.set_option("display.max_columns", 25) pd.set_option("display.width", 140) models_df = model_catalog() print(f"Total registered models: {len(models_df)} | Locally available in this kernel: {len(available_models(installed_only=True))}") print("Models by family:", models_df.groupby("family")["model"].count().to_dict()) display(models_df)

Total registered models: 37 | Locally available in this kernel: 31
Models by family: {'automl': 4, 'deep_learning': 8, 'ml': 5, 'native': 2, 'statistical': 18}

	model	family	runtime	package	package_url	available	local	spark	ray	gpu	vertex_automl	bigquery	exog	future_covariates	past_covariates	static_covariates	explainability	global_mode	hybrid_mode
0	arima_plus	native	bigquery	bigquery-ml	https://cloud.google.com/bigquery/docs/bqml-in...	True	False	False	False	False	False	True	True	True	False	False	True	False	False
1	timesfm	native	bigquery	bigquery-ml	https://cloud.google.com/bigquery/docs/bqml-in...	True	False	False	False	False	False	True	True	True	True	False	False	False	False
2	chronos2	deep_learning	python	chronos-forecasting	https://github.com/amazon-science/chronos-fore...	True	True	True	True	True	False	False	True	True	True	True	False	True	False
3	chronos_bolt	deep_learning	python	chronos-forecasting	https://github.com/amazon-science/chronos-fore...	True	True	True	True	True	False	False	False	False	False	False	False	True	False
4	neuralprophet	deep_learning	python	neuralprophet	https://neuralprophet.com/	True	True	True	True	True	False	False	False	False	False	False	False	True	True
5	patchtst	deep_learning	python	neuralforecast	https://nixtlaverse.nixtla.io/neuralforecast/i...	True	True	True	True	True	False	False	False	False	False	False	False	True	False
6	tft	deep_learning	python	neuralforecast	https://nixtlaverse.nixtla.io/neuralforecast/i...	True	True	True	True	True	False	False	True	True	True	True	False	True	False
7	tide	deep_learning	python	neuralforecast	https://nixtlaverse.nixtla.io/neuralforecast/i...	True	True	True	True	True	False	False	True	True	True	True	False	True	False
8	timesfm_torch	deep_learning	python	timesfm	https://github.com/google-research/timesfm	True	True	True	True	True	False	False	True	True	True	True	False	True	False
9	tsmixer	deep_learning	python	neuralforecast	https://nixtlaverse.nixtla.io/neuralforecast/i...	True	True	True	True	True	False	False	True	True	True	True	False	True	False
10	catboost	ml	python	catboost	https://catboost.ai/	True	True	True	True	False	False	False	True	True	True	False	True	False	False
11	lightgbm	ml	python	lightgbm	https://lightgbm.readthedocs.io/	True	True	True	True	False	False	False	True	True	True	False	True	False	False
12	random_forest	ml	python	scikit-learn	https://scikit-learn.org/	True	True	True	True	False	False	False	True	True	True	False	True	False	False
13	regression_lags	ml	python	numpy	https://numpy.org/	True	True	True	True	False	False	False	True	True	True	False	True	False	False
14	xgboost	ml	python	xgboost	https://xgboost.readthedocs.io/	True	True	True	True	False	False	False	True	True	True	False	True	False	False
15	auto_arima	statistical	python	statsforecast	https://nixtlaverse.nixtla.io/statsforecast/	True	True	True	True	False	False	False	True	True	True	False	False	False	False
16	auto_ces	statistical	python	statsforecast	https://nixtlaverse.nixtla.io/statsforecast/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
17	auto_theta	statistical	python	statsforecast	https://nixtlaverse.nixtla.io/statsforecast/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
18	autoets	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
19	croston	statistical	python	numpy	https://numpy.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
20	fft	statistical	python	scipy	https://scipy.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
21	holtwinters	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
22	kalman	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	True	True	True	False	False	False	False
23	naive_drift	statistical	python	numpy	https://numpy.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
24	naive_mean	statistical	python	numpy	https://numpy.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
25	naive_moving_average	statistical	python	numpy	https://numpy.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
26	naive_seasonal	statistical	python	numpy	https://numpy.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
27	prophet	statistical	python	prophet	https://facebook.github.io/prophet/	True	True	True	True	False	False	False	True	True	True	False	False	False	False
28	sarimax	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	True	True	True	False	False	False	False
29	stl_bagging	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
30	tbats	statistical	python	statsforecast	https://nixtlaverse.nixtla.io/statsforecast/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
31	theta	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	False	False	False	False	False	False	False
32	ucm	statistical	python	statsmodels	https://www.statsmodels.org/	True	True	True	True	False	False	False	True	True	True	False	False	False	False
33	vertex_l2l	automl	vertex_automl	google-cloud-pipeline-components	https://docs.cloud.google.com/gemini-enterpris...	True	False	False	False	True	True	False	True	True	True	True	True	True	False
34	vertex_seq2seq	automl	vertex_automl	google-cloud-pipeline-components	https://docs.cloud.google.com/gemini-enterpris...	True	False	False	False	True	True	False	True	True	True	True	True	True	False
35	vertex_tft	automl	vertex_automl	google-cloud-pipeline-components	https://docs.cloud.google.com/gemini-enterpris...	True	False	False	False	True	True	False	True	True	True	True	True	True	False
36	vertex_tide	automl	vertex_automl	google-cloud-pipeline-components	https://docs.cloud.google.com/gemini-enterpris...	True	False	False	False	True	True	False	True	True	True	True	True	True	False

In [3]:

metrics_df = metric_catalog()
print(f"Total registered metrics: {len(metrics_df)} "
      f"({(metrics_df['kind'] == 'point').sum()} point + {(metrics_df['kind'] == 'interval').sum()} interval)")
display(metrics_df)

metrics_df = metric_catalog() print(f"Total registered metrics: {len(metrics_df)} " f"({(metrics_df['kind'] == 'point').sum()} point + {(metrics_df['kind'] == 'interval').sum()} interval)") display(metrics_df)

Total registered metrics: 21 (16 point + 5 interval)

	metric	kind	direction	needs_intervals	needs_train_history	needs_seasonal_period	mean_optimal
0	mae	point	lower	False	False	False	False
1	rmse	point	lower	False	False	False	True
2	mse	point	lower	False	False	False	True
3	mape	point	lower	False	False	False	False
4	smape	point	lower	False	False	False	False
5	wape	point	lower	False	False	False	False
6	mase	point	lower	False	True	False	False
7	rmsse	point	lower	False	True	False	True
8	bias	point	zero	False	False	False	True
9	coverage	interval	higher	True	False	False	False
10	pinball	interval	lower	True	False	False	False
11	mase_seasonal	point	lower	False	True	True	False
12	maape	point	lower	False	False	False	False
13	interval_score	interval	lower	True	False	False	False
14	interval_width	interval	lower	True	False	False	False
15	ope	point	lower	False	False	False	False
16	rmsle	point	lower	False	False	False	False
17	msse	point	lower	False	True	False	True
18	msis	interval	lower	True	True	True	False
19	r2	point	higher	False	False	False	True
20	cv	point	lower	False	False	False	True

Act 3 · Deterministic Synthetic Panel with Exogenous Covariates & Hierarchy¶

sample_data() calls the exact same deterministic panel generator (scale_forecasting.data_gen.generator.generate_panel) used to seed the 100,000-series BigQuery benchmark tables. Passing with_exog=True and with_hierarchy=True adds:

• Static Hierarchy Attributes: region (NORTH, SOUTH, ...) and category (A, B, ...)
• Known-Future Covariates (future_covariates): promo_flag and price_index
• Observed-Past Covariate (past_covariates): temperature (known historically, unknown in the future horizon)

In [4]:

import matplotlib.pyplot as plt
from scale_forecasting.playground import sample_data

panel = sample_data(n_series=6, history=365, freq="D", with_exog=True, with_hierarchy=True)
print(f"Panel shape: {panel.shape} | Unique series: {panel['ts_id'].nunique()}")
print("Columns:", list(panel.columns))
display(panel.head())

fig, axes = plt.subplots(2, 3, figsize=(14, 5.5), sharex=True)
for ax, (ts_id, g) in zip(axes.flat, panel.groupby("ts_id")):
    archetype = g["archetype"].iloc[0]
    region = g["region"].iloc[0]
    cat = g["category"].iloc[0]
    ax.plot(pd.to_datetime(g["ds"]), g["y"], lw=1.1, color="#1967D2")
    ax.set_title(f"{ts_id} ({archetype} · {region}/{cat})", fontsize=9.5)
    ax.grid(alpha=0.25)
fig.suptitle("Synthetic Time-Series Archetypes (Daily Panel)", fontsize=12)
fig.tight_layout()
plt.show()

import matplotlib.pyplot as plt from scale_forecasting.playground import sample_data panel = sample_data(n_series=6, history=365, freq="D", with_exog=True, with_hierarchy=True) print(f"Panel shape: {panel.shape} | Unique series: {panel['ts_id'].nunique()}") print("Columns:", list(panel.columns)) display(panel.head()) fig, axes = plt.subplots(2, 3, figsize=(14, 5.5), sharex=True) for ax, (ts_id, g) in zip(axes.flat, panel.groupby("ts_id")): archetype = g["archetype"].iloc[0] region = g["region"].iloc[0] cat = g["category"].iloc[0] ax.plot(pd.to_datetime(g["ds"]), g["y"], lw=1.1, color="#1967D2") ax.set_title(f"{ts_id} ({archetype} · {region}/{cat})", fontsize=9.5) ax.grid(alpha=0.25) fig.suptitle("Synthetic Time-Series Archetypes (Daily Panel)", fontsize=12) fig.tight_layout() plt.show()

Panel shape: (2190, 9) | Unique series: 6
Columns: ['ts_id', 'archetype', 'region', 'category', 'ds', 'y', 'promo_flag', 'price_index', 'temperature']

	ts_id	archetype	region	category	ds	y	promo_flag	price_index	temperature
0	s_000000	smooth_seasonal	NA	enterprise	2021-01-01	233.786	1	107.555	9.374
1	s_000000	smooth_seasonal	NA	enterprise	2021-01-02	217.825	1	108.811	7.711
2	s_000000	smooth_seasonal	NA	enterprise	2021-01-03	229.825	0	110.024	11.371
3	s_000000	smooth_seasonal	NA	enterprise	2021-01-04	209.891	0	111.190	8.136
4	s_000000	smooth_seasonal	NA	enterprise	2021-01-05	161.967	0	112.304	6.044

Act 4 · Single-Series Fit, Rolling-Origin Backtest & Conformal Intervals¶

Let's run a single model (theta) on s_000000 with backtest=True (3 rolling-origin folds of 28 days each). Under the hood, run_model() validates the panel with validate_panel(), builds a RunConfig, and calls worker.run_cell():

1. Rolling-Origin Backtest (backtest_cell): Fits the model on each fold's training window, predicts the validation horizon, and scores all 21 evaluation metrics.
2. Conformal Interval Calibration (calibration.py): Uses out-of-fold signed residuals to calibrate empirical prediction bands (yhat_lower, yhat_upper) at the nominal 0.80 level.
3. Visual Inspection (build_predictions_frame + plot_forecasts_frame): Overlays historical actuals, out-of-fold backtest predictions, and the future forecast fan chart.

In [5]:

from scale_forecasting import (
    build_predictions_frame,
    explain_forecast_frame,
    plot_forecast_explanation,
    plot_forecasts_frame,
)
from scale_forecasting.playground import run_model, summarize

run = run_model("theta", data=panel, ts_id="s_000000", horizon=28, backtest=True)
print(summarize(run))

# Convert the single-cell result into the standard long-format predictions frame used by the SDK
pred_rows = (
    run.result.predictions.rename(columns={"ds": "forecast_date"})
    .assign(ts_id=run.result.ts_id, model_type=run.result.model_type)
    .to_dict("records")
)
oof_rows = (
    run.result.oof.rename(columns={"ds": "forecast_date"})
    .assign(ts_id=run.result.ts_id, model_type=run.result.model_type)
    .to_dict("records")
)
hist_rows = run.series[["ts_id", "ds", "y"]].to_dict("records")

pf = build_predictions_frame(pred_rows, oof_rows=oof_rows, history_rows=hist_rows)
fig = plot_forecasts_frame(pf, history_tail=120)
plt.show()

# 4-panel Forecast Explainability & Structural Attribution (Trend + Regime Shift + Seasonality + Exog + Uncertainty)
exog_df = run.series[["ts_id", "ds", "promo_flag", "price_index", "temperature"]]
exp_df = explain_forecast_frame(pf, covariate_df=exog_df, seasonal_period=7)
display(exp_df.tail(10))
plot_forecast_explanation(exp_df, history_tail=120)
plt.show()

from scale_forecasting import ( build_predictions_frame, explain_forecast_frame, plot_forecast_explanation, plot_forecasts_frame, ) from scale_forecasting.playground import run_model, summarize run = run_model("theta", data=panel, ts_id="s_000000", horizon=28, backtest=True) print(summarize(run)) # Convert the single-cell result into the standard long-format predictions frame used by the SDK pred_rows = ( run.result.predictions.rename(columns={"ds": "forecast_date"}) .assign(ts_id=run.result.ts_id, model_type=run.result.model_type) .to_dict("records") ) oof_rows = ( run.result.oof.rename(columns={"ds": "forecast_date"}) .assign(ts_id=run.result.ts_id, model_type=run.result.model_type) .to_dict("records") ) hist_rows = run.series[["ts_id", "ds", "y"]].to_dict("records") pf = build_predictions_frame(pred_rows, oof_rows=oof_rows, history_rows=hist_rows) fig = plot_forecasts_frame(pf, history_tail=120) plt.show() # 4-panel Forecast Explainability & Structural Attribution (Trend + Regime Shift + Seasonality + Exog + Uncertainty) exog_df = run.series[["ts_id", "ds", "promo_flag", "price_index", "temperature"]] exp_df = explain_forecast_frame(pf, covariate_df=exog_df, seasonal_period=7) display(exp_df.tail(10)) plot_forecast_explanation(exp_df, history_tail=120) plt.show()

model      : theta
series     : s_000000  (365 obs)
engine     : spark
status     : ok
forecast   : 28 steps
horizon    : 2022-01-01 → 2022-01-28
metrics    : mae=20.992, rmse=27.562, mse=868.198, mape=0.100, smape=0.109, wape=0.108, mase=0.806, rmsse=0.901, bias=-19.168, coverage=0.988, pinball=13.928, mase_seasonal=1.539, maape=0.099, interval_score=278.556, interval_width=278.461, ope=0.098, rmsle=0.138, msse=0.925, msis=20.459, r2=0.372, cv=0.141

	ts_id	model_type	segment	ds	y_true	yhat	yhat_lower	yhat_upper	trend_baseline	level_shift	seasonal_effect	covariate_effect	oof_residual	interval_width	cov_promo_flag	cov_price_index	cov_temperature
383	s_000000	theta	forecast	2022-01-19	NaN	229.731851	212.769441	291.902236	283.046109	-75.702006	-42.500641	4.050408	NaN	79.132795	4.437616	0.008648	-0.395856
384	s_000000	theta	forecast	2022-01-20	NaN	241.982758	222.942752	303.211413	283.473299	-75.702006	-34.427255	4.050408	NaN	80.268661	4.437616	0.008648	-0.395856
385	s_000000	theta	forecast	2022-01-21	NaN	285.889898	265.627669	345.896330	283.913297	-75.702006	1.917987	4.050408	NaN	80.268661	4.437616	0.008648	-0.395856
386	s_000000	theta	forecast	2022-01-22	NaN	326.950255	306.688027	386.956688	283.356285	-75.702006	33.684912	4.050408	NaN	80.268661	4.437616	0.008648	-0.395856
387	s_000000	theta	forecast	2022-01-23	NaN	336.288757	320.952732	396.376869	283.710607	-75.702006	41.596991	4.050408	NaN	75.424137	4.437616	0.008648	-0.395856
388	s_000000	theta	forecast	2022-01-24	NaN	310.767908	277.204897	370.856020	283.045334	-75.702006	20.950685	4.050408	NaN	93.651123	4.437616	0.008648	-0.395856
389	s_000000	theta	forecast	2022-01-25	NaN	251.882568	223.912853	305.535526	282.183880	-75.702006	-21.222680	4.050408	NaN	81.622673	4.437616	0.008648	-0.395856
390	s_000000	theta	forecast	2022-01-26	NaN	232.212102	198.194206	316.815728	274.722817	-75.702006	-42.500641	4.050408	NaN	118.621521	4.437616	0.008648	-0.395856
391	s_000000	theta	forecast	2022-01-27	NaN	237.325853	208.331255	326.952776	262.409629	-75.702006	-34.427255	4.050408	NaN	118.621521	4.437616	0.008648	-0.395856
392	s_000000	theta	forecast	2022-01-28	NaN	279.859717	250.865118	369.486640	250.320060	-75.702006	1.917987	4.050408	NaN	118.621521	4.437616	0.008648	-0.395856

Act 5 · Training Regimes Offline: local vs. global vs. hybrid¶

scale-forecasting supports three distinct training regimes via model_params[model]['training_mode']:

• local: Fits an independent model instance per (ts_id, model) cell (worker.run_cell).
• global: Pools all series in the panel into a single cross-series neural network (worker.run_panel with training_mode='global'), supported by all 5 deep_learning architectures (neuralprophet, tide, tsmixer, patchtst, tft).
• hybrid (Local + Global Shared-Plus-Series Network): Combines shared cross-series trend/seasonality representations with per-series local parameters (worker.run_panel with training_mode='hybrid', supported by neuralprophet).

In [6]:

from scale_forecasting.config import RunConfig
from scale_forecasting.worker import run_cell, run_panel

regime_cfg = RunConfig.model_validate({
    "run_name": "playground-regimes",
    "data": {"source_table": "playground", "freq": "D", "horizon": 14},
    "models": ["neuralprophet"],
    "model_params": {"neuralprophet": {"epochs": 15}},
    "backtest": {"enabled": True, "n_folds": 2, "horizon": 14, "step": 14},
})

# 1. Local per-series fit on all 6 series
local_results = [
    run_cell(g.reset_index(drop=True), "neuralprophet", regime_cfg)
    for _, g in panel.groupby("ts_id", sort=True)
]
# 2. Global pooled fit across all 6 series simultaneously
global_results = run_panel(panel, "neuralprophet", regime_cfg, params={"training_mode": "global", "epochs": 15})
# 3. Hybrid local+global fit across all 6 series
hybrid_results = run_panel(panel, "neuralprophet", regime_cfg, params={"training_mode": "hybrid", "epochs": 15})

comparison_rows = []
for mode_label, res_list in [("local", local_results), ("global", global_results), ("hybrid", hybrid_results)]:
    for r in res_list:
        comparison_rows.append({
            "training_mode": mode_label,
            "ts_id": r.ts_id,
            "wape": r.metrics.get("wape"),
            "mae": r.metrics.get("mae"),
            "mase": r.metrics.get("mase"),
            "coverage": r.metrics.get("coverage"),
        })
regime_df = pd.DataFrame(comparison_rows)
display(regime_df.groupby("training_mode")[["wape", "mae", "mase", "coverage"]].mean().reset_index())

from scale_forecasting.config import RunConfig from scale_forecasting.worker import run_cell, run_panel regime_cfg = RunConfig.model_validate({ "run_name": "playground-regimes", "data": {"source_table": "playground", "freq": "D", "horizon": 14}, "models": ["neuralprophet"], "model_params": {"neuralprophet": {"epochs": 15}}, "backtest": {"enabled": True, "n_folds": 2, "horizon": 14, "step": 14}, }) # 1. Local per-series fit on all 6 series local_results = [ run_cell(g.reset_index(drop=True), "neuralprophet", regime_cfg) for _, g in panel.groupby("ts_id", sort=True) ] # 2. Global pooled fit across all 6 series simultaneously global_results = run_panel(panel, "neuralprophet", regime_cfg, params={"training_mode": "global", "epochs": 15}) # 3. Hybrid local+global fit across all 6 series hybrid_results = run_panel(panel, "neuralprophet", regime_cfg, params={"training_mode": "hybrid", "epochs": 15}) comparison_rows = [] for mode_label, res_list in [("local", local_results), ("global", global_results), ("hybrid", hybrid_results)]: for r in res_list: comparison_rows.append({ "training_mode": mode_label, "ts_id": r.ts_id, "wape": r.metrics.get("wape"), "mae": r.metrics.get("mae"), "mase": r.metrics.get("mase"), "coverage": r.metrics.get("coverage"), }) regime_df = pd.DataFrame(comparison_rows) display(regime_df.groupby("training_mode")[["wape", "mae", "mase", "coverage"]].mean().reset_index())

	training_mode	wape	mae	mase	coverage
0	global	0.504413	23.290220	1.680554	0.738095
1	hybrid	0.386277	22.470124	1.126287	0.672619
2	local	0.406539	22.152474	1.110474	0.964286

Act 6 · Hierarchical Aggregation & All 7 FPP3 Reconciliation Methods Offline¶

When hierarchy.enabled=True, build_hierarchy() rolls the bottom-level series (ts_id) up every level of the tree (__total__ $\rightarrow$ region $\rightarrow$ region/category $\rightarrow$ ts_id) and constructs the summing matrix $S$. After base models forecast every node independently, reconcile_forecasts() projects the incoherent base forecasts $\hat{y}$ onto the coherent subspace $\tilde{y} = S G \hat{y}$ using any of the 7 FPP3 reconciliation estimators:

• bottom_up, top_down, middle_out, ols, wls_struct, wls_var, mint_shrink

In [7]:

from scale_forecasting import (
    build_hierarchy,
    build_hierarchy_frame,
    plot_hierarchy_frame,
    reconcile_forecasts,
    verify_coherence,
)

hier_cfg = RunConfig.model_validate({
    "run_name": "playground-hierarchy",
    "data": {"source_table": "playground", "freq": "D", "horizon": 14},
    "models": ["theta"],
    "hierarchy": {
        "enabled": True,
        "levels": [["region"], ["region", "category"]],
        "reconciliation_methods": ["bottom_up", "top_down", "middle_out", "ols", "wls_struct", "wls_var", "mint_shrink"],
        "middle_level": ["region"],
    },
    "backtest": {"enabled": True, "n_folds": 2, "horizon": 14, "step": 14},
})
full_panel, spec = build_hierarchy(panel, hier_cfg)
print(f"Hierarchy nodes: {spec.n_nodes} total ({spec.n_bottom} bottom + {spec.n_aggregated} aggregate) | S shape: {spec.summing_matrix.shape}")

# Fit a fast statistical model ('theta') on every node in the hierarchy
node_cells = [
    run_cell(g.reset_index(drop=True), "theta", hier_cfg)
    for _, g in full_panel.groupby("ts_id", sort=True)
]
base_preds = pd.concat([
    c.predictions.rename(columns={"ds": "forecast_date"}).assign(ts_id=c.ts_id, model_type=c.model_type)
    for c in node_cells
], ignore_index=True)
base_oof = pd.concat([
    c.oof.rename(columns={"ds": "forecast_date"}).assign(ts_id=c.ts_id, model_type=c.model_type)
    for c in node_cells if c.oof is not None
], ignore_index=True)

# Compare all 7 FPP3 reconciliation methods and verify exact mathematical coherence
methods = ["bottom_up", "top_down", "middle_out", "ols", "wls_struct", "wls_var", "mint_shrink"]
coherence_rows = []
for m in methods:
    rec_df = reconcile_forecasts(
        base_preds, spec, m,
        history_df=full_panel, oof_df=base_oof,
        middle_level=["region"] if m == "middle_out" else None,
    )
    max_discrepancy = verify_coherence(rec_df, spec)
    coherence_rows.append({"method": m, "max_coherence_error": max_discrepancy, "coherent": max_discrepancy < 1e-6})

display(pd.DataFrame(coherence_rows))

# Visualize node rollups across hierarchy levels for mint_shrink
mint_df = reconcile_forecasts(base_preds, spec, "mint_shrink", history_df=full_panel, oof_df=base_oof)
hf = build_hierarchy_frame(mint_df.to_dict("records"))
display(hf)
plot_hierarchy_frame(mint_df)
plt.show()

from scale_forecasting import ( build_hierarchy, build_hierarchy_frame, plot_hierarchy_frame, reconcile_forecasts, verify_coherence, ) hier_cfg = RunConfig.model_validate({ "run_name": "playground-hierarchy", "data": {"source_table": "playground", "freq": "D", "horizon": 14}, "models": ["theta"], "hierarchy": { "enabled": True, "levels": [["region"], ["region", "category"]], "reconciliation_methods": ["bottom_up", "top_down", "middle_out", "ols", "wls_struct", "wls_var", "mint_shrink"], "middle_level": ["region"], }, "backtest": {"enabled": True, "n_folds": 2, "horizon": 14, "step": 14}, }) full_panel, spec = build_hierarchy(panel, hier_cfg) print(f"Hierarchy nodes: {spec.n_nodes} total ({spec.n_bottom} bottom + {spec.n_aggregated} aggregate) | S shape: {spec.summing_matrix.shape}") # Fit a fast statistical model ('theta') on every node in the hierarchy node_cells = [ run_cell(g.reset_index(drop=True), "theta", hier_cfg) for _, g in full_panel.groupby("ts_id", sort=True) ] base_preds = pd.concat([ c.predictions.rename(columns={"ds": "forecast_date"}).assign(ts_id=c.ts_id, model_type=c.model_type) for c in node_cells ], ignore_index=True) base_oof = pd.concat([ c.oof.rename(columns={"ds": "forecast_date"}).assign(ts_id=c.ts_id, model_type=c.model_type) for c in node_cells if c.oof is not None ], ignore_index=True) # Compare all 7 FPP3 reconciliation methods and verify exact mathematical coherence methods = ["bottom_up", "top_down", "middle_out", "ols", "wls_struct", "wls_var", "mint_shrink"] coherence_rows = [] for m in methods: rec_df = reconcile_forecasts( base_preds, spec, m, history_df=full_panel, oof_df=base_oof, middle_level=["region"] if m == "middle_out" else None, ) max_discrepancy = verify_coherence(rec_df, spec) coherence_rows.append({"method": m, "max_coherence_error": max_discrepancy, "coherent": max_discrepancy < 1e-6}) display(pd.DataFrame(coherence_rows)) # Visualize node rollups across hierarchy levels for mint_shrink mint_df = reconcile_forecasts(base_preds, spec, "mint_shrink", history_df=full_panel, oof_df=base_oof) hf = build_hierarchy_frame(mint_df.to_dict("records")) display(hf) plot_hierarchy_frame(mint_df) plt.show()

Hierarchy nodes: 17 total (6 bottom + 11 aggregate) | S shape: (17, 6)

	method	max_coherence_error	coherent
0	bottom_up	1.136868e-13	True
1	top_down	1.136868e-13	True
2	middle_out	1.136868e-13	True
3	ols	3.410605e-13	True
4	wls_struct	3.410605e-13	True
5	wls_var	1.136868e-13	True
6	mint_shrink	2.273737e-13	True

	model_type	level	n_series	n_points	mean_yhat	sum_yhat	max_coherence_residual
0	theta_mint_shrink	total	1	14	830.535936	11627.503106	2.273737e-13
1	theta_mint_shrink	aggregate	10	140	166.107187	23255.006212	2.273737e-13
2	theta_mint_shrink	bottom	6	84	138.422656	11627.503106	2.273737e-13

Act 7 · Multi-Model Bake-Off, Two-Tier Feature Attributions, Ensemble Weights & Forecaster.explain()¶

bakeoff() runs multiple candidate models head-to-head on a single series and combines them using both:

• Calculated Ensembling (inverse_error): Weights each model inversely proportional to its validation loss.
• Learned Stacking (nnls): Fits non-negative least squares weights across out-of-fold backtest predictions.

We also inspect Two-Tier Feature Attributions (build_attributions_frame() and plot_attributions() — Tier 1 global driver importance and Tier 2 per-horizon-step TreeSHAP attributions for xgboost), visualize learned stacking weights with plot_ensemble_weights(), and preview Forecaster.explain(validate_data=False) on a 5-family RunConfig before any cloud job is launched.

In [8]:

from scale_forecasting import (
    Forecaster,
    build_attributions_frame,
    build_ensemble_weights_frame,
    plot_attributions,
    plot_ensemble_weights,
)
from scale_forecasting.ensembler import fit_learned
from scale_forecasting.playground import bakeoff

bo = bakeoff(data=panel, ts_id="s_000000", models=["theta", "holtwinters", "stl_bagging", "xgboost"], horizon=28, n_folds=2)
display(bo.leaderboard[["model", "kind", "status", "wape", "mae", "rmse", "mase", "coverage", "interval_score"]])

# Fit learned stackers (nnls, ridge, xgb) on the bake-off base models to inspect and plot stacking weights
stack_cfg = bo.config.model_copy(update={"ensemble": bo.config.ensemble.model_copy(update={"strategies": ["nnls", "ridge", "xgb"]})})
oof_frames = []
xgb_cell = None
for m in ["theta", "holtwinters", "stl_bagging", "xgboost"]:
    c = run_cell(bo.series, m, stack_cfg)
    if m == "xgboost":
        xgb_cell = c
    if c.oof is not None:
        oof_frames.append(c.oof.rename(columns={"ds": "forecast_date"}).assign(ts_id="s_000000", model_type=m))

# Inspect Two-Tier Feature Attributions (Tier 1 global + Tier 2 per-horizon-step TreeSHAP) on xgboost
global_attr_df = build_attributions_frame(
    [{"ts_id": "s_000000", "model_type": "xgboost", "fit_diagnostics": xgb_cell.diagnostics}],
    level="global",
)
local_attr_df = build_attributions_frame(
    [
        {"ts_id": "s_000000", "model_type": "xgboost", "forecast_date": row["ds"], "yhat": row["yhat"], "explanations": exp}
        for row, exp in zip(xgb_cell.predictions.to_dict("records"), xgb_cell.explanations or [])
    ],
    level="local",
)
display(global_attr_df.head(8))
plot_attributions(global_attr_df, model_type="xgboost", ts_id="s_000000", title="Tier 1 Global TreeSHAP Importance (xgboost)")
plt.show()
plot_attributions(local_attr_df, model_type="xgboost", ts_id="s_000000", title="Tier 2 Local Per-Step TreeSHAP Attributions (xgboost)")
plt.show()

learned_w, _, _ = fit_learned(pd.concat(oof_frames, ignore_index=True), stack_cfg)
weight_rows = [
    {"ts_id": "s_000000", "model_type": f"ensemble_{strat}", "is_ensemble": True, "best_params": wmap}
    for strat, wmap in learned_w.items()
]
w_df = build_ensemble_weights_frame(weight_rows)
display(w_df)
plot_ensemble_weights(w_df)
plt.show()

# Preview Forecaster.explain() on a 5-family production RunConfig (no cloud needed with validate_data=False)
preview_cfg = RunConfig.model_validate({
    "run_name": "enterprise-preview",
    "python_runtime": "spark",
    "data": {"source_table": "source_series_iceberg", "horizon": 28, "series_limit": 500},
    "models": ["theta", "autoets", "lightgbm", "xgboost", "neuralprophet", "vertex_tide", "arima_plus", "timesfm"],
    "compute": {
        "families": {
            "deep_learning": {"runtime": "ray", "hardware": "gpu", "gpu_type": "L4", "min_workers": 1, "max_workers": 2},
            "automl": {"runtime": "vertex_automl", "automl_mode": "tabular_workflow", "hardware": "gpu", "gpu_type": "L4"},
        },
    },
    "features": {"holidays": ["US"], "future_covariates": ["promo_flag"]},
    "backtest": {"enabled": True, "n_folds": 2, "horizon": 28, "step": 28},
    "ensemble": {"enabled": True, "strategies": ["inverse_error", "nnls", "ridge"]},
})
plan_df = Forecaster(preview_cfg).explain(validate_data=False)
display(plan_df)

from scale_forecasting import ( Forecaster, build_attributions_frame, build_ensemble_weights_frame, plot_attributions, plot_ensemble_weights, ) from scale_forecasting.ensembler import fit_learned from scale_forecasting.playground import bakeoff bo = bakeoff(data=panel, ts_id="s_000000", models=["theta", "holtwinters", "stl_bagging", "xgboost"], horizon=28, n_folds=2) display(bo.leaderboard[["model", "kind", "status", "wape", "mae", "rmse", "mase", "coverage", "interval_score"]]) # Fit learned stackers (nnls, ridge, xgb) on the bake-off base models to inspect and plot stacking weights stack_cfg = bo.config.model_copy(update={"ensemble": bo.config.ensemble.model_copy(update={"strategies": ["nnls", "ridge", "xgb"]})}) oof_frames = [] xgb_cell = None for m in ["theta", "holtwinters", "stl_bagging", "xgboost"]: c = run_cell(bo.series, m, stack_cfg) if m == "xgboost": xgb_cell = c if c.oof is not None: oof_frames.append(c.oof.rename(columns={"ds": "forecast_date"}).assign(ts_id="s_000000", model_type=m)) # Inspect Two-Tier Feature Attributions (Tier 1 global + Tier 2 per-horizon-step TreeSHAP) on xgboost global_attr_df = build_attributions_frame( [{"ts_id": "s_000000", "model_type": "xgboost", "fit_diagnostics": xgb_cell.diagnostics}], level="global", ) local_attr_df = build_attributions_frame( [ {"ts_id": "s_000000", "model_type": "xgboost", "forecast_date": row["ds"], "yhat": row["yhat"], "explanations": exp} for row, exp in zip(xgb_cell.predictions.to_dict("records"), xgb_cell.explanations or []) ], level="local", ) display(global_attr_df.head(8)) plot_attributions(global_attr_df, model_type="xgboost", ts_id="s_000000", title="Tier 1 Global TreeSHAP Importance (xgboost)") plt.show() plot_attributions(local_attr_df, model_type="xgboost", ts_id="s_000000", title="Tier 2 Local Per-Step TreeSHAP Attributions (xgboost)") plt.show() learned_w, _, _ = fit_learned(pd.concat(oof_frames, ignore_index=True), stack_cfg) weight_rows = [ {"ts_id": "s_000000", "model_type": f"ensemble_{strat}", "is_ensemble": True, "best_params": wmap} for strat, wmap in learned_w.items() ] w_df = build_ensemble_weights_frame(weight_rows) display(w_df) plot_ensemble_weights(w_df) plt.show() # Preview Forecaster.explain() on a 5-family production RunConfig (no cloud needed with validate_data=False) preview_cfg = RunConfig.model_validate({ "run_name": "enterprise-preview", "python_runtime": "spark", "data": {"source_table": "source_series_iceberg", "horizon": 28, "series_limit": 500}, "models": ["theta", "autoets", "lightgbm", "xgboost", "neuralprophet", "vertex_tide", "arima_plus", "timesfm"], "compute": { "families": { "deep_learning": {"runtime": "ray", "hardware": "gpu", "gpu_type": "L4", "min_workers": 1, "max_workers": 2}, "automl": {"runtime": "vertex_automl", "automl_mode": "tabular_workflow", "hardware": "gpu", "gpu_type": "L4"}, }, }, "features": {"holidays": ["US"], "future_covariates": ["promo_flag"]}, "backtest": {"enabled": True, "n_folds": 2, "horizon": 28, "step": 28}, "ensemble": {"enabled": True, "strategies": ["inverse_error", "nnls", "ridge"]}, }) plan_df = Forecaster(preview_cfg).explain(validate_data=False) display(plan_df)

	model	kind	status	wape	mae	rmse	mase	coverage	interval_score
0	holtwinters	base	ok	0.098104	19.716912	26.835893	0.755259	0.500000	119.839309
1	ensemble_inverse_error	ensemble	ok	0.106172	21.544101	29.359942	0.825276	NaN	NaN
2	xgboost	base	ok	0.112572	22.841938	31.155169	0.874991	0.053571	224.679197
3	stl_bagging	base	ok	0.113094	23.052544	30.493329	0.883072	0.375000	178.833948
4	theta	base	ok	0.115866	23.499630	31.542060	0.900184	0.982143	283.769321
5	ensemble_nnls	ensemble	ok	0.115866	23.499630	31.542060	0.900184	NaN	NaN

	ts_id	model_type	compute_engine	feature	importance
0	s_000000	xgboost	None	lag_7	0.862970
1	s_000000	xgboost	None	month	0.032247
2	s_000000	xgboost	None	lag_14	0.031003
3	s_000000	xgboost	None	dow	0.018567
4	s_000000	xgboost	None	doy	0.016951
5	s_000000	xgboost	None	lag_1	0.011887
6	s_000000	xgboost	None	lag_28	0.007921
7	s_000000	xgboost	None	lag_3	0.006987

	ts_id	ensemble_model	strategy	base_model	weight	wape
0	s_000000	ensemble_nnls	nnls	holtwinters	0.000000	None
1	s_000000	ensemble_nnls	nnls	stl_bagging	0.000000	None
2	s_000000	ensemble_nnls	nnls	theta	1.085770	None
3	s_000000	ensemble_nnls	nnls	xgboost	0.000000	None
4	s_000000	ensemble_ridge	ridge	holtwinters	0.416270	None
5	s_000000	ensemble_ridge	ridge	stl_bagging	-0.330450	None
6	s_000000	ensemble_ridge	ridge	theta	1.114132	None
7	s_000000	ensemble_ridge	ridge	xgboost	-0.102504	None
8	s_000000	ensemble_xgb	xgb	holtwinters	0.000000	None
9	s_000000	ensemble_xgb	xgb	stl_bagging	0.033383	None
10	s_000000	ensemble_xgb	xgb	theta	0.848006	None
11	s_000000	ensemble_xgb	xgb	xgboost	0.118612	None

	run_id	family	job_key	runtime	machine_type	workers	hardware	gpu_type	models	training_modes	n_series	n_folds	expected_cells	covariates	depends_on
0	enterprise-preview-6669e98516e2	statistical	sf-enterprise-preview-6669e98516e2-statistical-a1	spark	None	NaN	cpu	None	theta, autoets	theta:local, autoets:local	500	2	1000	future(1)
1	enterprise-preview-6669e98516e2	ml	sf-enterprise-preview-6669e98516e2-ml-a1	spark	None	NaN	cpu	None	lightgbm, xgboost	lightgbm:local, xgboost:local	500	2	1000	future(1)
2	enterprise-preview-6669e98516e2	deep_learning	sf-enterprise-preview-6669e98516e2-deep_learni...	ray	None	NaN	gpu	L4	neuralprophet	neuralprophet:local	500	2	500	future(1)
3	enterprise-preview-6669e98516e2	automl	sf-enterprise-preview-6669e98516e2-automl-a1	vertex_automl	None	NaN	gpu	L4	vertex_tide	vertex_tide:global	500	2	500	future(1)
4	enterprise-preview-6669e98516e2	native	sf-enterprise-preview-6669e98516e2-native-a1	bigquery	sql	1.0	sql	None	arima_plus, timesfm	arima_plus:local, timesfm:local	500	2	1000	future(1)
5	enterprise-preview-6669e98516e2	ensemble	sf-enterprise-preview-6669e98516e2-ensemble-a1	bigquery	sql	1.0	sql	None	ensemble_inverse_error, ensemble_nnls, ensembl...	barrier	500	2	1500	future(1)	sf-enterprise-preview-6669e98516e2-statistical...

Act 8 · Forecaster Workbench: Quantiles, FVA, Horizon Decay, Promo/Price Scenarios, Inventory Policy & Exception Triage¶

Beyond model fitting and stacking, the Forecaster Workbench (docs/forecaster_workbench.md) gives demand planners, merchandisers, and econometricians direct access to:

1. Forecast Value Added (bo.fva_df()): Step-by-step accuracy gain over naive_seasonal with Harvey–Leybourne–Newbold (1997) modified Diebold–Mariano (dm_pvalue) and paired Wilcoxon significance tests.
2. Lead-Time Horizon Decay (bo.horizon_decay_df()) & Trigg Bias Audit (bo.bias_df()): Per-step error degradation ($h = 1 \dots H$) and chronic directional bias tracking signals.
3. Autocorrelation-Aware Time Rollups (bo.rollup_time(freq="W")) & Exception Triage (bo.exceptions_df()): Weekly replenishment totals with empirical AR(1) error covariance intervals and composite severity triage.
4. Operational Inventory & Replenishment Policy Translation (bo.inventory_policy_df()): Translates calibrated forward forecasts, empirical AR(1) residual autocorrelation ($\hat{\rho}_1$), and supplier lead-time variability ($\mu_L, \sigma_L$) into continuous-review $(s, S)$ and periodic-review $(R, S)$ Reorder Points (ROP), Safety Stock (SS), Order-Up-To Levels ($S$), Recommended Order Quantities ($Q$), and Newsvendor Holding vs. Stockout Cost tables.
5. Counterfactual Promo & Price Scenario Simulation (simulate_scenarios): Evaluate promotional lifts and price elasticity shocks (+, *, =) with revenue impact without refitting.

In [8]:

from scale_forecasting import compute_demand_profile, summarize_scenarios
from scale_forecasting.playground import run_model

# 1. Demand regime profile (Syntetos-Boylan-Croston ADI vs CV^2 classification)
profiles_df = pd.DataFrame(
    [{"ts_id": tid, **compute_demand_profile(g["y"])} for tid, g in panel.groupby("ts_id")]
)
display(profiles_df)

# 2. Forecast Value Added (FVA) waterfall with modified Diebold-Mariano p-values, bias audit & weekly rollup
wb_bo = bakeoff(
    data=panel,
    ts_id="s_000000",
    models=["naive_seasonal", "theta", "holtwinters", "xgboost"],
    horizon=14,
    n_folds=3,
    floor=0.0,
    quantiles=[0.10, 0.50, 0.90, 0.95],
)
display(wb_bo.fva_df()[["model_type", "fva_stage", "mean_metric", "fva_pct", "p_value_dm", "significant_at_05"]])
display(wb_bo.bias_df()[["ts_id", "model_type", "bias_pct", "tracking_signal", "bias_status"]])
display(wb_bo.rollup_time(freq="W").head(6))

# 3. Operational Inventory & Replenishment Policy (ROP, Safety Stock, Order-Up-To & Newsvendor Cost)
inv_policy_df = wb_bo.inventory_policy_df(
    lead_time=7.0,
    lead_time_std=1.0,
    service_level=0.95,
    on_hand=800.0,
    on_order=200.0,
    order_multiple=10.0,
)
display(
    inv_policy_df[
        [
            "ts_id",
            "model_type",
            "lead_time_demand",
            "error_autocorr_rho",
            "safety_stock",
            "naive_iid_safety_stock",
            "reorder_point",
            "recommended_order_qty",
            "expected_total_cost",
        ]
    ]
)

# 4. Counterfactual "What-If" Promo & Price Simulator with Revenue Impact
promo_run = run_model(
    "regression_lags",
    data=panel,
    ts_id="s_000000",
    horizon=14,
    future_covariates=["promo_flag", "price_index"],
    floor=0.0,
)
scen_df = promo_run.simulate_scenarios(
    scenarios={
        "promo_push_10pct_off": {"promo_flag": ("=", 1.0), "price_index": ("*", 0.90)},
        "price_increase_8pct": {"promo_flag": ("=", 0.0), "price_index": ("*", 1.08)},
    },
    price_col="price_index",
)
display(summarize_scenarios(scen_df))

from scale_forecasting import compute_demand_profile, summarize_scenarios from scale_forecasting.playground import run_model # 1. Demand regime profile (Syntetos-Boylan-Croston ADI vs CV^2 classification) profiles_df = pd.DataFrame( [{"ts_id": tid, **compute_demand_profile(g["y"])} for tid, g in panel.groupby("ts_id")] ) display(profiles_df) # 2. Forecast Value Added (FVA) waterfall with modified Diebold-Mariano p-values, bias audit & weekly rollup wb_bo = bakeoff( data=panel, ts_id="s_000000", models=["naive_seasonal", "theta", "holtwinters", "xgboost"], horizon=14, n_folds=3, floor=0.0, quantiles=[0.10, 0.50, 0.90, 0.95], ) display(wb_bo.fva_df()[["model_type", "fva_stage", "mean_metric", "fva_pct", "p_value_dm", "significant_at_05"]]) display(wb_bo.bias_df()[["ts_id", "model_type", "bias_pct", "tracking_signal", "bias_status"]]) display(wb_bo.rollup_time(freq="W").head(6)) # 3. Operational Inventory & Replenishment Policy (ROP, Safety Stock, Order-Up-To & Newsvendor Cost) inv_policy_df = wb_bo.inventory_policy_df( lead_time=7.0, lead_time_std=1.0, service_level=0.95, on_hand=800.0, on_order=200.0, order_multiple=10.0, ) display( inv_policy_df[ [ "ts_id", "model_type", "lead_time_demand", "error_autocorr_rho", "safety_stock", "naive_iid_safety_stock", "reorder_point", "recommended_order_qty", "expected_total_cost", ] ] ) # 4. Counterfactual "What-If" Promo & Price Simulator with Revenue Impact promo_run = run_model( "regression_lags", data=panel, ts_id="s_000000", horizon=14, future_covariates=["promo_flag", "price_index"], floor=0.0, ) scen_df = promo_run.simulate_scenarios( scenarios={ "promo_push_10pct_off": {"promo_flag": ("=", 1.0), "price_index": ("*", 0.90)}, "price_increase_8pct": {"promo_flag": ("=", 0.0), "price_index": ("*", 1.08)}, }, price_col="price_index", ) display(summarize_scenarios(scen_df))

	ts_id	demand_class	adi	cv2	zero_share	n_obs
0	s_000000	smooth	1.0000	0.0800	0.0000	730
1	s_000001	intermittent	2.2462	0.1500	0.5548	730
2	s_000002	smooth	1.0000	0.0461	0.0000	730
3	s_000003	smooth	1.0000	0.2359	0.0000	730
4	s_000004	smooth	1.0083	0.1891	0.0082	730
5	s_000005	smooth	1.0000	0.1186	0.0000	730

	model_type	fva_stage	mean_metric	fva_pct	p_value_dm	significant_at_05
0	naive_seasonal	Baseline	0.071739	0.000000	1.000000	False
1	holtwinters	Statistical	0.058537	0.184019	0.359089	False
2	theta	Statistical	0.066424	0.074080	0.188744	False
3	xgboost	Machine Learning	0.064970	0.094352	0.345297	False
4	ensemble_inverse_error	Ensemble	0.059426	0.171636	0.004795	True
5	ensemble_nnls	Ensemble	0.065205	0.091070	0.001612	True

	ts_id	model_type	bias_pct	tracking_signal	bias_status
0	s_000000	theta	-0.053917	-34.091514	under_forecasting
1	s_000000	ensemble_nnls	-0.051720	-33.313469	under_forecasting
2	s_000000	naive_seasonal	-0.056755	-33.227891	under_forecasting
3	s_000000	ensemble_inverse_error	-0.037186	-26.281689	unbiased
4	s_000000	xgboost	-0.036031	-23.292384	unbiased
5	s_000000	holtwinters	-0.003387	-2.430432	unbiased

	ts_id	model_type	period_start	period_end	n_steps	y_true	yhat	yhat_lower	yhat_upper	naive_sum_lower	naive_sum_upper	error_autocorr_rho
0	s_000000	ensemble_inverse_error	2023-01-01	2023-01-01	1	NaN	292.208249	280.092924	314.614253	280.092924	314.614253	0.377949
1	s_000000	ensemble_inverse_error	2023-01-02	2023-01-08	7	NaN	1655.708980	1602.855254	1854.850609	1555.951050	2022.469913	0.377949
2	s_000000	ensemble_inverse_error	2023-01-09	2023-01-14	6	NaN	1392.984432	1329.961899	1536.881139	1283.885456	1632.310430	0.377949
3	s_000000	ensemble_nnls	2023-01-01	2023-01-01	1	NaN	288.020138	277.938276	313.606009	277.938276	313.606009	0.359005
4	s_000000	ensemble_nnls	2023-01-02	2023-01-08	7	NaN	1645.013399	1605.525548	1847.779562	1568.842700	2025.983165	0.359005
5	s_000000	ensemble_nnls	2023-01-09	2023-01-14	6	NaN	1382.315167	1341.492579	1536.089029	1311.295406	1648.179544	0.359005

	ts_id	model_type	lead_time_demand	error_autocorr_rho	safety_stock	naive_iid_safety_stock	reorder_point	recommended_order_qty	expected_total_cost
0	s_000000	ensemble_inverse_error	1650.482077	0.377949	450.680078	422.207910	2101.162155	1110.0	507.925539
1	s_000000	ensemble_nnls	1640.297870	0.359005	451.307379	422.616487	2091.605249	1100.0	508.632520
2	s_000000	holtwinters	1643.936899	0.497734	456.108679	417.395214	2100.045578	1110.0	514.043682
3	s_000000	naive_seasonal	1707.582000	0.306857	435.153504	421.798814	2142.735504	1150.0	490.426777
4	s_000000	theta	1622.666062	0.384011	437.664971	411.620303	2060.331033	1070.0	493.257250
5	s_000000	xgboost	1633.306763	0.302813	445.792859	422.139372	2079.099621	1080.0	502.417544

	scenario	ts_id	model_type	n_steps	baseline_units	scenario_units	delta_units	delta_pct	baseline_revenue	scenario_revenue	revenue_delta	revenue_delta_pct	mean_implied_elasticity
0	baseline	s_000000	regression_lags	14	3450.257697	3450.257697	0.000000	0.000000	320083.21338	320083.213380	0.000000	0.000000	NaN
1	promo_push_10pct_off	s_000000	regression_lags	14	3450.257697	3715.734916	265.477219	0.076944	320083.21338	310395.311129	-9687.902251	-0.030267	-0.824643
2	price_increase_8pct	s_000000	regression_lags	14	3450.257697	3363.957188	-86.300509	-0.025013	320083.21338	337343.451276	17260.237896	0.053924	-0.284181

---

Where next¶

• Next in Track 1 (Local Sandbox & Plugins): 09_custom_models_and_metrics.ipynb — author custom 1-file BaseModel and BaseMetric plugins in local memory.
• Move to Track 2 (Cloud Runtimes): 01_bigquery_native_sql.ipynb — execute ARIMA_PLUS and TimesFM 2.0 (AI.FORECAST) directly in BigQuery SQL.
• Documentation: Getting started · Models reference · Evaluation metrics reference · Cost estimates & controls