Feature Engineering¶
features
¶
Feature engineering for the Python models — pure.
Everything here is declarative-from-config and runs inside the execution node, so no data crosses the network. Two entry points:
build_features(series, cfg) -> (y, X)— turn one raw series frame into the fit inputs:y(target Series indexed by ds, transform applied) andX(exog + holiday- Fourier + level-shift + lag columns aligned to
y, or None when no feature is configured). build_future_features(y, X, cfg) -> DataFrame | None— the forecast-horizon design frame, indexed by the future dates, with the same columns in the same order.holiday_frame(cfg) -> DataFrame— the one canonical holiday calendar (ds,holidaycolumns) computed fromfeatures.holidays, fed to both Python models and the BQML custom-holiday input so "holiday" is identical everywhere.
Two frames, because a forecast row is not a history row. Most of what we build is a
deterministic function of the date — holiday flags, Fourier phase, the level-shift step —
so the horizon's true values are computable, not guessable, and computing them is the only
way an exog-aware model sees the right seasonal phase for the dates it is forecasting. Only
user-supplied features.exog is genuinely unknown ahead of time; that alone falls back to
a documented stand-in. Backtesting never needed this (its "future" is in-sample, so
backtest_cell slices the real X) — which is exactly why the gap was easy to miss: the
folds scored on correct features while the shipped forecast did not.
Transforms come in two flavors. none/log1p are stateless — a model inverts with
only ctx.transform (a name). boxcox is stateful: its λ is fit per series (MLE),
so it must travel with the cell. We fit λ once in the worker (fit_transform_lambda),
put it on ctx.transform_lambda, and every apply/invert call passes that same λ —
never refit at predict, so the backtest folds and the final fit use one λ. λ = None
for the stateless transforms.
Public surface: build_features, build_future_features, holiday_frame,
apply_transform, invert_transform, fit_transform_lambda, level_shift_step.
ModelVariantSpec
dataclass
¶
ModelVariantSpec(
model_name,
base_model,
variant_tag,
scope_capability,
training_mode,
cov_profile,
is_univariate,
covariates_used,
effective_cfg,
clean_params,
model_cls,
intended_features=None,
)
Resolved execution contract and orthogonal taxonomy for one models entry.
apply_transform
¶
Apply the configured target transform to y (forward direction).
lam is the fitted Box-Cox λ from fit_transform_lambda (ignored by the stateless
transforms). Required when transform == "boxcox".
Source code in src/scale_forecasting/features.py
build_features
¶
Build (y, X) fit inputs for one series.
series is one ts_id's rows with the configured date/target (and optional covariates)
columns. y is returned indexed by ds, sorted, with fold-safe target hygiene
(grid_policy, trim_leading_zeros, censored_col, outlier_policy) and the transform
applied, and with the index's frequency set whenever the dates are regular (see
_with_inferred_freq). X carries any configured dynamic covariates (exog,
future_covariates, past_covariates), an is_holiday flag, Fourier terms,
trading_days terms, level_shift, is_outlier (when outlier_policy="flag"),
and lagged covariates — aligned to y — or None when nothing is configured.
Source code in src/scale_forecasting/features.py
1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 | |
build_future_features
¶
The design frame for the forecast horizon, indexed by the future dates.
Source code in src/scale_forecasting/features.py
clean_model_params
¶
Return a copy of params with per-variant feature-override meta-keys stripped (pure).
Source code in src/scale_forecasting/features.py
clean_training_target
¶
Apply fold-safe censored demand recovery (censored_col) and outlier treatment
(outlier_policy) strictly on the provided training window (pure).
Returns (cleaned_y, outlier_flag_array_or_None).
Source code in src/scale_forecasting/features.py
1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 | |
compute_cov_profile
¶
Compute the realized covariate profile after narrowing to model_cls capabilities.
Source code in src/scale_forecasting/features.py
compute_demand_profile
¶
Syntetos–Boylan–Croston (2005) demand regime classification (pure).
Computes the Average Demand Interval (adi = n_obs / n_nonzero), the squared coefficient of
variation of non-zero demand (cv2 = (std / mean)^2), zero_share, and n_obs, and
assigns one of five canonical regimes:
"cold_start":n_obs < min_cold_start_obsor fewer than 3 non-zero observations"smooth":adi < 1.32andcv2 < 0.49"erratic":adi < 1.32andcv2 >= 0.49"intermittent":adi >= 1.32andcv2 < 0.49"lumpy":adi >= 1.32andcv2 >= 0.49
Source code in src/scale_forecasting/features.py
effective_config_for_model
¶
Return a RunConfig narrowed to model_name's variant overrides and model_cls tiers.
Used inside the worker and HPO so that in a multi-model or multi-variant benchmark under
on_unsupported_covariates="fallback" (the default), a univariate model or a variant with
narrowed covariates never has its target truncated by exog_lags of an unused covariate tier
or receives columns it should not consume. The run identity (run_id and model_hash) is
always computed from the original cfg before calling this helper.
Source code in src/scale_forecasting/features.py
expand_model_variants
¶
Expand include_univariate_baselines and ablation_modes into concrete models and
model_params with capability-aware filtering and semantic deduplication (pure).
Source code in src/scale_forecasting/features.py
764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 | |
extract_static_covariates
¶
Validate and extract series-constant static covariates for one ts_id (pure).
Static covariates (features.static_covariates, e.g. region, category, archetype)
carry series-level metadata for global/hybrid models and hierarchical grouping. Because they are
constant within a single series (and may be categorical strings), they are deliberately omitted
from the single-series local design matrix X — where a constant column is collinear with the
model intercept — and surfaced here as a clean {col: value} mapping.
Source code in src/scale_forecasting/features.py
fit_transform_lambda
¶
Fit the transform's stateful parameter on y (the full series' target).
Only boxcox has one: its λ, fit by maximum likelihood (scipy.stats.boxcox_normmax)
on the raw target. Returns None for the stateless transforms (none/log1p), whose
inverse needs no fitted state. Box-Cox is defined only for strictly positive data, so a
non-positive value raises ConfigError naming the constraint (symmetric with log1p's
y >= -1 rule — we validate, never silently shift). Deterministic: same series → same λ.
Source code in src/scale_forecasting/features.py
holiday_frame
¶
Canonical holiday calendar for the run.
features.holidays lists ISO country codes (e.g. ["US"]). Returns a frame with
ds (datetime64[ns]) and holiday (name) columns, empty when none configured.
A generous fixed year window keeps the calendar deterministic; callers filter by ds.
Source code in src/scale_forecasting/features.py
invert_transform
¶
Invert a transform on forecasts/bounds so the frame is in original units.
lam is the same fitted Box-Cox λ used in the forward direction (ignored by the stateless
transforms), so invert(apply(y)) == y.
Source code in src/scale_forecasting/features.py
level_shift_step
¶
Detect one abrupt level shift in y and return it as a 0/1 step dummy.
The shipped example data contains exactly this pattern — data_gen.generator gives every
archetype a level_shift_prob and applies a single additive jump partway through the
history — and it is the common real shape too (a re-baselined product, a store
reopening, a units→cases change). A model that cannot see it fits the average of two
regimes and is biased for the whole horizon.
The encoding is a step, not a spike: 0 before the changepoint, 1 from it onward, and
(in build_future_features) 1 across the entire horizon, because a level shift persists
— that is what distinguishes it from an outlier. A regression model then learns the jump
as one coefficient and carries it forward.
Detection is a single-changepoint binary segmentation, computed in one vectorized pass
(O(n) via cumulative sums) because this runs once per series across millions of them: for
every admissible split, standardize the gap between segment means by a robust noise
estimate and take the argmax. Returns all zeros when nothing clears _LEVEL_SHIFT_SIGMA
or the series is too short to split — never a spurious constant column.
Source code in src/scale_forecasting/features.py
local_seasonal_baseline
¶
Fast, fold-safe local trend + seasonal baseline for outlier and stockout imputation.
Source code in src/scale_forecasting/features.py
regime_compatible_models
¶
Filter models to those suited for demand_class, never returning an empty list.
Source code in src/scale_forecasting/features.py
regularize_series_grid
¶
Regularize missing calendar timestamps (grid_policy) and trim pre-launch leading zeros
(trim_leading_zeros) for one series (pure).
Source code in src/scale_forecasting/features.py
resolve_model_variant
¶
Resolve a <base_model> or <base_model>@<tag> entry into its ModelVariantSpec.
Source code in src/scale_forecasting/features.py
451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 | |
select_models_for_series
¶
Return the active model list for series under cfg.features.regime_routing.
Source code in src/scale_forecasting/features.py
should_stamp_variant_diagnostics
¶
Whether run_cell / run_panel should stamp taxonomy diagnostics onto CellResult.
Source code in src/scale_forecasting/features.py
variant_diagnostics_for
¶
Return the canonical scope/covariate taxonomy and spec-hash metadata for model_name.