Every change to the served model, dated and in order, newest first. Each entry says what changed, why it was tried, and whether it cleared the pre-registered gate it was measured against.
How to read this
We test far more than we ship. Most tested changes here are marked not adopted: the change was measured against a bar set before the test ran, and it did not clear it, so nothing changed. That is the point of a changelog like this: the record includes what we tried and rejected, not just what shipped. See how the model works for the architecture, and model health for the live track record.
Model changeAdopted
Scoring probability, level-term candidate (a)
Anytime-goal model adds a season-derived level term
The anytime-goal ("scoring probability") model shipped four changes together: an ice-time term in its per-game scoring rate, removal of a positional rate that had been double-counted, a corrected goalie save-percentage denominator, and a level adjustment the model now re-derives from the current season's own games played so far instead of a fixed constant carried from history.
Why: Held out, the model had been running low relative to how often players actually score, and unevenly so: worse for low-minute players and for defencemen specifically, which meant the same number meant different things depending on who it was attached to. The fix targets both the overall level and that structure.
Still runs about 3% low against the observed rate. The honest claim is discrimination (AUC 0.6924 vs 0.5000 for a base rate), not calibration.
Gate: Pre-registered: hold the model's level within ±3% of the observed rate on a held-out season, with every ice-time bucket and every position separately inside a tolerance band, and the ranked board's ordering preserved above a correlation floor. The shipped candidate reached -3.15% (missing the ±3% band by 0.15 percentage points), every ice-time bucket within ±5.3%, every position within ±4.2%, cleared the ordering criteria, and held level across three different seasons (+0.71% / -2.17% / -3.15%), the first candidate measured to do that. Shipped by owner decision despite missing the level band, with the model's own performance across the 2026-27 season named as the real held-out test, to be re-checked in December.
Goalie projections scale to the season's actual game count; two calculation defects fixed
The goalie season-projection model now scales every workload assumption (the starts cap, role priors, and total projected workload) to the number of games the league has actually scheduled for the season, rather than assuming a fixed 82-game season every time. In the same pass, two calculation defects were fixed: the projected win rate had been computed as points divided by twice games played, which is a win rate plus half the overtime-loss rate rather than a true win rate, and the league save-percentage anchor was pulled from a backward-looking average of a league save percentage that has been falling year over year.
Why: A model that always assumes a full season is silently wrong the moment the schedule is not a full season, and the win-rate and save-percentage defects were both biasing every goalie's projection in the same direction, compounding each other.
Gate: The schedule-awareness change cannot be scored against history, because every backtested season already had a full 82-game schedule, so it is validated by construction and a dedicated regression test rather than a held-out gate. The two calculation-defect fixes were backtested separately and reduced projection error on every measure checked, so both shipped alongside it.
Not adopted: regressing preseason team strength toward an underlying-play rating
Tested replacing the preseason team-strength rating's flat regression toward the league average with a version that regresses each team toward a rating derived from its own underlying shot-quality play the previous season, across eighteen candidate variants.
Why: A team's record can diverge sharply from its underlying play (a team that controlled the puck all year but got unlucky in goal, for instance), and a purely record-based rating with a flat regression has no channel to see that. This tested whether the model's own shot-quality data could recover some of that signal without any market or roster input.
Gate: Pre-registered: a season-level projection-accuracy gain against a fixed bar, checked across multiple held-out seasons, without materially hurting calibration. None of the eighteen variants cleared the bar (the best reached a fraction of it), though the underlying signal was real and directionally correct, and standings-ranking accuracy improved modestly in every variant tried.
Not adopted: blending sportsbook pricing into preseason team ratings
Tested blending the sportsbooks' own opening-week pricing into the preseason team-strength ratings that seed the season forecast (projected points, playoff odds, division odds), instead of relying solely on the model's own record-based rating.
Why: The market prices information an outcomes-only rating structurally cannot see, such as an offseason trade or a key injury return, and this tested whether folding some of that in would sharpen the preseason forecast specifically.
Gate: Pre-registered: an improvement in the season forecast's own projected-points accuracy against a fixed bar. The market signal was real and statistically distinguishable from zero, but fell well short of the bar the forecast is judged on, so the preseason ratings were left on the model's own rating alone.
Served win-probability model switches to en95+trend
The engine serving every win-probability prediction on the site switched to en95+trend: an elasticnet logistic regression over a reduced feature matrix with every team-identity feature removed, plus three added features tracking each team's recent trend in underlying shot-quality form.
Why: A multi-day comparison of candidate engines found this architecture beat the previous production engine on accuracy and calibration together, while dropping team-identity features that let a model lean on "this team is just good" instead of reading the game in front of it.
Gate: Chosen for beating the incumbent engine on accuracy and calibration together in walk-forward testing, not against a single pre-registered pass/fail bar. Reproduces the certified figure every page on the site shows: 60.70% accuracy across 7,428 games, Brier 0.2336.
