How PuckCast's Confidence Grades Work (and Why They Matter)

Most prediction models don't tell you how confident they are. They give you a number and let you figure it out. PuckCast does it differently.

Every game gets a confidence grade from A+ down to C based on how sure the model is about its pick. The idea is simple: if you're going to use predictions to make decisions, you should know which ones the model actually feels good about and which ones are basically coin flips.

Here's how the whole system works.

#The grades

The model outputs a win probability for every game. That probability, combined with how stable the inputs are, determines the grade.

 

That's 5,002 games tested through walk-forward validation across four seasons. The model never sees the games it's being tested on during training.

The pattern is clear: higher confidence = higher accuracy. An A+ pick hits almost 4 out of 5 times. A C pick is barely better than guessing.

#What makes an A+ pick

It's not just about a high win probability. The model also needs the inputs to be clean.

A+ picks usually look like this:

• Big probability gap between the two teams (65%+)
• Confirmed starting goalie, healthy, no question marks
• Normal rest for both teams (no back to backs)
• Favored team playing well recently
• No major injury unknowns

When all of that lines up, the model assigns maximum confidence. When even one of those things is off, the grade drops. That's the whole point of the system.

#Where the model gets it wrong

The 79.3% hit rate means 20.7% of A+ picks lose. That matters just as much as the wins, so here's where those misses come from.

Goalie changes after the pick goes out. This is the biggest one. The model grades a game based on a confirmed starter, then that goalie gets scratched a few hours before puck drop. By that point the pick is already published. If you see an A+ pick and the goalie situation changes, treat that pick as void.

Back to back fatigue. Earlier versions of the model didn't weight this heavily enough. The current version is more conservative on B2B games, which is why you rarely see an A+ grade when a team is on the second night of a back to back.

#Live season vs backtest

Important distinction: the 79.3% A+ number comes from backtesting across thousands of historical games. That's useful for validating the model works, but it's not the same as live results.

The live 2025-26 season is the number that actually matters. You can see the full rolling record, updated daily, on the performance page.

#How to use the grades

The grades aren't meant to be followed blindly. Here's the way I'd think about it:

1. Check the goalie situation first. Always. If the listed starter changes on the predictions page, the grade is meaningless.

2. Focus on A and A+ for your shortlist. The model grades every game, but A and A+ are where the edge is clearest. Those are the games worth paying attention to.

3. Don't overreact to a bad week. The model will have cold stretches. That's variance, not a broken system. Zoom out to the full season sample before drawing conclusions.

4. Track it yourself. The performance page shows everything. Use it to see how the model does over time rather than on any individual night.

The model is a tool. How you use it determines how much value you get from it.

---

All accuracy numbers from walk-forward validation across 5,002 games (4 temporal folds, 16 seasons of training data). Live season results updated daily at /performance.