When browsing through Twitter (X?) on Wednesday afternoon, I saw a game score card from the preseason matchup in Buffalo on Tuesday night.
HockeyStatCards is a Patreon site that is run by Cole Palmer which provides visuals on game score - a stat developed by my friend Dom Luszczyszyn. I typically ignore these visuals when they come across my feed, just like I ignored this one, but I realized at some point Wednesday night that I had quite a few thoughts on single-game player evaluation. With that, I invite you down this rabbit hole with me.
What is the purpose of evaluating one game?
I ask myself the same question. I think this stems from “the eye test” in which a game is the largest agreeable unit in which you can evaluate a player with a strong recall of their events. Scouts will make a trip into a city to watch a player play one game, create their report, and move on to the next. Even if coaches, who are with their team the whole season, tried to expand their evaluation period over a month, they would still have to look back on their game notes as their ability to recall a game 23 days ago is going to be weak.
Ultimately, evaluating players on a game-to-game basis for coaching and scouting just makes sense. However, those in the analytics sphere struggle in this area.
The strength of game score is that it is rather easy to understand. Dom has updated the stat since originally introducing it to the hockey world in July of 2016, but in summary, “Game Score is a linear weight model with the weights for each stat within it being derived according to the frequency of goals occurring from them.” More simply put, if there are 10 shots on goal for every goal, then shots on goal must be divided by 10 in order to scale them with goals. One shot on goal is worth 0.1 goals.
What game score excels at is taking (mostly) box score stats and brings them down to one number. But I think this veers away from what coaches, scouts, management, and analysts are really trying to do in player evaluation which is parse out who is contributing most to their on-ice results. To do that in a one-game unit using only NHL play-by-play data is nearly impossible.
Hungary 2022
I grew up a racing fan. More specifically, a NASCAR fan. I had a diecast car for pretty much every racer, and I spent my weekends as a young child on the floor with my cars watching the race. As I got older, my fandom of motorsport grew to other series, including Formula 1. One of my favorite parts of F1 is the sheer amount of data that they collect, analyze, and implement into their decision-making process. They are nerds who hire drivers to get the most out of their science projects. It also shows that just because every team has an analytics staff does not mean they are all equal.
Of course, I am a fan of one of the worst decision-making teams in Scuderia Ferrari. The historic team is the only one that gets an extra lump sum amount on top of their prize money, and they still can’t get strategy right. The nimbler, modern team that is dominating the sport right now, gets things right.
Perhaps there was no better example of this than in Hungary last season. Ferrari was up front after a good qualifying session while Red Bull was further back in the pack. But with an unusually cool, damp day, the teams would be forced into a more unorthodox strategy for this particular race.
As Red Bull Principal Strategy Engineer Hannah Schmitz explained,
Throughout the weekend, it’s very collaborative…it’s not always just about the numbers and the data. The drivers also have a lot of feeling, as well, in the car, so we try to take that on board. Take on board what the race engineers are looking at maybe different data to the strategists…at the end of the day it is a strategy call, but we try to just work as a team and use as much information as we can.
What Red Bull understands, and where Scuderia Ferrari seems to falter, is that evidence-based decision-making combines information from many different sources with different strengths and weaknesses in order to make the best decision possible. There is no sport that I am aware of in which you should operate purely algorithmically. There’s always some human touch.
Evidence-Based Decision-Making and Time Horizons
If you Google “Evidence-based decision-making framework,” you will find many different results. One of the illustrations I like are the Venn diagrams, combining all of the available sources of information. In hockey, the four sources of information, as I see it, are players, coaches, scouts, and analysts.
The type of decision that has to be made will dictate which collaboration (if any) is needed and which party has the greatest strength. For example, how the penalty killing team defends the entry may dictate a small change in how the powerplay approaches the next entry. As the game goes on, say at the first intermission, the coaching staff will have the opportunity to pitch in what they are seeing and provide adjustments. As we get to 10 games, the analytics staff may have the ability to offer insight that will help the coaching staff with their adjustments.
A scout may go to an OHL game in October and find a potential diamond-in-the-rough and latch on to them. Come January, the analytics staff may be able to offer some insight into players that haven’t caught the eye of the scouts yet so that they can track them in the back half of the season.
Players are able to gather information on a microscopic scale incredibly quickly but lack the ability to see the whole picture. Coaches can gather information quite quickly as well and may be able to see the bigger picture better than the players but will be prone to biases as the time horizon expands. Analytics may take a long time to get running, but it gets more and more useful as time goes on. Isolated, each have their downfalls. In concert, they can become a powerful machine.
This rabbit hole started with a simple game score chart and my itchiness surrounding single-game statistics, but it eventually moved into the conclusion. There are times when the information gathered through the eyes and intuition is superior, and there are times when data-driven insights are superior. Therefore, having people with experience and expertise in each, as well as the ability to know how and when to combine the two, is crucial. Otherwise, you end up like Scuderia Ferrari.