[article] Yost: A new way to measure goaltending performance

[poetica]

Yost: A new way to measure goaltending performance

If there’s one area in hockey analytics that’s lagging behind, it’s primarily in the realm of goaltender studies. Outside of multi-year even-strength save percentage – which, of course, requires a goaltender to face thousands of shots over a series of seasons – there simply isn’t a ton to measure a goaltender’s true talent level.

While save percentage is a decent metric over longer samples, it’s subject to the same, wild fluctuations that shooting percentage experiences in smaller ones. For one brief example, consider quickly the below table of six goaltenders around the league:

Anyone willing to take the first three goaltenders over the next three goaltenders going forward?

One of the things that clouds smaller sample save percentages, aside from random variance, is fluctuation in a team’s ability to prevent shooters from generating shot-attempts in scoring chance areas. This is particularly important, because we know shot distance correlates well with shooting percentage. Shot distance (as a proxy for “shot quality”) may pale in importance compared to territorial control in today’s National Hockey League, but it still holds a sliver of importance. And, since team effects aren’t equal across all thirty teams, it is data worth considering.

Over at War on Ice, Andrew Thomas has started to compile save percentages for goaltenders adjusted based on the quality of the shots they have seen. His adjustments weight the likelihood of every shot against becoming a goal against based on the distance of said shot. Thus, his adjusted save percentage both captures the goaltender’s ability to stop shots, and the possibility that certain goaltenders simply face shots of varying difficulty due to team effects. The adjustments are fairly small, but in some cases, important.

For example, if a goaltender stops 92% of shots he’s faced, is he playing well? Generally, the answer is yes. But what if he went through a three-week stretch where opposing teams haven’t been able to muster much in the scoring chance department? Is that 92% number still good? Is it possible that 92% is underwhelming, failing to meet expectations?

By grabbing the unadjusted and adjusted EVSV% at War on Ice, we can quickly graph out what goaltenders are seeing their save percentages inflated a bit by relatively easy shot distance faced, and what goaltenders are seeing their save percentages deflated by relatively difficult shot distance faced.

What I’ve done is charted the difference between the ‘Unadjusted EVSV%’ and ‘Adjusted EVSV% (here referred to as ‘Delta Quality’). Negative goaltenders see their save percentages drop after accounting for shot distance faced. Positive goaltenders see their save percentages rise after accounting for shot distance faced.

Click here for a graphical representation of the above chart.

At the poles, we can see a few notable goalies who see a fairly significant reduction or bump based on the shot distance of opposing shooters. The two most interesting goalies at the bottom, to me, are Craig Anderson and Tuukka Rask. Craig Anderson (.963 EVSV%, .954 aEVSV%) has been absolutely stellar for Ottawa, but maybe a small reason why the Senators have been piling up wins despite poor possession numbers is that their starting goaltender has seen slightly easier than expected shots against. Tuukka Rask (.914 EVSV%, .904 aEVSV%) is notable for another reason entirely – he’s struggled for Boston in the early going, but it’s not because he’s facing particularly difficult shots. Rask’s been one of the best goaltenders in the league for a long time and this is almost certainly a blip on the radar but, based on this, it’s possible he’s been even a bit worse than initially thought.

On the other side, we see the polar opposite of Tuukka Rask in Henrik Lundqvist. Lundqvist (.916 EVSV%, .926 aEVSV%) has also seemed to struggle in the early parts of the season, but he sees a nice little bump in his adjusted percentages because the shots he’s faced have come from in-tight. Cam Ward (.889 EVSV%, .900 aEVSV%) has seen his percentages sliding for years now, and I think skepticism about his game is warranted at this point. But, here’s one data point in his favor – Carolina wasn’t particularly kind as a team to Ward, and he ends up with the most favorable adjustment in save percentage of any goalie in the league. Still, it should be noted that Ward’s adjusted save percentage doesn’t speak highly of him as a goalie.

Adjusted save percentage is far from the singular data-point every hockey executive wants, but it is another tool in the tool belt – one that will help us just a bit in contextualizing

[Kesler+Horvat]

I saw this and it is way to confusing lol.

[Nuxfanabroad]

Does anyone here use analytics in determining a mate? What would THAT look like?

[poetica]

I saw this and it is way to confusing lol.

Really? Why?

Basically, it just looks at how a goalie is performing at even strength and then adjusts according to the quality of shots they are facing. So, goalies facing a lot of low quality shots (easier saves) would see their adjusted SV% drop while goalies facing a lot of high quality shots (harder saves) would see their adjusted SV% increase. It's meant to provide a more accurate picture of how goalies are performing based on not just the number of shots they're facing but the quality as well.

Hope that helps.

[avelanch]

it's not that confusing, just a little ridiculous... i guess advanced stats guys are gonna advanced stats, but this isn't really that helpful.

[Master Mind]

Shot distance doesn't always determine the difficulty of the shot. What about a shot from the blue line, where the goalie is being screened?

I don't think there's any stats that can give a clear picture of who the top goalies are. You just have to watch them play.

[avelanch]

he'd have to adjust for % of the puck the goalie can see when the shot is taken and on the way in, and if the shot was tipped, how far out, and how many times to truely determine quality of shot... not just distance. in fact within a foot the quality of shot is almost zero, as it's usually just hack and slash into the goalie's pads at that point. yet this stat would say it's the highest quality shot there is.

[bigbadcanucks]

After watching Miller play so far this season and watching Luongo/Schneider over the past years, the thing that I notice is that with Miller, the number of second and third shot chances are far less. This to me means that a goalie giving up fewer juicy rebounds gives you a better chance of defending and preventing goals (pretty obvious, right?).

So, to all the advanced stats guys out there who are way smarter than I am, what do you think about the following as metrics for goaltending:

(1) SV% first shot -- higher the better

(2) Frequency of second chance shots -- lower the better (e.g., goalie faces 30 shots total, with 6 shots coming from rebounds...frequency is 6/30 or 0.200)

(3) SV% second chance shots -- higher the better (measurement of how many goals goalie gives up on rebound/second chance shots...so goalie gives up 6 rebounds, and of those six shots, opposing team scores 2 goals...SV% is 4/6 or 0.667).

(4) Overall SV% -- higher the better

Anyone think that would give a clearer picture of goalie performance?

[avelanch]

a rebound ratio would be nice to see, as well as a rebound placement stat (a rebound to the slot is way worse than a rebound to the corner/side boards.)

[TheXFactor]

"Relatively easy shot" and "Relatively difficult shot" is relative. A SOG is also relative to the player shooting the puck because, generally speaking, 5 SOG by a player like P. Kane is way more dangerous than 5 SOG by John Scott.

Number of SOG doesn't always mean a goalie had a more difficult game.

[Nuxfanabroad]

Incidentally, goalies are too damned large nowadays. Bigger nets, pretty-please.

[BanTSN]

Funny that PK sv% is being ignored, since that's when you need big saves from your goaltender the most.

[Fateless]

There are just way too many factors is determining which goalies are better for pure numbers, because you can't put some factors into numbers.

Some goaltenders rise to the occasion while others crumble. For example, we've seen many times where Miller lets in a few early goals, yet then rises to the occasion to give his team a shot and stands on his head. His overall numbers may not be great due to early goals, but he stopped shots when it really counted. This is hard to put into numbers.

There's is also the obvious factor that not every shot is equal. Someone above mentioned it, but a shot from Stamkos 20 feet out is going to be MUCH harder to stop than a Sestito shot from 20 feet out. So someone who stops 20 shots from Stamkos in a game is obviously more valuable than the guy who stopped 20 shots from Sestito, yet there are no numbers to determine that factor.

I agree with one of the first posts where you simply have to watch a goaltender to determine their value. Some goalies you can see play and just say "wow", even when they let in multiple goals while other goalies who post a shut-out played mediocre.

[gocanucks1916]

Pretty interesting. One thing that stands out to me here is just how freaking good Roberto Luongo is. Will never understand the hate he got here.

[BanTSN]

Pretty interesting. One thing that stands out to me here is just how freaking good Roberto Luongo is. Will never understand the hate he got here.

Um, how about those blowout losses in the playoffs?

I like him too, but it's pretty easy to understand why all the hate.

[poetica]

Visibility is certainly a huge factor, but how could it be quantified? It's not always possible even with the net cam to know if a goalie could see through a screen or traffic.

Rebounds, though, would be a great addition to advanced goalie stats and certainly seems possible, as discussed in an article at The Goalie Guild about advanced goalie stats. There's even an article from 2011 looking at rebounds for some NHL goalies. It's an interesting read (if obviously outdated), but mostly it just illustrates the problem with looking at rebound goals only, namely that it doesn't take into account team play. Teams that excel at clearing out rebounds or keeping people from hanging out in front of the net unchecked waiting for a rebound would lower a goalie's rebound GAA regardless of how good he actually was at controlling rebounds or making those saves. And then there's the article that questions whether or not goalies really do have a whole lot of rebound control at all.

War On Ice has some of the most detailed goalie stats I've been able to find. They have stats dating from 2008 until the present and include regular SV%, SV% for the 3 shot difficulties (location), Adjusted SV% (taking into account shot difficulty), and average shots per game. You can even check a goalie's detailed stats for an individual game.

Shot distance doesn't always determine the difficulty of the shot. What about a shot from the blue line, where the goalie is being screened?

I don't think there's any stats that can give a clear picture of who the top goalies are. You just have to watch them play.

Well, that's certainly true. Sadly, though, it still leaves a lot of room for personal bias. Look at all of the people who swear that Luo's a crappy goalie. But hey, that's the difference between stats and personal opinion, right?

[Master Mind]

Visibility is certainly a huge factor, but how could it be quantified? It's not always possible even with the net cam to know if a goalie could see through a screen or traffic.

I doubt it could, but it just further shows how impossible it is for stats to show how good a goalie is.

Well, that's certainly true. Sadly, though, it still leaves a lot of room for personal bias. Look at all of the people who swear that Luo's a crappy goalie. But hey, that's the difference between stats and personal opinion, right?

There will always be personal bias no matter what the stats say. Lu could have made 50 saves in each of his playoff losses, but people wouldn't care. If a goalie doesn't win, most people will say they're terrible anyway.

IMO when people start saying that quality goalies aren't very good (such as Lu, or at times Lundqvist, Price and Miller) then that usually tells me how much they know about the sport. Takes a team to win/lose.

[bigbadcanucks]

a rebound ratio would be nice to see, as well as a rebound placement stat (a rebound to the slot is way worse than a rebound to the corner/side boards.)

That's not really a rebound or second chance opportunity given up by the goalie.

[Ghostsof1915]

There's lies, damned lies, and advanced statistics.

[pianoman13]

I like it a lot. Guys even save percentage isn't a good way of measuring how good a goalie. If a team is giving up more odd man rushes against, the goalie will have a worse save percentage than if the team played a responsible defense.