step 5: if he’s a good passer, why doesn’t he pass good?

If you haven’t noticed, big Moneyball guy over here. But that one line really cuts to the core of the issue.

If he’s a good hitter, why doesn’t he hit good?

1. In our case, the problem is not that there are rich teams and there are poor teams…but that we are not evaluating the sport in reality. We care about passer rating because we think it serves as a proxy for passing performance – and to its credit, passer rating is a great proxy for how often you pass well.

2. But we aren’t correctly defining “pass well” and that’s the issue here. We can certainly agree that scoring points is what we care about…seeing how that’s how you win – so what we really want is to define passing in terms of how it helps you score points.

3. What passer rating does in a traditional sense is over-index on good/perfect passes (whatever we decide equals 3 points) and under-indexes on reception errors (which we give a 0 for).

4. In reality, when we look at how different pass qualities help or hinder your ability to win the rally overall, here’s what we see:

Rally Eff refers to (Won – Lost points) / all receptions. If you pass perfectly and ended up winning the rally at some point, +1…otherwise, -1.

5. Numbers are cool, but visuals are better. What we’re really concerned with is the accelerated drop-off on the right. 3pt and 2pt, pretty close together, even 1pt isn’t far away, but that R/ and R= fall off the cliff quickly. In comparison, passer rating (Perception) has a steady and equal drop between ratings.

6. So, the issue with passer ratings is that it assumes each outcome is equidistant from the next. 3-2 = 2-1 = 1-0. Or that 3-1 = 2-0. The problem here is that in coaching terms, that means that a player could pass a 2, 2, 1, 1 or a 3, 3, 0 ,0 and we should value them the same at a 1.5 passer rating. This doesn’t pass the eye test.

6.1 Just want to pause and call extra attention to this. First and foremost, if we use passer ratings (Perception) for 10 serves, both players are passing a 1.5 here:

Player A: 3, 3, 3, 3, 3, 0, 0, 0, 0, 0 = 1.5 average

Player B: 2, 2, 2, 2, 2, 1, 1, 1, 1, 1 = 1.5 average

6.2 But if we use the numbers on the right (Reality) a.k.a. the Expected Value (eV) of the reception, we start using the language of actual points to assess how a reception helped or hurt your ability to win the rally:

Player A: 0.282, 0.282, 0.282, 0.282, 0.282, -1.0, -1.0, -1.0, -1.0, -1.0 = -0.359 average

Player B: 0.162, 0.162, 0.162, 0.162, 0.162, 0.009, 0.009, 0.009, 0.009, 0.009 = 0.086 average

It’s the exact same 10 passes for each player, but with massive differences in outcome when we use a type of expected value rather than pure passer rating.

7. As we can see from chart on the right (the actual, points-based outcomes), when we translate pass qualities into a team’s overall sideout efficiency there’s clear bunching at the higher scores and massive changes in sideout% as you get towards 1 and 0 point passes. This is what we mean when we say “over-index” on good outcomes.

7.5 Here’s what over-indexing on good/perfect passes looks like from a statistical view:

Overall Passer Rating as a function of Perfect and Good pass %

7.6 Using just the percentage of Perfect and the percentage of Good passes, we can account for 92.7% of variance in the overall passer rating. !! That’s nuts. This is why if you use Volleymetrics, the GP% (good pass%) actually makes a ton of sense rather than using anything else. Your % of good/perfect passes will basically give you the passer rating in its entirety.

7.7 Ok… but what if I like what you’re talking about in terms of using actual points as our measurement going forward? – how does “passing well” (as defined by perfect/good passes) stack up with that? Not as well.

Adjusted R2 of 63.6% this time, trying to account for Expected SO%

7.8 Ok, so where’s the gap? If “passing well” doesn’t explain Expected Sideout %…then what does?

Expected SO% as a function of Error %

7.9 Here’s a model using only Error%. 76.6% of variance explained (meaning the Adjusted R-squared value in the bottom right). Not too shabby – and better still than using perfect/good passes. My point here being, we pay too much attention to good things that help us score – and not enough to the things that hurt us / help our opponent.

8. Ok? So now what? Why should I even care? Two-pronged answer.

8.1 First – at the very highest level where getting aced is less common (see Stanford / Penn State in upper left), you honestly might be fine with passer rating as a quick and easy way to evaluate passers. But as you move to the right and hit schools in the blue you might not recognize – or finally down to the RPI killers in orange and seafoam green – errors make up a larger piece of the pie and we should better account for them.

You might notice you can recognize some big names in red. You might also recognize fewer names in blue, even fewer in orange, and not many in our seafoam(?) green

8.2 Second and more important – volleyball struggles to speak a universal language. We have Point Scoring for when you’re serving, Sideout for when you’re receiving – Passer Ratings for reception, Attack Efficiency for hitters, terrible blocking and digging box score metrics, and basically nothing for setters.

9. We need a single, universal metric to talk about volleyball with.

10. Oh! Intro!

Expected Value (eV) and/or Point Win Probability (PWP)

^ open to naming suggestions.

eV — talk about everything in terms of efficiency. Perfect dig is worth 0.450, good block touch is worth 0.300, poor reception is worth -0.150, etc etc…. Since efficiencies are already standard nomenclature within the coaching world, this is certainly the more digestible stat.

PWP — talk about everything in terms of likelihood to win the rally. This is more in-line w/ Win Probability that you may have seen for some pro sports. Since every contact changes the likelihood of winning the rally, this is a possibility to recreate what other sports have. Spoiler alert: it’s really just (efficiency + 1) / 2. For example: (0.400 + 1) / 2 = 70% chance to win rally.

11. Using the same philosophy as Expected Sideout%, we can apply this concept to any situation, any skill, anything we want. We can have a universal metric applicable to the entire game of volleyball – are you helping or hurting our chances to win the point?

12. Reminder. We are trying to predict the future. Using the historical passer ratings of two teams entering a match only explains 6% of variance in future outcomes.

13. Using Expected Value (eV), we can account for 17% of variance (via the McFadden psuedo-R2). Nearly 3x better than Passer Ratings. That’s why we care. We’re moving in the right direction. Is serve receive the main factor in winning and losing? No, but we’ve proven that using actual points rather than arbitrary ratings is a better metric for predicting the future. We are on our way!

Step 6: If he’s a good hitter, why doesn’t he hit good?


Comments are closed.