Introduction
First off, let it be made clear that I am a die-hard Oilers fan. I moved to Edmonton the same year Gretzky left, and lived there for 20 years before moving to Ontario. Edmonton is still my sentimental home town, and the Oilers will always be my favourite hockey team. Nonetheless, the 2015/16 year has been such a disappointment that I just had to better understand the fundamentals of Oiler failure, as far as they can be understood. And so I spent a Sunday afternoon analyzing some data, calculating a few things, and writing this piece.
The question I ask is: are the Edmonton Oilers unlucky, or just bad?
This seems like an easy question to answer, but it isn’t straightforward when we consider the vagaries of sport in the real world. A mathematically average team can have a run of bad luck (a concept I will define in a moment) and lose more than they win, and have a run of good luck, winning more than they lose. The challenge is separating variation due to luck from systematic difference due to true differences in the quality of teams. If the Oilers are truly terrible, then we should expect them to have a record that is highly unlikely to have occurred by chance alone.
My approach to answering this question is to use a simple Monte Carlo simulation where I generate a theoretical distribution of team point totals. The simulation is calibrated using the average win, loss, overtime win and overtime loss record of teams in the 2015-2016 NHL season. This gives me a distribution that I can then use to calculate the chances of the Oilers being as bad as they are due entirely to bad luck.
What do I mean by luck?
My working hypothesis is that the Oilers are the victims of bad luck. To make sense of this statement I need to first define luck. This can be most easily understood by starting with an example. If I flip a coin 10 times, I have an expectation that heads will come up 5 times, but in practice, I know that over a small number of coin flips the numbers may not break down this way. I could get 7 heads, or 8 heads or even 9 heads and not think that the coin is systematically unfair—these outcomes have a roughly 12%, 4% and 1% chance of happening, respectively. But if I flipped it 20 times and it came up heads 20 times, I might think the coin is unfair, since the probability of 20 heads in 20 coin flips is around 1 in a million.
We typically interpret the expected variation (e.g., 7 heads in 10 flips) as variation due to chance, or luck, and the unexpected variation (e.g., 20 heads in 20 flips) as variation due to something systematic—such as weighted coins or magnets. Indeed, much of statistical inference boils down to distinguishing the one from the other. If the outcome of hockey games were analogous to flips of a fair coin—where the only thing determining a win or loss was some process equivalent to a coin flip—we would also have an expectation of how many points teams would get in a season. In fact, here is what it would look like:
Figure 1. A theoretical distribution of team points in an NHL season
This is the distribution of points we would expect in an 82 game season assuming that each team has a probability of wins, losses, over time losses and over time wins equal to the average in a recent 82 game season. What is important to note is that like a coin flip, we shouldn’t expect that each team would end up with the same number of points even if there was true parity between teams—where each team would have the same probability of wins, losses and ties. In a 30 team league, we could expect some teams to have a ‘lucky’ season and get 110 points and other teams to have an ‘unlucky’ season and get 75 points. On the other hand, fewer than 60 points or more than 120 points seems highly improbable if point totals were assigned through a random process in which all teams have an equal chance of winning games. Such extreme totals are analogous to flipping 18 heads in 20 coin flips, and strongly suggest that something other than chance is at work.
How much does luck determine seasonal point totals in the league as a whole?
Before getting to the Oilers specifically, I want to briefly consider the role of luck in the NHL as a whole. If luck was the only determinant of point totals in a season, the league would have perfect parity, so let’s call this the parity theory. Figure 1 is an illustration of the distribution of expected points for a team (over the course of a season) were parity theory true. I can overlay the parity theory histogram with the histogram of real data from several seasons to see how the distribution of real points differs from the distribution defined by luck alone:
Figure 2. Comparison of the real and theoretical distributions of team points
The red bars are from a season where points were determined by luck alone, and the blue bars are from actual data on the distribution of points. The purple bars are the overlap of the two distributions. We can see here that the real distribution of points is flatter than what we would expect by chance. Specifically, in the real world some teams do a little better and some do a little worse than we would expect if luck was entirely responsible for wins and losses. Furthermore, we see that a smaller proportion of teams produce the average number of points we would see if the points were distributed by chance alone; the red area of the graph is the area where parity theory over-predicts the real world. So in short, there is not perfect parity in the NHL, and team performance is probably different from what we would expect if the differences in performance were entirely due to chance.
It’s worth noting that over time the real distribution of points could converge or diverge from the distribution based on parity. Some rule changes—like the salary cap—may increase league parity, so over time we could see a tighter range of points.
How unlucky would the Oilers have to be?
Using real data on the points acquired between the 2006-2007 and 2015-2016 year (lock-out year excluded), I am able to estimate the probability of the Oilers getting their actual point total over this period by chance alone. This will tell me how unlucky the Oilers would have to be in order to perform as poorly as they do assuming complete parity in the NHL. Over this period, the Oilers got 642 points, averaging roughly 71 points per season. To determine how much the Oiler’s performance differs from chance we can compare the Oiler’s actual seasonal point totals to the distribution of points according to parity theory.
If we assumed parity in the league we should expect a team to have a roughly 1% chance of getting 71 or fewer points in a single season by chance alone. However, the Oilers have under-performed for nearly a decade, so more telling is how the 642 points looks over the 9 most recent (complete) seasons. Here is a theoretical distribution of team point totals over 9 seasons assuming league parity:
Figure 3. Theoretical distribution of total points over 9 seasons assuming perfect parity
As you can see, 642 is quite the anomaly; the x-axis is not scaled to include values less than 700 but it doesn’t need to be in order to illustrate how unlikely this point total could have occurred by chance alone. Based on my estimation, the probability of getting 642 or fewer points in 9 seasons by chance alone is considerably less than 1 in a billion. Yikes.
Conclusion: the Oilers have been a bad hockey team.
It’s not often I get to draw definitive conclusions about anything, but I think this analysis is pretty conclusive (and utterly unsurprising): the Oilers have really sucked. Maybe (hopefully!) they won’t suck next year, but the cold hard reality is that the Oilers have been an astronomical disaster. However, it is worth noting that the past is not a perfect predictor of the future. Unless there is some immutable structural force behind the Oilers failures, then the future may yet be bright. Especially with another choice draft pick in 2016!