Dynasty, in Theory: Everything Is Selection Bias

There's a lot of strong dynasty analysis out there, especially when compared to five or ten years ago. But most of it is so dang practical-- Player X is undervalued, Player Y's workload is troubling, the market at this position is irrational, and take this specific action to win your league. Dynasty, in Theory is meant as a corrective, offering insights and takeaways into the strategic and structural nature of the game that might not lead to an immediate benefit but which should help us become better players over time.

A Very Useful Razor

My favorite thing about this column is that it affords the opportunity to write about things that ostensibly have nothing to do with fantasy football. Most of my favorite pieces are just ruminations on whether it's better to be a specialist or a generalist; why measuring things is so dangerous; how we can choose how we view things not based on what is true, but on what is useful; when it's acceptable and when it's unacceptable to be correct; why comparisons make no sense (but are worth making anyway); and how we fool ourselves into believing what we see is all there is.

I often find that the best way for me to understand fantasy football is to first understand everything. And as far as "understand everything" goes, there are few tools as useful as a razor. No, not that kind, but a philosophical razor, a tool to efficiently narrow down possible explanations or avoid unnecessary actions.

Occam's razor is, of course, the best-known ("The simplest explanation is usually the best one"), but there are plenty of other terrific examples-- Popper's falsifiability principle ("if it's not falsifiable, it's not scientific"), the Sagan standard ("extraordinary claims require extraordinary evidence").

Sherlock Holmes has made his mark ("once you eliminate the impossible, whatever remains, no matter how improbable, must be the truth"), as has Albert Einstein ("make things as simple as possible, but no simpler"). And I've always been partial to Hanlon's razor: "Never attribute to malice that which is adequately explained by stupidity".

But my favorite is a bit more personal, my most-used problem-solving tool and my own humble contribution to the genre:

Pro tip: any time you see any cool, compelling, interesting, powerful, intriguing, or absurd statistic, you should probably just assume it's selection bias until proven otherwise.

— Adam Harstad (@AdamHarstad) October 2, 2018

What is Selection Bias, and Why Does it Ruin Everything?

Humans are pattern-finding machines. We see trends in small samples and we extrapolate them out to larger samples. It's something of a superpower-- all of science and technology grew out of our ability to discern patterns in the world around us.

It can also be our downfall. Selection bias is when there are issues with our smaller sample that render it unrepresentative and unfit for extrapolation.

It can take a myriad of forms, but here's an example of the classic error in football: from 2016 to 2022, Ezekiel Elliott had 41 games where he carried the ball 20 or more times and the Cowboys were 31-10 in those games, a 76% win rate. In all other games, Dallas was 39-34, a 53% win rate.

If we were simply to extrapolate from Dallas' 31-10 record, we might say that they should have given more carries to Ezekiel Elliott. But there's a problem. When Dallas trailed by 14 or more points during that span, they called run plays on just 27.8% of their snaps. When they led by 14 or more points, they called runs on 60.9% of their snaps.

Dallas didn't win those games because Elliott got 20 carries; Elliott got 20 carries because Dallas won those games. In this example, having a lead is a confounder, or some third factor that influences both of the variables we're looking at. Big leads lead to wins, and big leads also lead to carries.

This form of selection bias is everywhere; virtually every time someone notices a correlation, a confounder is likely to blame. For instance, an increase in per-capita ice cream consumption correlates with an increase in per-capita instances of drowning. Does ice cream cause people to drown? Of course not. When the weather is hot, more people buy ice cream and more people go swimming; the more people are swimming, the more drownings there will be.

Let's Get Weird

Hidden confounders (sometimes referred to as "lurking variables") are the most basic and well-known forms of selection bias, but there are much wilder and woolier examples out there.

For instance, did you know that married couples with children consistently report being less satisfied with their marriages than their childless peers? More than that, for couples without children, marital satisfaction remained steady over time, while for couples with children, the longer they were married the less satisfied they were, on average.

Reader, would you guess that this is actually because of selection bias? (Of course you would; my audience is known to be exceptionally sharp. This is, of course, itself a product of selection bias.)

The reason there are so few married couples without kids who hate each other is not because couples without kids never fall out of love, it's because... couples who hate each other and don't have kids all get divorced. (The phrase you might have heard is not, after all, "staying together for the golden retrievers".)

(Is this another example of a confounder? Nope. A confounder would be if some mysterious third factor influenced both having kids and being miserable-- feel free to insert a joke about tequila here. In this case, the bias goes in the other direction. Having kids and being miserable both influence some unstated third factor-- in this case, divorce rates. This form of selection bias is known as conditioning on a collider. Statistics is hard!)

If you study students at a college, you will find a negative correlation between their SAT score and their high school GPAs. Is this proof that smart kids are lazier? Of course not; this is a form of selection bias known as Berkson's Paradox that happens when you restrict the range of the sample. Students with terrible grades and terrible test scores didn't get into this college. Students with phenomenal grades and phenomenal test scores got into a better college. The end result is a mix of students with good grades and bad scores and students with good scores and bad grades.