Dynasty, in Theory: Who Do You Trust?

There's a lot of strong dynasty analysis out there, especially when compared to five or ten years ago. But most of it is so dang practical-- Player X is undervalued, Player Y's workload is troubling, the market at this position is irrational, and take this specific action to win your league. Dynasty, in Theory is meant as a corrective, offering insights and takeaways into the strategic and structural nature of the game that might not lead to an immediate benefit but which should help us become better players over time.

On Finding Answers

One of the worst-kept secrets to success is surrounding yourself with people who are better than you are. I don't know everything-- I'll pause here for a moment for you to collect yourselves at this revelation-- but if I don't know something, I usually know someone else who does, and that's nearly as good.

So it was earlier this week when a question occurred to me that I wasn't equipped to answer. (For those curious, it concerned how many fewer points an offense would score if it started every drive for the entire season at the 27-yard line vs. the 30-yard line.) I know how one goes about answering this, but I have neither the database nor the technical skill to do it myself and several of the public-facing models were in disagreement.

Fortunately, I'm on fairly friendly terms with someone whose work in this area I greatly respect, so I just asked them. They quickly responded, "I don't love our model (or most of the public ones), but..." And I just as quickly quipped that if you show me someone who loves their model, I'll show you a model I hate.

That got me thinking about trust.

Fantasy Football is a Trust Industry.

You know you can trust me because I tell you I'm trustworthy-- and a trustworthy person would never lie to you, so you know it must be true. But let's imagine for a moment that an untrustworthy person ever figured out this secret. They could just tell you they were trustworthy, and then you'd have to trust them. 

This is a very serious exploit, and I hope the genuinely untrustworthy never discover it.

(In reality, I think it's far less common to encounter someone nefarious seeking to gain an advantage at your expense than it is to come across someone who honestly believes they are trustworthy and are simply mistaken about that. As I often say in another column, the pool of people with a genuine edge is necessarily smaller than the pool of people who believe they have a genuine edge.)

Since saying "just trust me" is a flawed system for verifying trustworthiness, it falls on us to create something better.

Why Does Trust Matter, Anyway?

Football is an entertainment product. Fantasy football is an entertainment product built on the back of an entertainment product. And fantasy football advice... well, it's entertainment products all the way down. I strive to be engaging and thought-provoking. I like to think I'm occasionally funny (though I've known enough unfunny people who thought the same that I'm not hanging my hat on it.) Isn't that enough?

It's not. You read fantasy football articles because you care about your fantasy team and are looking to improve it. It's not enough to be engaging, I must also be useful. (As this column regularly shows, "useful" is a fairly flexible target.)

Most importantly, it's not enough for me to believe I can help you become a better fantasy football player-- the pool of people with a genuine edge is necessarily smaller than the pool of people who believe they have a genuine edge. I need you to believe that I can help you become a better fantasy football player. And that takes trust.

Trust is a hard commodity to come by these days. You can pick any player and  find an equal number of voices advising you to buy him and sell him at any given moment. Is the solution perhaps to trust whoever writes the most persuasively?

I sure hope not, because if so, there's trouble on the horizon.

We Should Probably Talk About LLMs

LLMs, or Large Language Models (such as ChatGPT), are typically marketed as "AI", or Artificial Intelligence. Insofar as words have specific meanings, this isn't true; they generate words without any understanding at all of what those words mean, something of a hallmark requirement for "intelligence".

Instead, they operate by picking up on and reproducing very complex statistical, probabilistic relationships. They are, in a very real sense, the world's most sophisticated and successful cargo cult. They can flawlessly recreate the forms of language without the slightest understanding of the function.

If the goal is to make persuasive arguments, LLMs can replicate the things that make an argument persuasive. This is good; the ability to leverage these undetectable statistical relationships is a powerful and useful tool for writers if they want to, say, write betting advice in the style of Hunter S. Thompson. 

But while passing a piece through ChatGPT to add a layer of polish might make it more convincing, will it make it more trustworthy? Of course not. LLMs are incapable of introspection (although they can provide a very convincing pantomime if given the right prompt). They're incapable of contrition (although one might argue that this hardly differentiates them from fantasy football writers). Whatever it is that makes a person trustworthy, Large Language Models lack it. (Unless the thing is "a keen awareness of the statistical relationships between words and concepts".)

If the quality of the writing isn't enough to establish trustworthiness, we must look elsewhere.

Do Good Players Make Good Analysts?

Some have suggested that fantasy writers should publish their own fantasy results so readers can see whose advice is worth following. I don't love this proposal; I think it conflates "good at fantasy football" with "helpful for others seeking to be better at fantasy football", which are two distinct things. (I suspect they are correlated, but not strongly enough to be a useful measure.)

Perhaps ironically, I suspect I'd do well if "showing the receipts" became the norm. I play in two dynasty leagues and have logged 28 team-seasons between them. I have a lifetime all-play winning percentage of 61% (this represents what my record would be if I played every team every week). I haven't had an all-play record below 50% since 2014, haven't had an actual winning percentage below 50% since 2011, and haven't finished below the league average in points scored since 2010.

I've made the playoffs 21 times vs. an expected 15.7, have made the championship game 12 times vs. an expected 5.2, and have won the title 6 times vs. an expected 2.6. Historically, 71% of my teams finish above the 50th percentile in points scored, and 36% finish above the 90th percentile. 

Does knowing this make you trust me more? It probably shouldn't. For starters, how do you know I'm not making it up? (I'm really not. Trust me.)

A more relevant question: what bearing does any of this have on anything I write here? When I model the odds of a team making the playoffs to show when win-now trades are most valuable, is that model less accurate if I miss the playoffs myself? Do more things become selection bias if I win another championship? (I sure hope not; too many things are selection bias already.) Are you really going to take advice about managing lots of teams at once from a guy whose maximum number of dynasty leagues is two?

In my opinion, "I've been successful so therefore what I'm saying is true" is merely a more convoluted way of saying "Just trust me". Perhaps I'm especially good at drafting rookies; if so, it doesn't make my advice on trading worthwhile. If veteran running backs have been undervalued in my leagues, it certainly doesn't mean they will be in yours. I propose a rule of thumb: it's my job to care about your teams; it's not your job to care about mine.

So How Do We Know Who To Trust?

We can't trust someone just because of their record or their writing. And we can't trust them just because they tell us to. Where does that leave us? I'm not sure, but here are a few common traits among the people who I trust when I have a question I need answered.

Accountability

"But Adam, didn't you just say record doesn't matter?" Thanks for the question, imaginary interlocutor, but no. I said it's not good to judge the usefulness of one thing based on the performance of something else entirely. I'm
actually a huge fan of accountability! It's tremendously useful for building trust.

When I write about streaming kickers off of the waiver wire every week, I don't assume you care how frequently my teams make the playoffs... but I imagine you do care how many points my top recommended waiver wire kickers average, which is why I track and publish that every week. (Historically, it's 7.5 points per game, but we've been a bit lucky so far this year and are currently sitting at 8.5, which is higher than 9 out of the top 12 kickers by preseason ADP.)

When I write about regression to the mean and say that "the worst performers in this metric will probably outproduce the best performers in this metric going forward", I figure you are entirely uninterested in my teams' current record. On the other hand, I reckon you're likely very curious about how often those predictions come true. (42 times in 55 tries so far, a 76% success rate.)

I find myself drawn to people who hold themselves accountable, and every week I place a bet that if I hold myself accountable, others will be drawn to me, too. It's just important for this accountability to be narrowly defined in a way that's relevant to the question at issue.

Also, accountability is no panacea (much as we might wish it to be). Some arguments (such as this one) are beyond easy categorization and analysis. There must be other markers we can look at than just a relevant track record of success.

Contrition. Sometimes.

I hate when people get so invested in a take that they can't admit it was wrong. It's one of the quickest ways to lose my trust. But I also hate it when people are so disinvested in their takes that they're admitting they were wrong the second the winds shift.