Most of the people I know were surprised by last week's midterm election results. Friends both liberal and conservative were pretty much convinced that something of a red wave would wash across the country. Instead, the Democrats maintained their majority in the Senate (and, depending on what happens in Georgia, might even enhance their position), and if Republicans take the House, it will be with the slimmest of margins. In my own Congressional District, MAGA firebrand Lauren Boebert is in an exceptionally tight race that has yet to be called despite the fact that experts wrote off Colorado District 3 as a Republican lock.
It's safe to say that not many people saw this result coming and experts from both sides of the political spectrum had a different forecast. For example, here's conservative commentator Henry Olsen's prediction in the Washington Post, and here's a survey of Democratic pre-election lamentations from The Wall Street Journal. Fox News ran a story with the headline "CNN, ABC, NBC Sunday panels issue final midterm predictions: 'very very bleak right now for Democrats'.
Now, I don't really want to talk about politics. Instead, I want to use this example to illustrate a more general trend: we tend to trust expert forecasts. Yet we shouldn't. While there are a few notable exceptions, trusting expert predictions is a mistake.
An expert is someone who has some knowledge or skillset that the average human lacks. When it comes to knowledge, an expert is someone who meets three conditions:
They make falsifiable judgments
They make accurate judgments
They have truth-related incentives
A judgment is falsifiable when it's capable of being false. If I tell you that your house may or may not have termites, I've not made a falsifiable claim. Nothing you find will show that my belief about termites is mistaken. This is part of the explanation for why fortune-tellers and astrologers aren't experts. "Something momentous will happen to you this year!" That's just not falsifiable.
Second, a judgment is accurate when it tracks reality. If I tell you that your house has termites, and it doesn't, I've not made an accurate judgment. Expert judgments are correct more often than layperson judgments, and that's part of what makes them experts. This is why you should be skeptical of your brother-in-law's claimed expertise on wine. The odds are good that he would fail a blind taste test miserably.
Third, a truth-related incentive is something that rewards you for getting to the truth. For example, university faculty have strong truth-related incentives. When we miss the truth in our research, our careers often suffer. In contrast, a political pundit on TV has no such incentive.
The world is full of experts, and often we ought to trust them. People ignore experts to their own detriment on everything from health to finance to personal relationships. If you doubt this, I encourage you to pick up Tom Nichol's book The Death of Expertise.
However, our trust in experts ought to be calibrated. Experts are experts in something. Expertise isn't a general category. You can be an expert in botany but a neophyte in programming, an expert in international relations and an ignoramus about astronomy. Expertise is always domain-specific.
So, here's the punchline: almost no expert domain includes the future. Being an expert in entomology doesn't confer expertise about whether ant numbers will increase or decrease over the next ten years. Being an expert in corporate finance doesn't confer expertise about future interest rate moves. And being an expert in political affairs doesn't mean you're an expert in elections.
In fact, in areas like these, expert predictions about the future are barely better than random chance. If episodes in your own life like the recent election don't convince you of this, check out Future Babble: Why Expert Predictions are Next to Worthless by journalist Dan Gardner. It's a layman's introduction to the scholarly research on expert predictions. It's a pessimistic story. If you had to pick between your local senator's prediction about the upcoming presidential election or a flip of the coin, there's no reason to prefer one over the other.
Now that's not to say that there aren't any experts with expertise in predicting the future. There are. Two good examples are weather forecasters and statistical analysts like the team at fivethirtyeight.com. In both cases, they make falsifiable, yet largely accurate predictions about the future and they have strong incentives to get them right. In other words, we have experts whose expertise is aimed at predicting the future, and both do a pretty good job with their forecasts.
The lesson is to trust experts when they make judgments that fall inside their domain and not outside their domain. Experts know more than we do within the boundaries of their expertise. It's almost always irrational not to trust them in those cases. But expert domains rarely extend to the future. If your MD tells you that your health would benefit from a statin, trust her. If she tells you that private health insurance will be gone in five years, just smile and nod. And if you stop relying on experts for predictions, you won't be surprised by what the future delivers, whether that's stock market performance or election results.