(2025-12-26) Caplan Tackling Tetlock

Bryan CaplanTackling Tetlock. Philip Tetlock, one of my favorite social scientists, is making waves with his new book, Expert Political Judgment. Tetlock spent two decades asking hundreds of political experts to make predictions about hundreds of issues.

This book is literally awesome – to understand Tetlock’s project and see how well he follows through fills me with awe.
And that’s tough for me to admit, because it would be easy to interpret Tetlock’s work as a great refutation of my own.

Most of my research highlights the systematic belief differences between economists and the general public, and defends the simple “The experts are right, the public is wrong,” interpretation of the facts. But Tetlock finds that the average expert is an embarassingly bad forecaster

Is my confidence in experts completely misplaced? I think not. Tetlock’s sample suffers from severe selection bias. He deliberately asked relatively difficult and controversial questions.

Experts really do make overconfident predictions about controversial questions. We have to stop doing that! However, this does not show that experts are overconfident about their core findings.
What Tetlock really shows is that experts can raise their credibility if they stop overreaching.