(I’m trying to get back into the habit of posting once a week or more, so do forgive me if the articles seem a little rusty or unoriginal while I get back into the swing of things!) There are good reasons to be sceptical of experts. In Phillip Tetlock’s book
Something that I think may be an effect is that in order for collective expertise to work, individual experts have to rely on their own subjective judgment calls. A naive layperson can probably beat the typical expert by taking the average views of all experts taken together (as their errors may cancel out), but when the typical expert adopts that as their rule, the field becomes degenerate.
Of course an expert can be heterodox in their public arguments and take the average in their pragmatic bets, and I suspect almost all experts do this to a degree, but I also expect they also entangle these to a degree.
Well, this article seemed a little rusty and unoriginal, but I enjoyed it anyway!
Seriously though, I really like the idea that prediction making and information gathering are separate skills. I'm sure there are plenty of predictors who succeed primarily on the back of the info that others have gathered, seems reasonable to value experts who succeed primarily on the back of the info they unearth.
Great post, but whilst it's certainly a good thing outsiders can fairly easily discern whether a study will replicate or not, that implies something along the lines of researchers themselves being dishonest/unintelligent or something, for taking research in their fields seriously or pretending to. Which is certainly quite scary.
I've been thinking about this a lot recently. A question I have a lot is, to what extent is expertise useful in a year and to what extent is it useful now. Like I think many experts are good at solving problems today, but figuring out which problems in their field will need to be solved in a year is kind of a different problem.
Experts have been useless since time immemorial. Pharaoh called his magicians, Nebuchadnezzar called his astrologers and necromancers and demonologists and snake charmers and wizards and wise men, and even Eve consulted with the snake.
Are experts useless?
Something that I think may be an effect is that in order for collective expertise to work, individual experts have to rely on their own subjective judgment calls. A naive layperson can probably beat the typical expert by taking the average views of all experts taken together (as their errors may cancel out), but when the typical expert adopts that as their rule, the field becomes degenerate.
Of course an expert can be heterodox in their public arguments and take the average in their pragmatic bets, and I suspect almost all experts do this to a degree, but I also expect they also entangle these to a degree.
Well, this article seemed a little rusty and unoriginal, but I enjoyed it anyway!
Seriously though, I really like the idea that prediction making and information gathering are separate skills. I'm sure there are plenty of predictors who succeed primarily on the back of the info that others have gathered, seems reasonable to value experts who succeed primarily on the back of the info they unearth.
Great post, but whilst it's certainly a good thing outsiders can fairly easily discern whether a study will replicate or not, that implies something along the lines of researchers themselves being dishonest/unintelligent or something, for taking research in their fields seriously or pretending to. Which is certainly quite scary.
Very glad to have you regularly posting again!
This is very convincing
I got 16 out of 21
I've been thinking about this a lot recently. A question I have a lot is, to what extent is expertise useful in a year and to what extent is it useful now. Like I think many experts are good at solving problems today, but figuring out which problems in their field will need to be solved in a year is kind of a different problem.
Experts have been useless since time immemorial. Pharaoh called his magicians, Nebuchadnezzar called his astrologers and necromancers and demonologists and snake charmers and wizards and wise men, and even Eve consulted with the snake.
And yet, they're still around.