It's True Unless It Isn't
On average, right-wing people are more conscientious than left-wing people. This is basically a universal law - it’s true in the UK, it’s true in the US, it’s true in Switzerland (and this study is looking at the relationship between voting for right-wing populists and being conscientious, where you might expect the relationship to be different), and so on. It’s true when you control for income, it’s true when you control for age. It’s just something that’s true - like saying that right-wing people tend to be older and left-wing people tend to be younger.
Except, it isn’t always true. In China, trait conscientiousness is associated with being left-wing. It’s also the case that in China, being more authoritarian is associated with being more left-wing. Things in China are different - left-wing people who support the CCP tend to have the characteristics of right-wing people in other countries. But even that story doesn’t give you the whole picture, because left-wing people in China still tend to be higher in trait Openness than right-wing people (as they are everywhere else). This isn’t just a reversal of the patterns in other countries, something weirder is going on. The other example I gave of something that’s ‘true’ isn’t always true either: older people aren’t always more right-wing. In Israel, younger voters are more right-wing. In the social sciences lots of things are true, except where they aren’t.
The examples I’ve given are about different countries - social science research might indicate that something appears to be true pretty much everywhere and then it turns out that it isn’t true in some country we haven’t looked at yet. This doesn’t just apply to different countries, it also applies to different eras. Women used to disproportionately vote Conservative in the UK, now they disproportionately vote Labour. This also seems like it might be a problem when you try to replicate studies. If some social science paper from a decade ago found that left-wing people tend to be X, and some attempt to replicate the paper found that actually, no, left-wing people tend to be Y, it can be tricky to know whether the original paper was bogus or whether things are just different now.
Another problem here is that people have sticky beliefs - updating is hard. So if you’re an epidemiologist or a virologist or whatever, and the evidence seems to indicate that lockdowns are a good idea, you might hold on to this view for too long. This is sort of like an anchoring effect - people tend to rely too heavily on the first piece of information they hear when making decisions. Michael Story wrote about how people with sticky beliefs tended to screw up their analysis of the pandemic:
“The big lesson of all of this for me is that updating is really hard. Each change in the pandemic has left a new group of people high and dry, stuck to a no-longer operable model with outdated assumptions”.
Things are true, until they aren’t. I was also caught out by this a bit. When I was forecasting on how many people would catch covid in some country or another, I was caught off guard a few times by how quickly things would change. Some country would appear to have everything under control, there would be articles heaping praise on their handling of the virus, and then suddenly cases would skyrocket.
In a way, this is all completely obvious. Of course things can change, and of course social facts that appear to be true here and now may be untrue somewhere else and at some other point. But I do think that people tend to underestimate how much things can change, and how quickly they can change. And I guess that they also become overly confident in research that may well apply to some specific place at some specific time.
I definitely do this. One study I like is Andrew Hall’s paper on what happens in the House of Representatives in the United States when an ideologically extreme candidate wins a primary election in a coin-flip election over a more moderate candidate. It uses a nice quasi-experimental design, and finds that when an extreme candidate wins a primary, the chance that their party wins the seat decreases by between 35 and 54 percentage points. I’ve cited this study sometimes, using it to argue about whether the Labour Party in the UK should opt for more radical or more moderate candidates. But am I really sure this paper is strong evidence here? It uses data stretching back to 1980 (and until 2010) from elections in the United States, it seems like pretty poor evidence that the Labour Party in the United Kingdom today should opt for ideologically moderate candidates!
Some of your core opinions are probably determined in your pre-adult years. Cohort effects are powerful - people who were teenagers and young adults in times with relatively low inflation tend to underestimate the chance of inflation throughout their lives, and the opposite occurs with people who were teenagers and young adults in times with relatively high inflation. If there comes a time when the evidence begins to suggest that ideologically extreme candidates are more likely to win elections than ideologically moderate candidates, I think I will update extremely slowly as the new evidence rolls in. It’s true for now (I think), but there may be a time when it isn’t. And I’m pretty sure that I’ll be caught off guard when it happens.