Cargo-cults and scientific uncertainty

“It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty…”

Richard Feynman

Around 2010, I was talking to Jo Glanville, then editor of Index on Censorship, about evidence that science had a problem with bad research design or opportunistic reporting of unreliable results – leading to ‘findings’ that were no such thing. Nowadays, this has become known as the replication crisis, and has been closely followed by an Open Science movement. Some people don’t like the word ‘crisis’ as it alienates researchers who feel personally attacked. But there is unarguably a good deal of soul-searching going on, and a fierce conversation about research methods, research-funding, career incentives and much besides.

The piece I finally wrote for Jo was about the need to declare up-front the doubts and uncertainties in research, and also about the pressures to suppress them. Thinking about the early days of the replication saga the other day, and wondering what I’d said about uncertainty back then, I dug out an old draft. It was around the time of the tentative announcement of the discovery of the Higgs Boson – and I thought that the tentativeness was important. Interestingly, most of the big themes in the replication saga now were already beginning to feel familiar then (Some say they were known or suspected more than twenty years ago). Anyway, here it is with few adaptations from the piece that appeared in an edition of Index called ‘Dark Matter – What’s Science Got to Hide’ (volume 40 number 4), published in 2011.

Earlier this summer, scientists thought they might have found the ‘God particle’ – big news for physics, as you can tell by the name.

Actually, the name is unfortunate and smacks too much of showbiz. I say that because the God particle might turn out not to exist, and bringing up the curtain on an empty stage when you’d promised God wouldn’t look too clever. Hype in these cases is risky.

Properly known as the Higgs Boson, this pip of a speck of a fragment of matter was postulated nearly 50 years ago to explain how other sub-atomic particles have mass. It popped into view, or seemed to, in a 27 km circular tunnel under the French-Swiss border known as the large hadron collider, dubbed the world’s largest experiment.

Inside the LHC, proton particles are smashed together at fiendish speed while detectors examine short-lived traces of sub-atomic wreckage. The process sounds straightforward, though doubtless somewhat techie – at £6bn for the kit and running costs, it ought to be. So why did researchers say they only thought they might have found the Higgs Boson? Was it in the wreckage – or more properly as I understand formed by the energy in the wreckage – or not?

Maybe. Not sure. Might be able to tell you later. Can you come back? 

And that, believe it or not, is an underrated standpoint in science. The is-there/isn’t-there saga of the Higgs Boson illustrates an important principle; that there’s more uncertainty in science than science showbiz sometimes lets on. 

There is also a compelling case to make this uncertainty explicit.  They’d look prize lemons at the LHC if they announced that they’d found God, later to back down. Of which type of behavior, more later. For needless to say, not everyone sees the value in uncertainty this way. Disclosing it, like many forms of disclosure, is sometimes judged inconvenient or trivial. 

The science ideal holds that all findings are provisional. Anyone can take away any result and try to replicate it. If it doesn’t stand up, they can say so. But the issue here goes beyond arguing with people who have doubts about your results. It extends to proclaiming the uncertainty surrounding one’s own results. It is about being first to point up the doubts, not the defensive last to acknowledge that yes, there might be something after all in other people’s ifs and buts. 

Why should anyone volunteer such humility? Because having done the research, they probably know already, or should do, where the doubts are. In part, this reassures us that enthusiasm for the answer is less likely to have run away with itself. 

But since people may have invested any combination of heart, soul, years, energy, belief, money and reputation in their results, and since politics, employment and more money might depend on them, that’s easier said than done. How would you feel about being invited to doubt your own research? A wholehearted, voluntary declaration of the uncertainties takes courage. 

Is such an expectation naive? Let’s first consider some reasons why uncertainty arises and then look at the case for talking more about it. 

Imagine you are seeking to confirm the existence of the last tiger in the jungle. You find a rough outline of a footprint that looks promising. How can you be sure from this one footprint that there’s really a tiger about? For you know that chance alone might from time to time produce a mark that looks tiger-like but isn’t the real thing. To be more confident, you look for others, you seek confirmation. And until you have it, you remain cautious and uncertain. Proof in such a case is in large part statistical. How many footprints of what quality do we need before we believe? 

Just so with the Higgs Boson – and also with another physics fragment lately in the news, the WIMP, or weakly interacting massive particle, sometimes known as dark matter. Researchers at the LHC found footprints resembling those that they expected for the Higgs Boson. But they did not find enough amidst an immense quantity of data to be confident that this was the real thing. Was it, they wondered, merely an imposter shaped by chance debris? 

So they announced their initial excitement but expressed caution, looked some more, only to be disappointed: still not enough footprints. Since then, they have begun to hunt the Higgs Boson in another part of the sub atomic jungle. They’re frank that it might not exist. They say they simply don’t know. 

A little further along the continuum of belief and evidence, WIMP hunters in Italy were reported recently to have found 67 hints of its existence from a quantity of experiments that suggested this was not likely to be the result of mere chance, but more likely to be evidence of the real thing. But they went no further. They did not announce a discovery, merely said that their results added to the weight of other evidence. 

Mathematically-based science has a reputation for seeking certainty: hard facts nailed down with hard numbers. But statisticians, of all people, urge a modus operandi at odds with this reputation. Faced with uncertainty, they say, we must exercise caution – and by the way, there’s oodles more uncertainty in nature’s slow disclosure of its secrets than we might expect. An awful lot that appears meaningful at first sight turns out otherwise. There’s nothing necessarily underhand in this. It is simply that in a big world chance sure gets around. Nature seems almost to play games with us, dangling meaning then snatching it away. 

To one who cut his intellectual teeth in the humanities, it was a surprise to find that practitioners of a quantitative discipline went on about why their own findings might be wrong. Done properly, it is a mightily impressive act of self-discipline and rigour.

Those hunting the Higgs Boson similarly spoke of suggestive findings not conclusive ones. They took pains to explain to journalists the risk of error brought about by chance resemblance. Don’t get too excited, they said. Though this was perhaps easier for them than others because they would be as curious if the Higgs boson didn’t exist as if it did. That is unusual. Negative findings – nothing doing – don’t often set the pulse racing.

This type of uncertainty – the uncertainty arising from chance patterns and not-always-reliable hints – is one of many. There is also, for example, the structural uncertainty of modeling. That is, is your understanding of the way in which this event combined with that force leads to such-and-such an outcome as part of some natural process, and through which you hope to pump some numbers to see how they come out at the other end, a sufficiently accurate or useful representation of what really happens? 

And we can’t talk about uncertainty these days without also mentioning what is now known colloquially as Black Swan or Rumsfeldian uncertainty, the unknown unknowns that might bite you on the ankle with something you never thought of. Blindness to the possibility of Black Swans in finance, for all its quantitative sophistication, is often now said to have contributed to the 2008 banking crisis. 

The analogy with the world of finance is useful. Banks and hedge funds rely on highly-paid mathematicians and economists – ‘quants’ – to evaluate risk. One quant I know of – more inclined to doubt than others – now teaches the numerati of the financial world to go to work each day applying what he calls devil’s advocate risk management. Imagine, he says, that the bank has just lost $5bn. Now figure out how it happened – before it actually does. Since by definition it is not possible to know for sure about Black Swans in advance, the essence of this approach is to be restless for scenarios in which all that you fervently believe to be true, built as it may be on all manner of neat mathematics, is nevertheless wrong. 

Again, the humanities student is struck by how much quantification benefits from a good imagination for falsification.  

These varieties of uncertainty need to be individually evaluated depending on the scientific research at issue. The uncertainty can even sometimes be roughly quantified. That is, it might be possible to say how likely you think you are to be wrong. 

So far, so responsible. And also all rather tedious, wouldn’t you agree? Especially if you are worried about what other people will do with your admission of uncertainty – if you think they might use it, for example, as an excuse to ignore you, or worse. 

And therein lies part of the danger. Caution in the face of uncertainty seldom makes news. It sounds plodding. In a sense, it is plodding. Doubt is restrained, wearing a frown, suggesting (sometimes erroneously) inaction; confidence strides out with its teeth done. Uncertainty stands re-counting its fingers; assurance elbows the queue for headlines and a new grant. Bold claims, that’s what tend to grab attention and win influence. Discoveries bright and hot. Breakthroughs. Shiny promises of new treatments, new technologies. Sunlit uplands of understanding. Or, on the other side, dire warnings and sudden dangers revealed.

In those cases where science also informs political decision-making, it’s hardly a fair contest. The brassy assertion that the new dawn is within reach or a new threat imminent is an easy rhetorical win and might even on occasion be true. The hesitant confession of uncertainty is derided as ignorance, weakness or indecision. One only has to imagine the politician who stands before the electorate to say: “erm…” No, the time for action is now, says the other side.

Let’s take the dangerous example of climate science. Uncertainty in climate science can be affirmed without in any way implying that the whole thing is a global hoax. To say that there are uncertainties here is to state the obvious – and ought to be uncontroversial. But has the mainstream view on climate science suffered on occasion from a seeming desire to deny it? 

In the controversy in 2009 now known as ‘climate-gate’, a server at the University of East Anglia was hacked and emails emerged suggesting – among other things – reluctance to disclose data to critics, partly it seems for fear that they might mine it with a skeptical or mischievous eye for any uncertainty or other evidence useful to their cause. 

A science writer at the The New York Times commented that some scientists were ‘so focused on winning the public-relations war that they exaggerate their certitude — and ultimately undermine their own cause.’

There was also controversy surrounding extravagant claims in a report by the International Panel on Climate Change about glacial melting. At about this time, The Times newspaper interviewed Sir John Beddington, the government’s chief scientific adviser and a man with few doubts at all about the broad validity of mainstream climate science. The paper summarized his view thus: ‘Public confidence in climate science would be improved [my emphasis] if there were more openness about its uncertainties, even if that meant admitting that skeptics had been right on some hotly-disputed issues.’

Several inquiries eventually exonerated the University of East Anglia. Professor Beddington said: ‘I don’t think it’s healthy to dismiss proper skepticism. Science grows and improves in the light of criticism. There is a fundamental uncertainty about climate change prediction that can’t be changed.’

The claim that Himalayan glaciers would disappear by 2035 was, he reportedly said, part of a wider problem.

‘Certain unqualified statements have been unfortunate. We have a problem in communicating uncertainty. There’s definitely an issue there. If there wasn’t, there wouldn’t be the level of skepticism. All of these predictions have to be caveated by saying, ‘There’s a level of uncertainty about that’.’

So the question here is not only about the advisability of entering an argument. It’s whether scientists censored the sophistication of their own understanding for the sake of a calculation about appearances. Not quite showbiz, but not far off. I suspect they probably knew – and still know – better than their opponents where the uncertainties are. Perhaps airing them feels like doing the devil’s own work, perhaps there’s fear of identifying potential weaknesses to attack. For science as public understanding, this is, I’d argue, ultimately anti-science. Acknowledging that there’s uncertainty is not weakness but strength for at least two reasons: a) It fosters trust. b) It’s usually true.  

Seeking to resist the public recognition of uncertainty (if that’s what happened), as if it would prove the proverbial nail in a shoe on which the whole battle could be lost, was ultimately bad strategy. No matter the short-term justification, it seems to have ended by exciting suspicion: ‘what are you trying to hide?’ 

That argument applies regardless of the merits of the theory that climate change is a real result of human activity. You might be pleased at the damage done to that mainstream view; you might be appalled. That there has been damage seems hard to deny. 

There’s growing acceptance that all this could be handled better. In 2010, a review of the processes and procedures of the IPCC by the Inter Academy Council identified the communication of uncertainty as a key area for improvement.

Perhaps there’s too much shame in being honestly uncertain, not enough in pretending to be right or confident. But in the league table of obstructive vanities, unwillingness in science to try to anticipate possible error or publicly explore the uncertainties ranks high.

That is, provided science and not politics is the game. In politics, winning on a false prospectus seems to be thought of the rough and tumble, at least by some. So what if there were no weapons of mass destruction in Iraq; the principle justifying invasion remains correct, they might say, and if a dodgy dossier swayed a timid public, ah well.   What’s a doubt or two weighed against a greater cause? Science inevitably becomes embroiled in this from time to time. When it does, if it also becomes economical with the actualite, it’s asking for trouble.  

To all that, it might be said that hedging your bets only matters if your bets turn out badly. Why bother with this probabilistic fence-sitting if you’re usually right in the end?  

So let’s take another example where we might feel a personal interest in knowing that the standard answers are correct: medical research. 

Professor John Ioannidis of Tufts University is an authority on error in medical research. He finds plenty. I interviewed him a few years ago when few seemed to know quite how to respond to his arguments and evidence. One of his most cited papers is called: Why Most Published Research Findings Are False. 

Ioannidis selected 49 of the most important research papers in medicine over 13 years – judging their importance by how often they were cited. Of these, 45 announced effective interventions. Thirty-four of these claims had been retested, and 14 of these, or 41%, had been shown, he has said, “to be wrong or significantly exaggerated.” He throws out aphorisms such as: “The hotter a scientific field… the less likely the research findings are to be true.”

A few people don’t like the way Ioannidis counts errors, but he would have to be very wrong indeed to be easily dismissed. At any rate, he poses uncomfortable questions. If so many findings turn out wrong in part or whole, what happens in the first place to the evidence that ought to exist to suggest that they were wrong? Are researchers too gung-ho in the hunt for a ‘finding’? At the very least, is there too much readiness to discount the uncertainties? 

To this suspicion that something is being lost we must also add the filtering effects of the media. For even when uncertainties are proclaimed loudly in their original academic publication, they tend to disappear when reported more widely. For example, if an academic paper offers a range of uncertainty around say, a health risk, the media will typically seize on the figure at the most scary end of that range and say “could be as high as…” The possibility at the lower end of the range vanishes. Some guidance in the media about whether a reported figure comes with reasonable hope of accuracy or is the kind that would struggle to hit a barn door would be useful. But that’s not showbiz. 

It’s also been shown in studies published in the Lancet, the Journal of the American Medical Assocation, and elsewhere, and particularly in the work of Oxford University’s Doug Altman – who first explained it to me – that there are malign filtering effects within the academic community. Altman makes the apparently undemanding argument that research ought to be reported fully and accurately, but offers much evidence that this does not happen, citing missing, incomplete or ambiguous information, in particular a lack of detail about methods and results, misleading interpretation and selective reporting. See here for example.

Research claiming to have found something (positive) is more likely to be published than research that says it can find nothing either way (inconclusive) or that there is nothing to be found (negative). In fact, if you find nothing, you’re less likely to bother offering it for publication in the first place. These effects are varieties of what’s known as publication bias.

Publication bias is now a well-known and largely acknowledged phenomenon – a few journals say they are even trying to counteract it – but it has been only more recently explored than you might expect.  Publication bias encourages research that claims to find a resounding ‘yes’ rather than a hesitant ‘maybe’. The cumulative effect is to weed out many of the cautious in favour of the confident.

One final example that involves media, politics and uncertainty. When Britain’s medical authorities said in 2009 that there was a risk from swine flu that 65,000 people might die and one third of the population become infected during the coming winter, they also said that the figure could be as low as 3,000 deaths – a level not at all unusual for the effects of your everyday humdrum winter flu.

As ranges of uncertainty go, that’s pretty wide. This was ‘would it hit a barn door?’ territory. It was also the territory of frequent statements in Parliament, wall-to-wall news coverage, the stockpiling of medicines, school closures and plans to vaccinate everyone and the dog. Some public health specialists were unhappy that the 65,000 figure ever saw the light of day. It was an extreme estimate, the result of taking the top end estimate for the infection rate and combining that with the top end the estimate for the fatality rate. 

‘I would steer you well away from that 65,000 figure’ said the BBC’s medical correspondent Fergus Walsh, in a rare media moment of anti-hype. 

One for health officials in future might be to ask what would happen if they ‘fessed up to the uncertainty: ‘Honestly? We just don’t know. Might be bad – and here’s how bad if we really ramp up the worst-case scenarios, but we think that’s a big outside bet. Might be nothing at all unusual for a winter. Seriously. Nothing much. Let’s not be complacent, so watch this space.’ Whether that would imbue more panic or less is a moot point. 

Thinking about how you’d bet on the possibilities is an idea advocated by David Spiegelhalter, Professor of the Public Understanding of Risk at Cambridge University. He also runs a website called Understanding Uncertainty. 

I should declare an interest. I know the Professor and have been influenced by him. A newspaper comment piece of his recently was headlined: ‘Scientists need the guts to say: “I don’t know.”’

He wrote: ‘A popular view of scientists is that they deal with certainties, but they are (or should be) the first to admit the limitations in what they know.’ 

Mind you, there are also dangers in elevating respect for uncertainty. Junk or pseudo science might attempt to clothe itself in the respectability of rational skepticism to attack a well-reasoned scientific consensus. Commercial interests have, on occasion, attempted to throw a vicious spanner in the works of scientific understanding and the public interest by suggesting that there are serious grounds for doubt and inaction even in the face of compelling evidence – of which perhaps the most notorious example is how smoking can damage your health. 

And science is not always in the dark, of course. At some point, findings often become reliable. So some varieties of uncertainty might give way, eventually. But surely, the sooner admitted and confronted, the sooner overcome.  

In what has come to be known as the ‘Cargo Cult’ address to the California Institute of Technology in 1974, the celebrated physicist Richard Feynman said: 

‘It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty – a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid – not only what you think is right about it… Details that could throw doubt on your interpretation must be given, if you know them.  You must do the best you can—if you know anything at all wrong, or possibly wrong—to explain it….

‘In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.’ *

On holiday this year, I found myself among a group of hospital consultants and asked how often they harboured serious doubts about whether the treatment they offered did any good. ‘All the time,’ they said. 

‘But sometimes people don’t want to know about the uncertainties,’ said one. ‘They just want to be told they’ll get better.’ And sometimes such wishful thinking – via the placebo effect – works. 

But I wouldn’t want to rely on wishful thinking about results in science in general. I doubt the God particle believes in it. At least, in the spirit in which this article has been written, I think that’s what I think. Naturally, I welcome your doubts.  

***

Feynman, fabulous as usual, explained the cargo cult analogy like this: ‘In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to imitate things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas—he’s the controller—and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.’