When ‘probable’ is probably wrong

Over-confidence in probabilities and some simple adding up.

Speaking at an event, I showed a slide of a BBC headline: ‘Unemployment falls’ it said: ‘by 3,000’.

And then the gag – the uncertainty around that number: plus or minus 77,000.

This means,’ says the ONS ‘that we are 95% confident the actual change in unemployment was somewhere between an increase of 74,000 and a fall of 80,000.’

So much for the confidence of the BBC’s ‘fall’.

‘But…’ said someone in the audience as we looked at another example of uncertainty (around immigration), the central estimate is still the most likely number.

He was right, it is. Given the available data, ‘down 3,000’ is more likely to be the right number than any other particular number, despite the wide uncertainty around it.

But when latching onto the most probable number – and to probabilities in general – it pays to remember how improbable the most probable can be.

Think of everyone having 10 tickets for a lottery. Then say that I have 20. The most probable winner is me. But I will almost certainly not win. It is overwhelmingly probable that someone else will win – someone whose chances of winning are actually only half as likely as mine. Similarly, it is overwhelmingly probable that unemployment that month did not fall by 3,000, even though 3,000 is the most probable number.

You (probably) know all this. Even so, I think it’s easy to forget in the moment. Applied to probabilities like the Bank of England’s GDP estimates and forecasts, even the Bank’s famed uncertainty could be interpreted with less confidence. Here’s one of its fan charts.

I like these fan charts. They were a great innovation, giving a sense of the inevitably wide uncertainty both in the Bank’s own forecasts, and the substantial uncertainty around present and historic data.

They show the best range of estimates as a band of dark green in the middle. These are judged to have 30% probability. It’s tempting to say this central band contains the most likely outcome, and to feel this is where we should focus our expectations. Well, kind of. It’s more likely than any other 30% band – in the Bank’s judgement. But this central estimate is not most probably right; it is most probably wrong. The sum of the other probabilities is, of course, 70%. So, it is more than twice as likely as not – in the Bank’s judgment – that the eventual number will be outside even the range that includes the best estimate. Obvious, really.

But as so often in the hunt for information, it’s easy to grasp too hard at a straw of knowledge, and to forget how feeble it is.

Like the lottery tickets, you can have the best chance going, and it’s still no chance at all. GDP – past, present and future – is a bit better than a lottery, but we can still be tempted into betting with overconfidence – even when the Bank graphically illustrates that we’re most likely to be wrong.

Unspectacular, just worth remembering, that’s all.