The Hidden Half


The Hidden Half is a book of mysteries and errors that expose ignorance and uncertainty.

Here’s just one. In the mid 1990s in Germany, a new variety of crayfish appeared overnight, in a fish tank. It had spontaneously started cloning itself – no males, all females – an entirely new species of clones, never seen in the wild.

This was amazing enough. But when scientists heard about them and put them in tanks to study, they went further: they also standardised the environment, rearing these clones in exactly the same, carefully lab-controlled conditions.

Now, everything was the same. Genes and environment, the lot. So, what did they look like, these clones in an identical environment? About the same? Maybe exactly the same? 

One was 20 times the weight of another. Every single one of several hundred studied had unique markings. They laid different numbers of eggs, had hugely different lifespans. They moved differently, rested differently. They were physically different. They were also behaviourally different: some liked a crowd, some were loners, some were dominant, some subservient, and so on, and on.

They were chalk and cheese. 

Why? The short answer is… we don’t know.

And with that, both sides of an argument – nature v nurture, genes v environment – that splits the world in two, are confronted with the fact that there is a great, gaping hole in our understanding that neither can fill. It’s a fact they studiously ignore. 

Normally, we say that if it’s not genes, it must be environment; if it’s not environment it must be genes. But this seems in some way to be neither. And with that, a pile of presumptions goes up in smoke and, if we are honest, we sit scratching our heads.

Because then, the central question in the book, as we’re forced to ask: what are we missing? What can explain this?

Nature/Nurture and the third force.

And that’s the hidden half. There’s something out there that means when we look for patterns and order – how one thing causes another – we fail to see it clearly. Somehow, the world hides its secrets. 

What goes for crayfish also goes to a large extent for people. There are hypotheses and good evidence that there must be some heavy infusion of randomness or noise in the developmental process, either biological or environmental. We have little understanding how this works in detail to produce any particular outcome – it is plausibly so volatile that we will never find systematic influences in this noise. We can’t even trace cause and effect in these instances after the event, and we are nowhere near being able to predict it.

I suggest in the book that the problem of hidden and disruptive influence is common, extends well beyond animal or human biology, and contributes to a collective blind-spot for the complexities of causality in economics, politics, research and elsewhere. Too often, we have been kidding ourselves about what we knew. 

A book about understanding that’s not about cognitive bias.

The usual culprits for failures of understanding are inside our heads, in cognitive limitation or bias. I argue that the problem is as much outside our heads as in, arising from the monstrously intricate nature of what’s out there to understand and analyse. The world doesn’t easily submit to the dreams of tidy minds.

To grasp all the relevant, volatile causal factors and their interactions to deliver robustly generalisable evidence for decision-making is a lot harder than often supposed, and often impossible; there’s a Hidden Half to what-causes-what that we are bound repeatedly to miss and that often makes fools of us.

The point is not that we shouldn’t try to find it, nor that every effort is certain to fail. Clearly not. It is that we’re far too confident, and seem to think we’ll routinely succeed – or even that we have succeeded when we were mistaken. Or maybe we flatter ourselves that an outcome was down to understanding and judgement, when in truth it was just dumb luck.

We dream of a world of clarity, order and understanding. Sometimes, we find it. Sometimes we only think we do. The book could be described as a catalogue of evidence of the various ways in which optimistic, intellectual self-confidence is both common and badly mistaken, in a world where half the machinery is beyond us.