Occam’s razor: The weaknesses of simple-looking explanations

By Gordon Rugg

So what is Occam’s razor anyway, and why should anyone care?

The core concept is brief: Other things being equal, we should choose the simplest valid explanation whenever possible.

Amateurs often view this concept as a clean blade of truth, cutting straight to the heart of the matter. It’s widespread in politics, often phrased as “common sense” analysis.

That’s a nice idea, but reality is more complex, and Occam’s razor often causes more problems than it solves. Like the damaged, time-worn razor in the picture below, it’s far from being a flawless blade.

This article is about why simple-looking explanations often turn out to be complicated in reality, and why apparently complicated explanations often turn out to be simple.

Occam’s razor is a concept named after its originator, William of Occam (also spelled “Ockham”). He was a mediaeval scholar who wrote something to the effect of “entia non sunt multiplicanda praeter necessitatem”. There’s debate about his exact original wording. The meaning is approximately “Don’t bring more things into an explanation than you really have to”.

That idea was taken up and rephrased in various ways by numerous scholars over the centuries. Some of them were well aware of the pitfalls surrounding it; others weren’t. That’s where the trouble begins.

The razor of ancient wisdom

Something worth remembering about William of Occam was that he lived and died a long time ago. At the time of his death in 1347, the discovery of America was over a century in the future; the idea that the Earth was in orbit round the Sun would have been viewed as heresy if anyone had thought of it; concepts such as the scientific method or electricity or the internal combustion engine were five centuries away. It was taken for granted that everything in the material universe was composed of four elements in various proportions. I’ve blogged previously about where that particular approach took the Roman writer Vitruvius:

“6. To begin with fir: it contains a great deal of air and fire with very little moisture and the earthy, so that, as its natural properties are of the lighter class, it is not heavy.”

That doesn’t exactly inspire confidence. If we’re using the analogy of a razor cutting to the heart of the matter, then the four elements model would probably look like the razor below: good in its day, but not something that you’d want to use now.

However, there are plenty of people who try to simplify present-day debates by invoking old concepts, like someone trying to shave with a corroded ancient blade.

One common version of this approach is argument by authority. Its limitations are clear if we phrase it in terms of razors: “This person was much more significant than you, therefore their razor is better than yours”.

The razor in the picture below once belonged to King Frederick I of Württemberg. He may have been powerful and famous once, but his razor isn’t something you’d want to use now. It’s much the same with the ideas of ancient philosophers. Those philosophers may have been great in their day, and they may have been brilliant, and they may have made great innovations in their field, but their ideas are showing their age. Even Euclid’s geometry, which has arguably stood the test of time better than anything else from ancient Greek times, is now recognised as only a subset of geometry – there’s an entire field of non-Euclidean geometry, for instance.

The second common version of this approach is that there’s nothing new under the sun. This version argues that a modern concept is no more than a rephrasing of some ancient concept. “Particle physics? That’s just a version of the atomic theory that Democritus developed around 400BC.”

You don’t very often see this approach applied to the hard sciences, which know how to bite back (“So, just what did Democritus have to say about the Higgs boson?”) but you do often see it in the humanities. It works best when there are a lot of ancient texts that say something vague about the topic under discussion. It doesn’t work well when the ancient texts are specific, because then it’s usually clear that they’re factually wrong, but if the texts are vague, they can be re-interpreted to fit whatever argument their proponent is making, creating the superficial appearance of wisdom, without needing to get bogged down in sordid reality, such as actually fixing a problem.

The safety razor of scientific method

Early philosophers and theologians were pretty enthusiastic about applying logic and argument to almost anything, regardless of whether there was any tangible proof of that the topic of debate actually existed in the first place.

A key feature of the scientific method, which began a couple of centuries after the death of William of Occam, was that its scope was solidly grounded in phenomena that could be observed. This gave it less scope than the old philosophy, but, like the safety razor, it had built-in features that made it less likely than its predecessor to go significantly wrong before being stopped.

Armchair scientists are often very keen on the scientific method, and will readily criticise publishing researchers for departing from the proper Scientific Method™ as half-learned by those armchair pundits. Occam’s razor is quite often mentioned in those criticisms.

The reality, however, is that usually the key question isn’t how to make an explanation simpler; usually, the key question is just how simple the explanation needs to be. There’s a famous quotation usually ascribed to Einstein, to the effect that a scientific model should be made as simple as necessary, but no simpler. (Various phrasings occur; this one is arguably the most accurate with regard to how science works.) That’s why I phrased the opening description of Occam’s razor as: “Other things being equal, we should choose the simplest valid explanation whenever possible.”

Often, paradoxically, you get a much cleaner and more powerful explanation by using a more complicated model, because the universe happens to be more complicated than expected in that particular area.

The multi-blade razor of reality

Modern razors are usually multi-blade, because that gives a safer, closer shave. The one below has three blades.

It’s a similar story with much of modern science. The examples below involve three different reasons for apparently more complex explanations being better than simpler ones.

When reality is more complex than expected.

One common finding in research is that there are two or more varieties of something that was once thought to be a simple, single category.

Blood is a classic example. Until Victorian times, it was taken for granted that blood was simply blood. The discovery that there were numerous different blood groups, and that there were complicated interactions about which types could be safely transfused, was a complete surprise to everyone involved.

A less well-known example is that human beings have two very different types of memory, namely long term memory (LTM) and short term memory (STM). They’re different in the physical processes involved, and in capacity, and duration – LTM has enormous capacity and lasts for decades, whereas STM has a capacity of about seven items, and a duration of a few seconds.  That’s why car registration numbers and computer passwords and the individual part of a phone number after the area code are all about seven characters long, so that they’re easier for people to hold in STM for a few seconds. Again, there was no a priori reason to expect this, but any attempt to model how the brain works has to include this distinction.

In both these cases, reality consists of two or more categories for something that was once thought to be a single, unitary item. In other cases, the advantages of using an apparently more complex explanation come from being able to represent reality in a more powerful way, rather than coming directly from reality itself.

When more complex representations are better

An example of advantages from more complex representations is modelling human handedness.  The model that most people take for granted is a single line going from “completely left handed” at one end to “completely right handed” at the other end. In this model, ambidextrous people are usually placed in the middle of the scale.

In reality, however, many alleged ambidexters are in fact equally clumsy with each hand, whereas others are dextrous with both hands. We can represent this more accurately with a diagram like the one below.

The real added bonus from this representation is that you can use the same idea to make more sense of a lot of other issues as well.

Gender is an example – that’s where I first encountered this representation, in Bem’s work on androgyny. You can use the same representation to describe a person in terms of how “masculine” they are, and also to describe the same person in terms of how “feminine” they are (for simplicity, I won’t get into the issue of what, if anything, is meant by traditional views of masculinity and femininity).

In an increasing number of movies, there are female main characters who are high both in traditional feminine characteristics and also in traditional masculine characteristics – for instance, Sigourney Weaver in the Alien franchise, Linda Hamilton in Terminator II, Milla Jovovich in the Resident Evil franchise. There hasn’t been such a marked effect with regard to male main characters; an advantage of this representation is that you can easily show that significant absence visually, as a gap in the image.

So, although a solution may be more complicated in relation to a single problem, the same solution may also solve other problems as an added bonus.

The next subsection is about a very different type of complexity.

Systems and complexity

In systems theory it’s usually the interactions that are complex, not the components. Systems theory has been around for a long time, but it’s not as widely known as it should be, even though it’s invaluable for providing rigorous models of real-world situations where things often happen in ways that surprise people.

In systems theory, a system is usually defined in terms of a set of interconnected and interacting components. The schematic images below shows how a system is different from a simple collection of parts.

The first image shows several items that are not joined into a system – a battery, a bulb, and two lengths of wire.

These items, as a simple collection of things, won’t do anything except slowly age.

If, however, the same items are joined together in the right configuration, then they become a system, and they behave very differently as a system from how they behaved as individual entities.

Now, they do something completely new. When they’re combined like this, the bulb lights up. That’s a new property arising from the interconnections and interactions between the components. It’s an example of what’s called an emergent property.

This isn’t the only interesting thing that systems do. A common feature of systems is feedback loops. These are completely different from concepts like “positive verbal feedback” where you say something pleasant to someone; instead, positive feedback loops and negative feedback loops are key concepts in making systems safe.

In a feedback loop, one component of a system alters a second component in some way, and the second component in turn alters the first component. A classic example is a domestic heating system, where the heater makes the thermostat become more warm; when the thermostat reaches a critical temperature, it switches off the heater, as a negative feedback loop.

A positive feedback loop is similar, except that instead of the second component counteracting the effect of the first component, the second component increases the effect of the first component. A vivid example is forest fires, where the heat of the fire makes the air above the forest rise, creating winds that suck in colder air from ground level, which in turn make the fire burn more fiercely, and so on.

Usually negative feedback loops help to keep situations stable, and positive feedback loops tend to make situations become increasingly unstable.

Because systems have emergent properties that would be hard to predict from the properties of their individual components, and because the interactions between system components often produce large numbers of positive and negative feedback loops, it’s often difficult to predict their behaviour in advance.

This is a common issue in politics. Economies and societies are very large, very complex systems; they have emergent properties and feedback loops that are very different from those in smaller systems, or in individual components within a system. One common mistake in populist politics/economics, for instance, is to assume that national and international economies are just the same as a household budget, only on a larger scale. They’re not. The larger systems behave very differently from the smaller systems, in ways that often look very odd. That’s a common side-effect of feedback loops, which frequently lead to systems behaving in the opposite way to what you might expect at the start.

Unexpected outcomes are the topic of the next section of this article:

The electric razor of the totally unexpected

Another characteristic of apparently simple explanations is that they usually draw on familiar, everyday phenomena such as water flowing through a hose. That’s fine if you’re dealing with a problem which involves familiar, everyday mechanisms like the water in the hose. However, a lot of natural phenomena simply don’t behave like the everyday ones that we’re used to. Many people find this very unsettling.

Electricity is a prime example. It doesn’t behave the same way as water flowing along a pipe. It behaves like electricity, which is sometimes similar to water in a pipe, but more often different. When electricity was first installed in homes, a lot of people had trouble with the idea that electricity wouldn’t just flow to waste through any outlets that weren’t filled with something such as a plug or a light bulb. A lot of people still have trouble grasping how it really behaves, because there isn’t a good physical analogy that we can compare it to.

Quantum physics is another example of something that simply behaves on its own terms, and that most people find very hard to grasp.

There’s a neat idea from Kelly’s Personal Construct Theory that describes this conceptual problem. It’s called the range of convenience. Every descriptive term has its own range of contexts within which it can be meaningfully applied (the range of convenience). Outside that range, the term is meaningless.

Concepts from everyday experience such as “causality” are extremely useful within their range of convenience (everyday experience). However, that doesn’t mean that their range of convenience is infinite. Often, terms that are very useful for describing a system are meaningless if you try to apply them to the component parts of that same system. For instance, it’s perfectly sensible to apply the concept of “miles per gallon” to a car, as an entire system, but meaningless to try applying that same concept to individual parts of a car, such as the chassis or suspension.

Which takes us back to one of the quotes from the start of this article. A scientific model should be made as simple as necessary, but no simpler. Sometimes, that takes you into strange territory.

As a closing thought, here’s a new version of an old engineering adage, adapted to explanations.

You can have your explanation simple, accurate, or comfortingly like everyday life. Choose any two.

On which inspiring note, this article ends.