Occam’s razor: The weaknesses of simple-looking explanations

By Gordon Rugg

So what is Occam’s razor anyway, and why should anyone care?

The core concept is brief: Other things being equal, we should choose the simplest valid explanation whenever possible.

Amateurs often view this concept as a clean blade of truth, cutting straight to the heart of the matter. It’s widespread in politics, often phrased as “common sense” analysis.

That’s a nice idea, but reality is more complex, and Occam’s razor often causes more problems than it solves. Like the damaged, time-worn razor in the picture below, it’s far from being a flawless blade.

straight razor 4

This article is about why simple-looking explanations often turn out to be complicated in reality, and why apparently complicated explanations often turn out to be simple.

Occam’s razor is a concept named after its originator, William of Occam (also spelled “Ockham”). He was a mediaeval scholar who wrote something to the effect of “entia non sunt multiplicanda praeter necessitatem”. There’s debate about his exact original wording. The meaning is approximately “Don’t bring more things into an explanation than you really have to”.

That idea was taken up and rephrased in various ways by numerous scholars over the centuries. Some of them were well aware of the pitfalls surrounding it; others weren’t. That’s where the trouble begins.

The razor of ancient wisdom

Something worth remembering about William of Occam was that he lived and died a long time ago. At the time of his death in 1347, the discovery of America was over a century in the future; the idea that the Earth was in orbit round the Sun would have been viewed as heresy if anyone had thought of it; concepts such as the scientific method or electricity or the internal combustion engine were five centuries away. It was taken for granted that everything in the material universe was composed of four elements in various proportions. I’ve blogged previously about where that particular approach took the Roman writer Vitruvius:

6. To begin with fir: it contains a great deal of air and fire with very little moisture and the earthy, so that, as its natural properties are of the lighter class, it is not heavy.”

That doesn’t exactly inspire confidence. If we’re using the analogy of a razor cutting to the heart of the matter, then the four elements model would probably look like the razor below: good in its day, but not something that you’d want to use now.

Lund razor

However, there are plenty of people who try to simplify present-day debates by invoking old concepts, like someone trying to shave with a corroded ancient blade.

One common version of this approach is argument by authority. Its limitations are clear if we phrase it in terms of razors: “This person was much more significant than you, therefore their razor is better than yours”.

The razor in the picture below once belonged to King Frederick I of Württemberg. He may have been powerful and famous once, but his razor isn’t something you’d want to use now. It’s much the same with the ideas of ancient philosophers. Those philosophers may have been great in their day, and they may have been brilliant, and they may have made great innovations in their field, but their ideas are showing their age. Even Euclid’s geometry, which has arguably stood the test of time better than anything else from ancient Greek times, is now recognised as only a subset of geometry – there’s an entire field of non-Euclidean geometry, for instance.

royal razor

The second common version of this approach is that there’s nothing new under the sun. This version argues that a modern concept is no more than a rephrasing of some ancient concept. “Particle physics? That’s just a version of the atomic theory that Democritus developed around 400BC.”

You don’t very often see this approach applied to the hard sciences, which know how to bite back (“So, just what did Democritus have to say about the Higgs boson?”) but you do often see it in the humanities. It works best when there are a lot of ancient texts that say something vague about the topic under discussion. It doesn’t work well when the ancient texts are specific, because then it’s usually clear that they’re factually wrong, but if the texts are vague, they can be re-interpreted to fit whatever argument their proponent is making, creating the superficial appearance of wisdom, without needing to get bogged down in sordid reality, such as actually fixing a problem.

The safety razor of scientific method

safety razor

Early philosophers and theologians were pretty enthusiastic about applying logic and argument to almost anything, regardless of whether there was any tangible proof of that the topic of debate actually existed in the first place.

A key feature of the scientific method, which began a couple of centuries after the death of William of Occam, was that its scope was solidly grounded in phenomena that could be observed. This gave it less scope than the old philosophy, but, like the safety razor, it had built-in features that made it less likely than its predecessor to go significantly wrong before being stopped.

Armchair scientists are often very keen on the scientific method, and will readily criticise publishing researchers for departing from the proper Scientific Method™ as half-learned by those armchair pundits. Occam’s razor is quite often mentioned in those criticisms.

The reality, however, is that usually the key question isn’t how to make an explanation simpler; usually, the key question is just how simple the explanation needs to be. There’s a famous quotation usually ascribed to Einstein, to the effect that a scientific model should be made as simple as necessary, but no simpler. (Various phrasings occur; this one is arguably the most accurate with regard to how science works.) That’s why I phrased the opening description of Occam’s razor as: “Other things being equal, we should choose the simplest valid explanation whenever possible.”

Often, paradoxically, you get a much cleaner and more powerful explanation by using a more complicated model, because the universe happens to be more complicated than expected in that particular area.

The multi-blade razor of reality

Modern razors are usually multi-blade, because that gives a safer, closer shave. The one below has three blades.

mach 3 razor

It’s a similar story with much of modern science. The examples below involve three different reasons for apparently more complex explanations being better than simpler ones.

When reality is more complex than expected.

One common finding in research is that there are two or more varieties of something that was once thought to be a simple, single category.

Blood is a classic example. Until Victorian times, it was taken for granted that blood was simply blood. The discovery that there were numerous different blood groups, and that there were complicated interactions about which types could be safely transfused, was a complete surprise to everyone involved.

A less well-known example is that human beings have two very different types of memory, namely long term memory (LTM) and short term memory (STM). They’re different in the physical processes involved, and in capacity, and duration – LTM has enormous capacity and lasts for decades, whereas STM has a capacity of about seven items, and a duration of a few seconds.  That’s why car registration numbers and computer passwords and the individual part of a phone number after the area code are all about seven characters long, so that they’re easier for people to hold in STM for a few seconds. Again, there was no a priori reason to expect this, but any attempt to model how the brain works has to include this distinction.

In both these cases, reality consists of two or more categories for something that was once thought to be a single, unitary item. In other cases, the advantages of using an apparently more complex explanation come from being able to represent reality in a more powerful way, rather than coming directly from reality itself.

When more complex representations are better

An example of advantages from more complex representations is modelling human handedness.  The model that most people take for granted is a single line going from “completely left handed” at one end to “completely right handed” at the other end. In this model, ambidextrous people are usually placed in the middle of the scale.

In reality, however, many alleged ambidexters are in fact equally clumsy with each hand, whereas others are dextrous with both hands. We can represent this more accurately with a diagram like the one below.

ambidexterity v3
The real added bonus from this representation is that you can use the same idea to make more sense of a lot of other issues as well.

Gender is an example – that’s where I first encountered this representation, in Bem’s work on androgyny. You can use the same representation to describe a person in terms of how “masculine” they are, and also to describe the same person in terms of how “feminine” they are (for simplicity, I won’t get into the issue of what, if anything, is meant by traditional views of masculinity and femininity).

In an increasing number of movies, there are female main characters who are high both in traditional feminine characteristics and also in traditional masculine characteristics – for instance, Sigourney Weaver in the Alien franchise, Linda Hamilton in Terminator II, Milla Jovovich in the Resident Evil franchise. There hasn’t been such a marked effect with regard to male main characters; an advantage of this representation is that you can easily show that significant absence visually, as a gap in the image.

So, although a solution may be more complicated in relation to a single problem, the same solution may also solve other problems as an added bonus.

The next subsection is about a very different type of complexity.

Systems and complexity

In systems theory it’s usually the interactions that are complex, not the components. Systems theory has been around for a long time, but it’s not as widely known as it should be, even though it’s invaluable for providing rigorous models of real-world situations where things often happen in ways that surprise people.

In systems theory, a system is usually defined in terms of a set of interconnected and interacting components. The schematic images below shows how a system is different from a simple collection of parts.

The first image shows several items that are not joined into a system – a battery, a bulb, and two lengths of wire.

system componentsv2

These items, as a simple collection of things, won’t do anything except slowly age.

If, however, the same items are joined together in the right configuration, then they become a system, and they behave very differently as a system from how they behaved as individual entities.

system

Now, they do something completely new. When they’re combined like this, the bulb lights up. That’s a new property arising from the interconnections and interactions between the components. It’s an example of what’s called an emergent property.

This isn’t the only interesting thing that systems do. A common feature of systems is feedback loops. These are completely different from concepts like “positive verbal feedback” where you say something pleasant to someone; instead, positive feedback loops and negative feedback loops are key concepts in making systems safe.

In a feedback loop, one component of a system alters a second component in some way, and the second component in turn alters the first component. A classic example is a domestic heating system, where the heater makes the thermostat become more warm; when the thermostat reaches a critical temperature, it switches off the heater, as a negative feedback loop.

A positive feedback loop is similar, except that instead of the second component counteracting the effect of the first component, the second component increases the effect of the first component. A vivid example is forest fires, where the heat of the fire makes the air above the forest rise, creating winds that suck in colder air from ground level, which in turn make the fire burn more fiercely, and so on.

Usually negative feedback loops help to keep situations stable, and positive feedback loops tend to make situations become increasingly unstable.

Because systems have emergent properties that would be hard to predict from the properties of their individual components, and because the interactions between system components often produce large numbers of positive and negative feedback loops, it’s often difficult to predict their behaviour in advance.

This is a common issue in politics. Economies and societies are very large, very complex systems; they have emergent properties and feedback loops that are very different from those in smaller systems, or in individual components within a system. One common mistake in populist politics/economics, for instance, is to assume that national and international economies are just the same as a household budget, only on a larger scale. They’re not. The larger systems behave very differently from the smaller systems, in ways that often look very odd. That’s a common side-effect of feedback loops, which frequently lead to systems behaving in the opposite way to what you might expect at the start.

Unexpected outcomes are the topic of the next section of this article:

The electric razor of the totally unexpected

electric razor

Another characteristic of apparently simple explanations is that they usually draw on familiar, everyday phenomena such as water flowing through a hose. That’s fine if you’re dealing with a problem which involves familiar, everyday mechanisms like the water in the hose. However, a lot of natural phenomena simply don’t behave like the everyday ones that we’re used to. Many people find this very unsettling.

Electricity is a prime example. It doesn’t behave the same way as water flowing along a pipe. It behaves like electricity, which is sometimes similar to water in a pipe, but more often different. When electricity was first installed in homes, a lot of people had trouble with the idea that electricity wouldn’t just flow to waste through any outlets that weren’t filled with something such as a plug or a light bulb. A lot of people still have trouble grasping how it really behaves, because there isn’t a good physical analogy that we can compare it to.

Quantum physics is another example of something that simply behaves on its own terms, and that most people find very hard to grasp.

There’s a neat idea from Kelly’s Personal Construct Theory that describes this conceptual problem. It’s called the range of convenience. Every descriptive term has its own range of contexts within which it can be meaningfully applied (the range of convenience). Outside that range, the term is meaningless.

Concepts from everyday experience such as “causality” are extremely useful within their range of convenience (everyday experience). However, that doesn’t mean that their range of convenience is infinite. Often, terms that are very useful for describing a system are meaningless if you try to apply them to the component parts of that same system. For instance, it’s perfectly sensible to apply the concept of “miles per gallon” to a car, as an entire system, but meaningless to try applying that same concept to individual parts of a car, such as the chassis or suspension.

Which takes us back to one of the quotes from the start of this article. A scientific model should be made as simple as necessary, but no simpler. Sometimes, that takes you into strange territory.

As a closing thought, here’s a new version of an old engineering adage, adapted to explanations.

You can have your explanation simple, accurate, or comfortingly like everyday life. Choose any two.

On which inspiring note, this article ends.

Advertisements

5 thoughts on “Occam’s razor: The weaknesses of simple-looking explanations

  1. I feel like this article went straight over my head.

    The only real difference between “entia non sunt multiplicanda praeter necessitatem” (which apparently is old and rusty) and “A scientific model should be made as simple as necessary, but no simpler.” (which apparently is shiny and useful) that I can see is in scope (scientific models vs anything you might have to say to explain anything that is).

    You argue based on vague rephrasings of Occam’s razor (“Other things being equal, we should choose the simplest valid explanation whenever possible.”, “Don’t bring more things into an explanation than you really have to”), give a lengthy introduction that seems to prepare me to accept that Occam’s razor may be an outdated concept, then mention a few common mistakes in the application of Occam’s razor (most importantly that “praeter necessitatem” is often ignored and that the determination of what constitutes necessitas is the core factor in applying the razor correctly), and then close with yet another rephrasing of the razor.

    I don’t see how your rephrasings sharpen or otherwise modify the razor other than by restricting it to scientific models only.
    You provide a sensible modern interpretation of necessitas in Occam’s sentence that is entirely consistent with the sentence itself (i.e. doesn’t require any modifications to the sentence, so why change it up?).

    It’s particularly strange to me that your favorite rephrasing even features the word “necessary”.
    Do you think Latin “necessitas” is too vague to be a useful restriction and in need of a sharpening but then are fine with English “necessity”?
    And why do your English “translations” avoid the word “necessity” like the plague, instead speaking about “valid[ity]” and “[that which] you really have to [do]”.

    This article is just confusing to me.

    • Thanks for the comment.

      I think we may be at cross-purposes. The main point I was trying to make was that Occam’s razor is often interpreted to mean that we need to seek a single causal mechanism, whereas in reality many phenomena are better explained by the interaction between two or more causal mechanisms. I think that a lot of research has suffered from a needlessly protracted effort to find a single explanation.

      If there’s a fair amount of high quality evidence that is apparently contradictory or inconsistent, then this is often a sign that more than one mechanism is in play.

      If, on the other hand, there isn’t much evidence yet, and the evidence appears to be reasonably consistent, then it makes sense to assume that only one mechanism is involved.

      Does that help make more sense of the distinctions I was drawing?

      Gordon

  2. Pingback: Occam's razor: The weaknesses of simple-looking...

  3. Pingback: The Knowledge Modelling Book | hyde and rugg

  4. Pingback: Instrumental and expressive behaviour | hyde and rugg

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s