By Gordon Rugg
Often in life a beautiful idea is brought low by an awkward reality. Explosive leaf level fan out is one of those awkward realities (though it does have a really impressive sounding name, which may be some consolation).
So, what is it, and why is it a problem? Can it be a solution, as well as a problem? These, and other questions, are answered below.
Categorisation is a deeply embedded feature of the world, in areas as diverse as laws and supermarket aisle layout. Usually, we only notice it when it goes wrong, or does something unexpected, as in the example below.
Cropped from https://www.pinterest.co.uk/pin/AZgaUW8MtPVSTXzKvXeq_Yj3zNRCIRzDkAb6_24iKelbkm_FUX9JYxo/: Used under fair use terms
Many categorisation systems are hierarchical, where you have a few categories at the top, divided into sub-categories, then into sub-categories, and so on, as in the diagram below.
A common way of constructing hierarchies is to start at the top with the categories, and then work downwards, deciding on the sub-categories and further sub-divisions as you proceed. This is known as the top-down approach.
Media genre is a familiar example of this approach, and of the problems that it can encounter. For instance, at first it makes good sense to treat “film” and “music” as separate categories. However, this hits problems with opera and with musicals; where do they fit? There’s a similar familiar problem with supermarket aisle layout, with products such as dried fruit; should dried fruit be next to fruit, because it’s dried fruit, or next to flour, because it’s widely used in baking, like flour?
The “where do we file it?” problem is different from leaf level fan out, but the two issues often overlap, as in the examples below.
When you go down through the layers of a hierarchy, you eventually emerge from categories into specific instances; for example, from the category of blues to the instance of the song Hard time killing floor blues.
This level of specific instances is known as the leaf level, by analogy with a tree dividing into branches and then twigs and finally ending with leaves. Depending on the details and purposes of the categorisation, leaves may take different forms. For instance, you might choose to treat a particular song as a leaf, or you might treat a particular cover of that song as a leaf, or you might treat each separate CD containing a particular cover of that song as a leaf.
The diagram below shows several layers of categorisation, ending at leaf level with schematic leaves.
At first glance, this looks like a manageable hierarchy. However, if we see the same information in a different layout, then the problems start to become apparent.
The image above shows how the hierarchy is rapidly broadening at leaf level. This is leaf level fan out. In this particular example, the fan out is clearly visible, but with only a few tens of leaves, there isn’t a huge problem.
However, what often happens with real cases is that a hierarchy suddenly goes from a manageable number of twigs into an enormous number of leaves. Music categorisation is a classic example, with an enormous number of leaf-level songs being mapped onto a comparatively tiny set of twig-level categories.
Explosive leaf level fan out is when the fan out at leaf level is much greater than the fan out at higher levels of the hierarchy. It’s an informal term, so you can use your judgment about whether or not to apply it to a particular case.
A key issue about explosive leaf level fan out is that it can turn a problem from being difficult to being impractical to solve or impossible to solve. This often occurs with top down solutions proposed by inexperienced managers, who may design solutions that look okay for the first few levels, but that fall apart completely when they encounter explosive leaf level fan out.
An example is attempts to legislate against dangerous dog breeds. At first sight, this looked like a case of using existing categorisations of dogs, and specifying which breeds should be classified as dangerous, with the breeds treated as the leaf level.
However, it soon became apparent that in addition to problems such as how to categorise mixed-breed dogs, there was so much individual variation of dog behaviour within breeds that treating breeds as the leaf level was not sensible. Instead, it looked more sensible to treat each individual dog as a leaf. This led to explosive leaf level fan out, because of the huge number of individual dogs involved, making the entire approach infeasible just in terms of fan out.
So, explosive leaf level fan out can be a major problem. Sometimes, though, it can be the mirror image of a very useful solution.
What happens if, instead of going through a hierarchy from the top downwards, you go through it from the bottom upwards?
What often happens is that you start off with an apparently intractable number of individual cases; far too many to handle on a case-by-case basis.
However, if you have a good categorisation system, that enormous number of leaves will map onto a manageably small number of twigs. This lets you reduce enormous leaf level complexity to a much more tractable set of categories.
How do you decide whether a categorisation system is good, and how can you design one which is good? We’ve addressed these issues in our article on the 70/20 principle, which looks at how to design systems to handle common cases, fairly common cases, rare cases, and unique cases.
So, in summary: Explosive leaf level fan out is a useful concept to know, and something to watch out for when using or designing systems. As a problem, it can wreck a badly thought out system; as a solution, it can bring order to chaos.
The concepts described in this article draw heavily on graph theory, which is an extensive area of maths. We’ve blogged about it briefly here.
There’s more about the theory behind this article in my latest book:
Blind Spot, by Gordon Rugg with Joseph D’Agnese
There’s more about job applications and interviews in The Unwritten Rules of PhD Research, by myself and Marian Petre.
Although it’s mainly intended for PhD students, there are chapters on academic writing which are highly relevant to undergraduate and taught postgraduate students. It’s written in the same style as this article, if that’s any encouragement…
The Unwritten Rules of PhD Research, 2nd edition (Marian Petre & Gordon Rugg)