Subsystem optimisation and system optimisation

By Gordon Rugg

When a politician or a manager tells you that they’re going to take a “common sense” approach to a problem, they usually mean that they’re about to take a concept that works in one area, and apply it to a different area on the unstated assumption that it will work there too. Once in a while, it actually does. A lot of the time, though, it ends less than well.

There’s a technical description for one common way in which this approach comes to grief.

Sub-system optimisation does not necessarily lead to system optimisation.

It’s not exactly the most catchy line ever written, which might be why it’s not as widely known as it should be. This article is about what it means, and why it’s important.

A lot of the human world involves systems and subsystems. Some of them are physical, and some are abstract. A car engine is a physical system, which consists of subsystems such as the electrical subsystem and the fuel subsystem. A commercial company is an abstract system, which consists of subsystems such as the accounts department and the marketing department. A government is an abstract system which consists of subsystems such as departments and agencies.

Because systems are so pervasive and so important, they’ve been studied a lot over the years. Back in the nineteenth century, the sociologist Max Weber did some classic work on how bureaucratic systems actually behaved, as opposed to how they were officially supposed to behave. His findings have stood up solidly to the test of time. In the early twentieth century, researchers  such as von Bertalanffy started using a mathematical/engineering formalist approach to modeling systems and their behaviour, under the heading of General Systems Theory. Again, this work has stood up well to the test of time, and underpins a huge swathe of the infrastructure for present-day civilisation.

Both the sociological approach and the formalist approach produced findings which were robust, but which are highly counter-intuitive – in other words, the findings are the opposite of what you might expect. One of those findings is the topic of this article. If you set out to improve each of the sub-systems within a system, you don’t necessarily improve the system as a whole; in fact, you might make the system worse.

Why does that happen, and what can you do about it?

Here’s a fictional account of one way that it can happen, based on a real case that I saw at first hand.

First example: one subsystem’s gain is another subsystem’s loss

Here’s a diagram showing an imaginary company, Acme Roadrunner Supplies. The company contains various departments, such as Sales, Marketing, Product Support, Design, Production and Customer Relations.

The company as a whole is making a modest annual profit, and has a reasonable market share, plus a good reputation.

One day, trouble arrives in this quiet fairytale land, in the form of a new senior management team, who want to make their mark. They commission an internal costing exercise, to identify how well each department is performing as an internal cost centre. The results look like this.

The company as a whole is making a modest profit (mid green).

Sales and Marketing are generating high internal profits (dark green). Product Support, Design, and Marketing are all making small internal profits (light green). Customer Relations is breaking even (yellow).

The senior management team react to this with a memo instructing all departments to make themselves internally profitable, on the assumption that this will raise the overall profit of the company as a whole. This looks like trouble for Customer Relations. However, the head of Customer Relations is a veteran of corporate re-organisations, and knows just how to stop this idea in its tracks. It just takes one sentence at the right time. Here’s how it plays out.

Under the current system, all customer complaints come to Customer Relations. That makes a lot of sense. Dealing with customer complaints is a potential minefield for all sorts of reasons. For instance, if one of your staff responds to a customer’s complaint by apologising, does the apology amount to a legal admission of guilt? The legal side alone requires a lot of specialist knowledge, involving substantial training costs; however, that’s an investment that will pay for itself the first time it heads off a lawsuit. On a more positive note, if you treat customer complaints as a way of identifying flaws in your product, and improving it, then the complaints can actually be a long-term asset to your design team and your product support staff. Also, if you handle customer complaints well, then that can significantly improve your public profile. So, dealing with customer complaints properly can have high costs that are easy to measure exactly, but can also have high benefits that are harder to measure exactly.

We can add this to the diagram by showing customer complaints as a sub-subsystem within the box for Customer Relations. It’s the set of small bright red boxes. The rest of Customer Relations would make a small internal profit without it (as shown by the light green background in the rest of the Customer Relations box).

So, if you’re the head of Customer Relations, faced by an internal cost-measuring exercise, what can you do?

One simple and devastatingly effective strategy is to wait until you run into some of the other departmental managers over lunch, and mention in passing that you’re planning to move your department into internal profitability by delegating the handling of customer complaints to the individual departments within the company. So, for instance, if a customer wants to complain about the product not working properly, then their complaint will be handled by Product Support, who will have the specialist knowledge about why the product isn’t working as it should.

On the surface, this looks like a perfectly sensible approach, and there will almost certainly be ways of presenting it to senior management with costings that make it look more cost-effective than the current system.

In reality, as every departmental head around that lunch table will know, this is a recipe for disaster. Under this new system, every department will now need to train up at least one member of staff in handling customer complaints. That’s going to cost money. It will also cost time; a staff member handling a complicated customer complaint will have to take time away from their work on design, or whatever their normal role is. In addition, it will cause conflict and chaos, because each department will try to pass the problem over to a different department. If the customer complains that the product isn’t working properly, then Product Support will say that it’s a problem in the initial design, and pass it over to Design. Design will claim that the design was fine, but that the marketing material gave an inaccurate description of what the product would do, so they pass the complaint on to Marketing. With each hand-over, the customer is getting more irate, leading to a worse public perception of the company as a whole.

There’s a fixed cost in training, etc, for each of the customer complaints. That now appears in each of the other departmental boxes, as a small red box. Previously, the company only paid that fixed cost once, for Customer Services. Now, the company is having to pay it six times. In addition, the company is having to absorb the costs of internal buck-passing, as complaints get shuffled from one department to another (the second set of small red boxes) in addition to actually resolving the complaint (the third set of small red boxes). Customer Relations hasn’t escaped completely unscathed; it will still have to handle some complaints, and buck-passing from other departments, but it will be handling fewer complaints than before. The result looks something like this.

The overall result is that three departments will manage to show modest internal profits, including Customer Relations, but the company as a whole ends up with higher internal costs, less overall profit, and a worse reputation. You’ve optimised one sub-system, but at the expense of the other sub-systems, and of the system as a whole.

That’s one way for things to go wrong as a result of subsystem optimisation, because one subsystem’s gain is a greater loss to other subsystems. The next example shows how all the subsystems can improve, without any subsystem’s gain being another subsystem’s loss, but with the result being bad for the system as a whole.

Second example: all the subsystems improve, but the system gets worse

The next figure shows what that situation could look like. All the departments are now showing an internal profit – including customer relations. However, the company as a whole is now only just breaking even.

How could that happen? Here’s one way.

Before the new management team’s drive for internal efficiency, all the departments used the same software platform. There were a lot of grumbles about this, because each department felt constrained by that standardised system.

In this scenario, once they are told to improve their internal profitability, the departments agree that they will each shift to the software platform that best suits their needs. Product Support shift to Linux, Design shift to Macs, Customer Relations switch to an obscure platform that supports a highly efficient freeware expert system for handling complex legal questions, and so forth. Each of these changes is a great success; for instance, all the departments report that the new design documents are much easier to interpret, saving them significant amounts of work.

However, there is a new problem that arises. In the previous setup, all the departments could easily exchange documents etc. Here’s a schematic diagram showing a simplified version of those interactions (the thin black lines).

In the new setup, the situation is more complicated. For example, Design produces a brilliant mockup for a new product, but it can only be read by Macs with a particular piece of software installed. Some emails start being bounced back as undeliverable because of obscure interactions between the software systems that the different departments are using. So, although each department is individually better as a result of the new mixed economy, and although each department is more pleased with what they’re getting from the other departments once they actually manage to receive it, there’s a significant problem with the interactions between the new IT systems.

Who owns this problem? The different departments can all make a strong case for it not being their problem; they are all now more efficient and profitable, as a result of their new choice of IT platforms. There’s a strong case for this problem belonging to the organisation as a whole, in the form of the senior management team. When you represent the problem schematically, with thicker lines to show the higher level of complications and costs for communications in the new setup, then there’s an obvious answer.

It would make sense to have a new department just to handle the IT issues, where the black box is in the diagram. A new department will mean new costs, which might well defeat the whole point of the internal profitability exercise.

So, in this example, improvements in all the subsystems have led to a worsening of the system as a whole. It’s not what you might have expected at the outset, but it’s a very real problem, that you can see in numerous real-world cases once you start looking for them.

Lessons and applications

So, what happens when someone spots this risk? All too often, the scheme gets pushed ahead regardless, and ends badly for the system as a whole.

There are plenty of high profile examples in politics and in business. The privatisation of the British rail system is a classic case. It was marketed as a way of increasing efficiency via competition between the new private rail networks (the subsystems). In reality, it resulted in a tangle of interaction problems that still hasn’t been resolved, including a ticket system and a pricing structure that defy description.

Once in a while, though, someone manages to explain this issue to senior management in language that they understand, and the new scheme is quietly and swiftly dumped down the memory hole and forgotten. That’s what happened in the case I witnessed. So there’s a positive note on which to close; there can be a positive outcome…

Notes

If you have no idea what internal costings are, then there’s a whole new world of fascinating experiences awaiting you, ranging from the merely bizarre through to situations stranger than anything Kafka could have imagined.

If you’re interested in General Systems Theory, then it’s well worth a visit. It’s not terribly fashionable these days, for some reason, though its concepts and principles are now integral parts of fields where things need to work, such as engineering, electronics and computing. The core concepts are simple and easy to master, but often produce counter-intuitive findings, like the one discussed in this article.

If you’re interested in Weber’s work, then the core concepts of his work provide most of the plots for “Yes, Minister” and “Yes, Prime Minister”. That must prove something…

Advertisement