By Gordon Rugg
He cried in a whisper at some image, at some vision,–he cried out twice, a cry that was no more than a breath–
“‘The horror! The horror!’
This article is one in a series about the problem of identifying and clarifying client requirements, using the ongoing semi-humorous example of a client’s requirement for an image of an elephant. The previous episodes have looked at some bad ways of tackling the problem. This episode looks at methods for tackling difficult areas of requirements gathering, where people are for various reasons reluctant to talk about a particular topic. It also looks at the underlying reasons for that reluctance.
That takes us into some uncomfortable and morally ambiguous territory, which is why I’ve opened with a quote from Heart of Darkness. If you’re trying to fix real problems, then you need to know how to find out the realities, and that isn’t always fun.
Some parts of reality you’ll know about via personal experience. Other parts, though, are trickier. How do you find out what people actually do behind closed doors, or when they think nobody is watching? Here are some methods.
Shadowing is simple and effective. It is what it sounds like; you get someone’s permission to follow them round for a while, and you see what happens when you follow them. That can give some surprising insights. One of my students shadowed some IT managers. She also interviewed them, and used a couple of other methods to find out about what they did. It was only when she shadowed them that she discovered that they often worked straight through the day without a lunch break. For them, that was something so familiar that they hadn’t thought of mentioning it.
Shadowing can be particularly useful for spotting apparently minor points that are much more significant than they look. For example, if you shadow a parent who has a child in a pushchair, you’ll quite likely find that they use the disabled toilets, because those have much easier access and more space. They also avoid problems that would otherwise be awkward, such as what a father does if he has a small daughter with him who needs to go to the toilet – he can’t go into the women’s toilet, but taking her into the men’s toilets wouldn’t be a great option either. There are other groups who also routinely use the disabled toilets because of the extra space, such as older shoppers with wheeled shopping baskets. In fact, with regard to toilet provision, there are now effectively three genders, namely male, female and people needing extra space.
Once you start thinking that way, then you can start thinking about ways of adding other design features to help those categories of users. For instance, should you include more coat hooks than usual in disabled toilets, to accommodate children’s coats and parents’ coats?
Shadowing is also effective for spotting hassle points, where a feature is causing irritating problems that are individually minor, but that are affecting large numbers of people every day. Pushchairs and wheeled shopping baskets are again classic examples, as regards awkward provision of steps and stairs, or cramped access points. A little effort redesigning those points can make a big overall difference to the usability of a public space, and reduce the number of accidents and consequent lawsuits.
If you’re planning to use shadowing, it’s a good idea to do a couple of pilot sessions first, and to try analysing the data from those pilot sessions before you start your main work. That will identify the key low-level issues that you’ll need to get right – for instance, what the best method for recording your observations is, and how to measure what you’re observing. A useful tip is to treat swearwords as your best friend; whenever the person you’re shadowing swears, or causes someone else to swear, then you’ve found a problem that needs to be treated as a high priority. Another useful indicator is if the person you’re shadowing suddenly stops talking – to you, or the child in their pushchair – because they’re having to concentrate on something that they find risky or difficult.
Projective approaches can also be very effective. When I was working with some colleagues on the problem of why mature students dropped out of higher education courses, we knew why the students said that they dropped out. Far and away the most frequent stated reason was money problems. That was what they were saying, and that was what surveys across the country were reporting, but we suspected that it wasn’t the full story. So we used a projective questionnaire.
The first half of the questionnaire asked the respondents to list the five main reasons that they thought most mature students would give for dropping out of a course. The second part asked the respondents a very similar question, but with one key twist. It asked the respondents to list the five main reasons for mature students actually dropping out of courses that those students would be unwilling to mention. The questionnaire was projective because the respondents were answering as if they were someone else, not themselves. As we expected, that made a big difference to the results we obtained.
The results from the first half of the questionnaire were in close agreement with the results from previous surveys. That was a useful calibration result for our findings.
The results from the projective second half were very different, and included all sorts of things that people hadn’t been mentioning in the previous surveys, such as sex, drink, drugs and jealous partners.
Our questionnaire also asked the respondents to rate how serious they thought each of the issues really was. The ratings suggested that the “not talked about” reasons were not significantly more serious than the reasons that people were willing to talk about.
That doesn’t prove that those previously-unmentioned issues really were the causes of students dropping out. With projective techniques, there’s always a risk of tapping into urban legends. However, the results gave us some very interesting pointers about topics for follow-up research using other techniques such as indirect observation.
In brief, indirect observation involves making inferences from things that you can observe about things that you can’t observe directly. Here’s an example that brings together several key points, continuing the elephant theme. It’s a picture of a baboon picking out undigested seeds from elephant dung and eating them. Reality isn’t always pleasant, but if you want to improve a situation, then you need to understand the reality.
So what can you infer from this image via indirect observation? For a start, you can tell that an elephant has been here. You can actually tell a lot more from the elephant dung, if you know what to look for; you can tell what the elephant has been feeding on, and when it dropped the dung. You can tell other things as well, with much deeper significance. Why is the baboon foraging for seeds in elephant dung rather than gathering seeds directly off plants? Combined with the dry, bare ground, that behaviour suggests that this photo was taken in a hard season, with little food available.
That’s what you get if you look beyond any initial squeamishness you might feel. Once you get into the habit of looking for indirect evidence, and looking past any initial squeamishness, you soon start spotting sense in things that had previously looked random or unimportant. Often, getting past the squeamishness is a key point, since it lets you see the underlying patterns in reality much more clearly and completely.
Consultants who deal with organisations can tell a lot from indirect observation. Usually, they’ll have a pretty good idea of the state an organisation’s in before they even reach the client’s office door, as a result of knowing what tell-tale signs to look for.
The next sections look at underlying causes for people being reluctant to tell you the reality; there are several main reasons, each with different implications.
Unofficial working practices
The German sociologist Max Weber lived at a time when Prussian bureaucracy was in full flower, and he made full use of that opportunity to study organisational behaviour at its most elaborate.
One of the many interesting patterns he described involved the importance of unofficial working practices. These are practices that bend, break or ignore the official procedures, often for the simple reason that the official procedures just don’t work in the real world.
Here’s an example. Back in the old days, if you were on a building site and you uncovered a section of metal pipe, the official working practice was to call all the relevant organisations – the gas board, the electricity board, and the water board – and asked them if it was one of their pipes, and if so, whether it was live. What quite often happened was that they all denied responsibility for it, which left you with a problem. The unofficial working practice was then to give some luckless individual a hacksaw and a box of matches. If water came out, then the unofficial procedure was to grab the pipe tightly to keep the leakage to a minimum, and call the water board again. If no water came out and cables were visible, then the unofficial procedure was to stop sawing immediately, and call the electricity board again. If gas came out, the unofficial procedure was to strike a match immediately and light the gas; as long as you were fast, you were okay, because you’d just get a long jet of flame while you waited for the gas board to arrive, rather than an explosion.
That’s perhaps an extreme example. Every organisation has huge numbers of unofficial practices that are easier on the nerves than that one, but that people will still be reluctant to admit to. Many of those procedures are unofficial simply because the official ones are poorly designed. A classic case is when people are forced into unofficial practices so they can work around limitations in the computerised record system, where some software designer has either not known or not cared about how the system should be structured to meet the realities of the client’s needs.
Espoused theories versus theories in use
People usually know when they’re using unofficial working practices; they may not like being put into the position of having to use those practices, but they are aware that they’re using them.
Sometimes, though, people aren’t fully aware of what they’re really doing. There’s a useful distinction, introduced by Argyris, between how an organisation thinks that it behaves (the espoused theory) and how it actually behaves (the theory in use).
A classic example was a timber company where the management genuinely believed that it had a tough, no-nonsense, macho management style. In fact, when researchers studied it via shadowing, the management style involved a lot of informal bonding between management and staff, in contexts like barbecues and drinking together. The reality was that the management were paying a lot more attention to social bonds and communication than the management of most city offices.
Front and back versions
Unofficial practices usually involve people deliberately doing things that they’re not supposed to do; espoused theories often involve people doing things without realising that they do them. There’s another major reason for people not telling you about what actually happens; this one often involves professional pride.
The sociologist Erving Goffman used the analogy of theatre performances to illustrate an important regularity in everyday life. With theatre, there’s the performance that’s shown to the audience, and there’s the very different reality that happens backstage at each performance, with costume changes and scene changes and the fixing of numerous minor last-minute crises.
Goffman pointed out that we see the same in everyday life. For example, doctors and airline pilots present a professional persona to the public – calm, in control, knowing just what they’re doing – regardless of whether they’re actually panicked and without a clue about what to do next.
Goffman called the professional persona the front version, and the behind-the-scenes reality the back version. It’s a very useful distinction.
One interesting thing about back versions is that they’re often very different from what you might expect. A good example is the sociologist who was shadowing a couple of American cops who took a call from a woman who claimed her dog had been abducted by aliens from space. When the cops had finished taking her statement and returned to their car, the sociologist was surprised to see that they then completed all the paperwork as if it was a perfectly normal case. He’d been expecting them to use an unofficial working practice of just ignoring it.
Their decision rationale made complete sense of why the back version here was the same as the front version of how to handle the case. They explained that if they didn’t log the case in the official paperwork and the woman called the station later to ask how the case was going, then they would end up being dragged through procedural audits, which would be unpleasant and very time-consuming. From their point of view, it was much quicker and easier simply to log it like any other case, and then leave it to be someone else’s problem.
As an outsider, you’ll usually be given the front version when you meet people, whether as a designer or a researcher or a consultant or in some other capacity. If you’re aware of this, and aware of how it differs from espoused theories and from unofficial working practices, then you’re at least part-way towards finding out the information that you actually need to know.
The previous sections have contained a lot of discussion about unpleasant realities that we have to understand properly if we want to produce something better. It’s often ambivalent territory.
This article opened with a well-known quote from Conrad’s Heart of Darkness. I’ll close with the less famous ending of the book, which illustrates brilliantly just why people might face deep moral ambivalence about whether to tell the truth or to conceal it. The context is that Marlow, the narrator, has returned from his journey to find Kurtz, and is meeting Kurtz’s fiancée. She is utterly unaware of the horrors that Kurtz saw and the horrors that Kurtz perpetrated. Marlow is torn between the truth, and sparing her needless grief.
“I pulled myself together and spoke slowly.
“‘The last word he pronounced was–your name.’
“I heard a light sigh, and then my heart stood still, stopped dead short by an exulting and terrible cry, by the cry of inconceivable triumph and of unspeakable pain. ‘I knew it–I was sure!’ . . . She knew. She was sure. I heard her weeping; she had hidden her face in her hands. It seemed to me that the house would collapse before I could escape, that the heavens would fall upon my head. But nothing happened. The heavens do not fall for such a trifle. Would they have fallen, I wonder, if I had rendered Kurtz that justice which was his due? Hadn’t he said he wanted only justice? But I couldn’t. I could not tell her. It would have been too dark–too dark altogether. . . .”
Marlow ceased, and sat apart, indistinct and silent, in the pose of a meditating Buddha. Nobody moved for a time. “We have lost the first of the ebb,” said the Director, suddenly. I raised my head. The offing was barred by a black bank of clouds, and the tranquil waterway leading to the uttermost ends of the earth flowed somber under an overcast sky–seemed to lead into the heart of an immense darkness.
Conrad, “Heart of Darkness”
The baboon image is from here: