The observational techniques

By Gordon Rugg

Sometimes you can get the information you want via interviews and questionnaires. Often, though, you can’t.

For instance, if you want to know how long the average visitor spends on your website before leaving it, or what percentage of the population don’t have current car tax displayed in their car, or just how a skilled tennis player holds the racquet for a forehand, then interviews and questionnaires won’t get you very far.

In cases like the ones above, you’re trying to find out what people actually do, as opposed to what they say that they do.

There are various techniques that can be used for this. I’ve grouped them together under the label of “observational techniques”.

There are three very pragmatic questions that divide the observational techniques into neat groups.

• Will you be working and/or living among the people that you’re observing?
• Have you told the people involved that you’re observing them?
• Are you observing actions, or observing the consequences of actions?

If you’re an academic researcher, these questions have significant implications for ethical clearance. If you want to study people without having told them that you’re studying them, then this can raise all sorts of ethical issues. In some situations, the ethical issues will be minor; in others, the ethical issues will be big, and will take a lot of careful thought to work through.

The more formal names for the groups arising from the three questions above are:

• Participant observation or not
• Undisclosed observation or not
• Direct observation versus indirect observation

The following sections look at the various observational techniques in turn. This article is a brief overview; we’ll go into more detail about individual techniques in later articles.

Participant observation, whether disclosed and undisclosed

I’ve lumped these categories together, since some combinations of them are particularly common.

If you’re a part-time student holding down a day job, then it’s worth considering using participant observation for your student project. The day job offers the chance to do observation of the realities of the organisation where they work. Given that about 90% of research is conducted on undergraduate students, supervisors are often understandably keen to study the wider world. The results might well be publishable as a co-authored paper, which would boost the CV of the student, as well as building the supervisor’s links with the organisation.

Participant observation is widely used in the social sciences. It’s also used to some extent in software development, in cases where the software is safety-critical (e.g. getting air traffic control systems) since this approach improves the chances of finding out the real requirements.

In sociology a few decades ago, for instance, undisclosed partipant observation was quite commonly used. A classic example was the researcher who joined a Glasgow street gang, to see how they actually spent their time. This approach gave some very useful insights into the realities of life for the groups being studied, but was often ethically ambivalent and also dangerous for the researcher, with a real risk of injury or death if their cover was blown.

In ethnography, there’s a long tradition of disclosed participant observation, where the researcher lives and works among a community, openly studying them. This can still raise ethical problems such as duty of disclosure if the researcher discovers that one of the participants has been engaging in illegal activities, and can be dangerous on occasion if the fieldwork is being carried out in an unsafe location. However, many studies using this approach now take place in everyday western society – for example, studying the behaviour of call centre staff while working among them.

A rough rule of thumb with participant observation is that you hit diminishing returns fairly swiftly after about a couple of weeks. Two weeks is long enough to go through the group’s weekly routine once as a novelty, and then a second time as a habit. After that, some significant new insights will come at semi-predictable intervals, and others will be unpredictable and rare.

The semi-predictable new insights will tend to arise from cycles of time – for instance, the events that happen once a month, or once a term, or once a year. In the academic world, there’s a major cycle lasting three years, which is the duration of an undergraduate course. In some agricultural or fishing communities, there will be longer cycles reflecting weather patterns such as El Niño, or events such as periodic cicada infestations.

The unpredictable and rare events are often the most significant ones in terms of showing where the group’s boundaries are – events such as major crimes, and punishment of those crimes. However, by their very nature, they’re difficult to study directly, which is one reason for indirect observation being so useful. First, though, I’ll describe direct observation.

Direct observation

There are numerous forms of direct observation, which focus on different levels of detail, and are used for different purposes. Here are some examples which give a flavour of what you can do with direct observation.

Shadowing

One of my students shadowed IT managers (with their permission) to see what problems they encountered in their work. She also used other techniques such as interviews, to see which techniques picked up which problems. For the shadowing, she simply followed the managers around all day. It was only when she shadowed them that she discovered that most of the managers would often work through their lunch break, without stopping for a meal.

Observing interactions

Another of my students sat with a coffee in fast food outlets, observing the interactions between the staff and the customers: for instance, noting how often the member of staff smiled to the customer, or said something beyond the scripted questions and responses.

She was interested in identifying the features of the “human touch” that would be lost if the system was automated, but exactly the same approach could be used to compare the customer service approaches of different branches within the same organisation, or of different organisations.

Task analysis

When we were developing the Search Visualizer software, Jo and I did things like counting how many clicks of the mouse it took to carry out various tasks with the software. Our aim was to drive that number as low as possible, so that the software would be as user-friendly as possible.

There are numerous methods for analysing how people carry out tasks. In production line work, “time and motion” studies have been used for a century or more, to identify more efficient ways of getting the job done. In software design, having user-friendly software is a key selling point, so there’s been a lot of research into ways of studying how people use software.

Spotting compiled skills

Compiled skills are skills that a person has used so often that they no longer need to think about what they’re doing. A classic example is an experienced driver changing gear while driving; the action is so habitual that an experienced driver can do it while maintaining a conversation. They’re useful as an indication of experience and expertise.

When Ann Blandford and I were doing a case study on a company, the manager mentioned that he routinely used their production line software. This was something we wanted to check, since it had far-reaching implications for the design of their software interface. A little later, the manager wanted to demonstrate something to us, and he selected an option from the software. He made the selection very swiftly, at compiled skill speed; he could only have done that through routinely using the software, so his actions confirmed his words without his even realising what he had done.

Indirect observation

A lot of the events that researchers most want to study are rarely visible, whether because they’re simply rare, or because they’re normally kept hidden. If you want to study such events, one approach is indirect observation, where you observe something that’s caused by the thing you want to study, as opposed to observing the thing directly.

An example is a study of premarital sexual behaviour in the sixteenth century. Direct observation wasn’t an option to the twentieth-century researchers who conducted that study, but they did have access to sixteenth century records of marriages and of births. They also had access to church records. So, by indirect observation, they were able to conclude that in the community they studied, an unmarried woman of childbearing age was more likely to be sexually active than to be a regular churchgoer.

A quick digression for any sociologists reading this: Yes, there are obvious caveats about such a study, since societal norms change in subtle ways as well as obvious ones. For instance, premarital sex in some communities was accepted within the context of a “trial marriage” to check that the couple were compatible and fertile, but was unacceptable in other contexts within those same communities.

Most indirect observation is a lot more mundane; it often involves activities such as looking at wear patterns on artefacts, which can tell you a surprising amount. It’s a very useful approach to have in your toolbox.

Other approaches

There are various methods that are closely related to the ones mentioned above. Here are a couple of examples, as a taster.

There has been some excellent work using sociotechnical approaches, which look at how artefacts affect our social behaviours and vice versa. A classic example is the layout of a standard lecture theatre, where the seats all point to the front, where there is a screen for showing information. That physical layout works well with a power structure where the lecturer is in charge, and where the students are passive recipients of what the lecturer wants to tell them. It’s a very poor layout for active learning by the students, or for group work.

That example is so familiar that most people have difficulty imagining anything else, but imagine what would happen if someone produced a cheap 3D holographic projector. What would the best layout be for using that as a teaching aid? Probably you’d want to have the hologram in the centre of the room, with the seats arranged around it, but far enough back to let the students walk round the hologram and see it from all sides. Where would the lecturer fit into this new way of learning? It’s an interesting question, and shows how ubiquitous and un-noticed the interactions are between technology and social norms.

A method that fits well with sociotechnical approaches, but can be used independently of them, is STROBE (STRuctured OBservation of the Environment). This is a way of systematically examining a space to make inferences from its layout and contents. It can also be flipped round to help design the layout and contents for a space to make that space work more efficiently. I used that approach with my own office, and it’s made the office more attractive and also more efficient.

Summary

That’s a brief overview of the observational techniques. They’re very useful for dealing with tacit knowledge, and they’re a useful complement to the other techniques that we’ve already covered, both to check on the validity of what you’re finding with the other techniques, and to get at information that those other techniques can’t reach.

We’ll work through some of the techniques above, and others, in later articles.

Advertisement