Think aloud technique: A brief introduction

By Gordon Rugg

This is one in a series of tutorials about reports. “Reports” in this sense involve respondents reporting what they’re doing or thinking, or what someone else is doing or thinking. We’ll post a separate overview of the different varieties of report.

The topic of this article is think-aloud technique, which involves getting the respondent to think aloud while performing a task.

This is particularly useful if you’re trying to find out what’s in the respondent’s mind while they’re doing a task, and for finding out about people’s reactions to a product or a behaviour that they witness. For instance, you can show people mockups of software or of a product you’re designing, and get at information which would be missed by interviews and questionnaires.

The table below shows how think-aloud relates to similar methods.

Time	Technique(s)	Main knowledge issues involved
Future	Scenarios	Knowing the unknowable
Present	Think-aloud	Short Term Memory/working memory
Past	Critical Incident Technique Hard Case Technique	Active and passive biases in human memory

What it’s good for

Think-aloud is about the only technique that lets you find out what’s in someone’s Short Term Memory/working memory at a given point. This is particularly useful for finding out about:

• People’s immediate reactions to a product, situation, etc
• What information people are using when performing a task (e.g. what information they need to have visible on a computer screen

This information only stays in Short Term Memory for a few seconds, and is then usually lost completely, so it can’t be captured in interviews or questionnaires.

Materials

You need to give the respondent whatever it is that you want them to talk about. This may be a mockup of a web page or a product, or it may involve asking the respondent to carry out a particular task.

If you’re showing something to the respondent, it’s advisable to make sure that they can’t see it until you’re ready to start recording, otherwise you’ll miss their immediate reaction, which is often the most important information.

Procedures

It’s a good idea to give the respondent a brief demonstration of thinking aloud, using a topic that’s unrelated to what they’ll be discussing, so that you don’t prime them with material that they can feed back to you in their own think-aloud. For instance, if you’re asking them to comment on a web page design, you might do the demonstration by thinking aloud about the design of a pencil.

It’s advisable to audio record the session, since even a shorthand expert would have serious trouble keeping up with the exact words that the respondent is using. There are advantages in video recording, but for some reason, many respondents find this much more obtrusive than audio recording.

If the respondent is silent for more than a predetermined time (e.g. three seconds) you should prompt them with a non-directive probe, such as “Could you tell me what you’re thinking about now?” An unobtrusive way of keeping track of the silence is to count silently in your head: “One second, two seconds, three seconds”.

Hints and Tips

It’s important to clarify the viewpoint from which you want the respondents to answer. You can get very different responses from the same person depending on which viewpoint they’re using.

People vary widely in how much they say, and how fluent they are.

You need to resist the temptation to be helpful when the respondent is struggling to put things into words; otherwise, you might as well just write down your own opinions about what they should have answered, and not bother gathering any information from human respondents in the first place. It’s often difficult to resist the temptation, but it’s essential. Making supportive and encouraging noises sends out positive signals to the respondent without steering them too far in any particular direction.

What you get

The output from this technique is natural language text. When you listen to the recordings, they often appear ungrammatical and disjointed, which can be initially disconcerting.

It’s a good idea to transcribe each session, which makes it easier to analyse. One minute of recording will take about five minutes of transcription, depending on how talkative the respondent is, how clear the audio is, and how fast your typing is. When you’re transcribing:

• Never try to reword the transcript so it’s more grammatical – your task is to record what they did say, not what you think they should have said.
• Never remove swearwords etc – they’re often the most important information, showing how strongly the respondent feels about something.
• Check the spelling of any technical terms – for example “buccal smear” rather than “buckle smear”.
• Show silences using the convention of one full stop for each second of silence. The silences can be important, because they show the points where the respondent is having to stop and think.

It’s a good idea to transcribe the sessions yourself. Even the best professional transcribers will have trouble with the technical terms, and a surprising number of them will edit out swearwords and tidy up the grammar.

Here’s an example of a reasonably coherent think-aloud transcript, from a session about reactions to the label on a wine bottle, from the viewpoint of “Choosing a good value wine to go with a meal”.

“My first impression is that it’s quite a good wine, a.. quite expensive, that’s from the label, because it has, em, quite an old fashioned, em, hand-drawing illustration. It’s got shiny raised print for the name, and there’s quite a lot of gold on the label, but then when I look at the top, it’s got a metal screw top and, em, it’s been a bit bashed about, so I’m not sure it’s as good as I thought it was at first…. It’s obviously a red wine, by the colour of the bottle, because it’s dark…. It contains sulfites.. So I would try it but I wouldn’t have a very high expectation of it.”

[Note the full stops indicating pauses, at one second per full stop – these are not places where text has been cut.]

There are various ways of analysing think-aloud results. Ones that we often use are as follows.

Qualitative and quantitative analysis of the features mentioned

• Which features do the respondents mention?
• Which features do the respondents mention first?
• How often do different respondents use the same wording as each other?
• How many of the features are new and unexpected?
• How many of the features involve subjective versus objective features?
• How many of the features involve intrinsic versus extrinsic features?
• How many features do respondents use – few, or many?
• How much do respondents say – much, or little?
• How many respondents mention the same features?

This shows you:

• Which features are important to respondents: useful for e.g. marketing campaigns
• Which features the experts are spotting which the non-experts don’t spot: useful for e.g. risk assessment

Limitations

Because this technique elicits a flow of consciousness, the resulting information often doesn’t have explicit statements of cause and effect. For instance, in the example above, there is the statement: “…because it’s dark…. It contains sulphites.. So I would try it but I wouldn’t have a very high expectation of it.” It’s not clear how, or whether, the sulphites relate to the dark colour, or to the respondent’s willingness to try the wine, or to the respondent’s expectations of the wine.

Complementary techniques

Laddering can be invaluable for unpacking any subjective terms or technical terms that emerge during the session.

Laddering can also be invaluable for clarifying the respondent’s goals and values relative to features that are mentioned during the session – for instance, why someone prefers one option rather than another.

Interview-style questions can be useful for mopping up any questions about cause and effect, such as in the example above – you could ask: “You mentioned that it contains sulphites, and then said that you would try it but you wouldn’t have a very high expectation of it. Could you tell me whether that’s cause and effect, or whether those are unrelated points?”

Card sorts can be useful as a follow-up method, if you decide that you need to do a more systematic investigation of the topic. The cards could be similar items to the ones you used as the topics for the think-aloud sessions, or each card could have a term from the think-aloud written on it, such as a card for “contains sulphites”.

Notes

There are introductory tutorials about card sorts and laddering on the Hyde and Rugg blog site and web site.

General notes

Where it’s easy to locate the literature about a topic, we show it in bold italic, and don’t usually give specific references.

Where specific references are needed, and/or where a particular literature is not easy to locate, we usually cite references.

This tutorial is copyleft Hyde & Rugg; you’re welcome to use it for any non-commercial purpose, including academic lectures, provided that you include the “copyleft Hyde & Rugg” with it.

Advertisement