Finding the right references, part 3: Breadth, depth and the T model

By Gordon Rugg

In the previous article in this series, I looked at ways of getting a mental overview of the key concepts in an area.

In today’s article, I’ll look at how to decide which are the core articles that you need, in a way that should be swift, simple and manageable.

A key concept to hold on to is that everything in academic writing is done for a reason. If you know what the reasons are, then the whole process makes sense. If you don’t, then it looks like a huge, impenetrable swamp.

Academic writing uses references for several reasons. One particularly important reason is to show that the writer is aware of the key issues that need to be considered. Being aware of the key issues is important because it reduces the risk of re-inventing the wheel, or of going down the wrong road because of a mistaken initial assumption.

So how do you show that you know what the key issues are?

Options and choices

One way is to think in terms of what you’re doing as a series of options and choices. For instance, you’ve got an initial set of options as regards what topic to investigate; what are the other topics that you could have looked at instead? Here, you have a range of options, and you choose just one of those options.

Then there’s a choice about the overall framework that you use for your investigation into the topic you choose. In the social sciences, you have options such as case study versus action research versus field experiment versus formal experiment etc; in software development, you have options such as the waterfall methodology versus the spiral model versus various types of prototyping, and so on. Again, you choose just one of the options.

Each time you choose an option, you then need to show that you know the key issues involved in that option, so that you won’t be re-inventing the wheel or going down the wrong road.

This process is clearer if we visualise it in terms of showing breadth of knowledge (with regard to the range of options available) and then depth of knowledge about the option you choose. An important point to note here is that you’re only showing depth of knowledge about the option that you’ve chosen; you don’t need to show the same depth of knowledge about the ones that you haven’t chosen. That makes a big difference to how much you need to know (and, in consequence, to how much you need to read, and to what you need to read).

Here’s a diagram to illustrate this concept.

Across the top of the T are your options – case study versus action research versus field experiment versus formal experiment.

Down the vertical stem of the T are the in-depth details of your choice.

You repeat this process for every choice that you make – breadth and depth for the framework, breadth and depth for the method(s) that you use within that framework, breadth and depth for the analysis method(s) you use, and so on.

How does that translate into references and the literature? Like this.

When you’re writing your coursework or dissertation or whatever it is, when you reach the decision about which research framework to choose, you list the options available to you, and you include a reference for each of those options.

This shows the reader:

• Which options you know about
• That you’ve done some reading about each one

That’s a good start. (I’ll return later to the question of how to choose a suitable reference for each option.)

You can fit all of this into one sentence. It won’t be the most beautiful or readable sentence ever written, but that doesn’t matter. (I’ve blogged here about why academic writing is usually deliberately boring; there are very good reasons for it.)

Here’s what the resulting sentence might look like. I’ve used cliché names like Smith and Jones to reduce the risk of sinful copy-and-paste from this article into coursework…

Possible methodological frameworks for this study include a case study approach (Smith, 1996), action research (Jones, 2001), field experiments (Singh, 1989) and formal experiments (Baum, 1900).

Here’s another version of that sentence which sends out a different acceptable signal to the informed reader:

Possible methodological frameworks for this study include a case study approach (e.g. Smith, 1996), action research (e.g. Jones, 2001), field experiments (e.g. Singh, 1989) and formal experiments (e.g. Baum, 1900).

I’ll return to the different signals after the next section; I’ve mentioned the issue here to reassure any knowledgeable readers that I will be discussing the differences.

What you’ve done with this sentence is to demonstrate your breadth of relevant knowledge, backed up by relevant references, using a handful of relevant references and in a very few words (which is very useful when you’re up against a tight word limit).

Depth of knowledge

The next step is to show your depth of knowledge about your chosen option.

Again, there are reasons that guide the choice of the core set of references to demonstrate this knowledge. The diagram below shows the key principles.

For your in-depth description of your chosen option, there are three different types of core reference that you’ll need to include.

The first is the seminal reference. This is the reference that first introduced the relevant concept to the world, such as the journal article where Zadeh first described fuzzy logic. Sometimes it’s a journal article, sometimes it’s a book, sometimes it’s something else. The seminal reference is often old; for instance, the seminal reference for graph theory is from the 1740s. That’s fine; there’s no problem with the seminal reference being old.

When you’re using a particular method, you should if at all possible read the seminal reference yourself, rather than relying on second-hand descriptions of it. Sometimes that’s not a realistic option (for instance, the seminal reference for graph theory was written in Latin, which most people don’t speak). Sometimes there isn’t a single seminal reference; for instance, some methods have evolved out of undocumented craft skills. If you’re dealing with one of those situations, then you can get brownie points and more marks by explaining the situation, and by explaining what you’ve done about it (e.g. reading a translation, or citing some of the earliest articles where the concept emerged in the formal literature, respectively).

The next is the milestone reference. This is where the original concept was significantly changed in a later publication, usually by being improved in some way. There may be several significant milestone references within a method; if so, you should read and cite them all. Sometimes there aren’t any; if so, you can get extra brownie points and marks by remarking on how little change there has been to the original concept.

The third is the foundational reference. This is the one that you are going to use as your starting point.

A classic example is to take a good recent journal article, and to do a study that extends the work in that article in some way. For instance, the original article might be about working practices in one professional group, and you might apply the same approach to a different professional group, as a comparison with the original article. At a sordidly practical level, this means that you don’t need to re-invent the research design, etc; you simply use exactly the same research design as the original article, but apply it to a different group. You can also justify that research design because it was good enough to get through peer review for the foundational article.

Foundational articles need to be:

• reasonably recent (because otherwise there’s the risk that the field might have moved on since the article was published)
• reasonably reputable (because otherwise you’ll look as disreputable as the article you chose)
• very specific and detailed in their method section, so that you know exactly what they did and how they did it.

You’ll need to repeat this process for each option that you choose in your research design. It’s the same process each time, like a series of “T”s one after the other. If that doesn’t make sense, please let me know via the comments section, and I’ll go into more detail.

How to find what you need as efficiently as possible

There’s a difference between doing something efficiently and doing it well. This section is about finding the right references efficiently. I’ll look at how to do literature searches well in a later article. It’s a big topic, which needs at least one article of its own.

The quick, dirty and efficient way of finding the right references is to find some reasonable quality journal articles on the relevant topic, and to look through their “introduction” sections. Because they’re reasonable quality journal articles, their introductions will cite the seminal articles and milestone articles (otherwise they wouldn’t have got through the review process).

How can you tell which of the articles in those introductions are the ones you want?

Sometimes the introductions will describe the seminal articles as “seminal” – that’s about as broad a hint as you could ask for. Often, they’ll phrase it in terms of  mentioning when the relevant concept was introduced, with a reference at the end of that sentence. Milestone articles are often described in terms of developing the original concept, or extending it.

You’ll soon spot the same references being cited across all or most of the introduction sections, particularly the earliest references. It’s a good bet that most or all of those references will be ones that you’ll need to read and cite too.

Where do you go to find good journal articles on the topic you’re working on?

This where specialist bibliographic databases, containing articles about a particular field, are handy; for instance, a key resource for medical research is the MEDLINE bibliographic database. The Inspec bibliographic database is a major resource for technical and scientific articles; there are similar databases for most disciplines. Your friendly librarians will usually be very pleased to help you with bibliographic databases; they’re usually glad to see students doing it right, rather than just using Google on the Internet.

This is also where specialist search engines come in, such as Google Scholar for most topics, or PubMed for medical topics. These search engines are usually linked to specialist bibliographic databases, and usually offer sophisticated search mechanisms to help users find relevant records more swiftly and efficiently. Again, librarians are usually more than happy to help, if you show an interest in this.

A few minutes with a suitable search engine and/or a suitable bibliographic database should rapidly turn up some suitable journal articles that you can use to identify the “breadth and depth” references that you need; there’s a fair chance that the database where you find those first journal articles will also contain the seminal and milestone articles.

Turning problems to advantage

In this article and in the previous ones in this series I’ve talked about using some references as a means to an end – for instance, using Wikipedia articles to get a quick overview of a topic before getting into the academic literature about it, or using the first journal articles you find to identify the references that you really need.

You might not be able to cite those “means to an end” references as solid evidence, but there are other ways that you can use them in your write-up.

In the case of non-specialist references such as Wikipedia articles, you can often cite what one of those references says, and then contrast it with the more sophisticated version of the same topic that you get from one of the journal articles on the same topic. This can make you look like a sophisticated heavyweight who knows why the non-specialist literature needs to be treated with care.

The “means to an end” journal articles can also be useful, in a different way. They are often useful as illustrative example references. Suppose, for instance, that you’re writing about fuzzy logic, and you used “means to an end” articles by Fletcher about washing machine control and by Bowyer about car engine optimisation. Neither of these topics is directly relevant to your own project on fuzzy logic and central heating systems.

What you can do in this situation is to write something roughly along the following lines.

Fuzzy logic has been successfully applied to various topics, such as washing machine control (Fletcher, 2002) and car engine optimisation (Bowyer, 2005).

This demonstrates that the method you’re using has been successfully used elsewhere (so you’ve got supporting evidence to back up your idea of applying it a real world problem). It also gives the impression that you’ve been doing wider reading beyond the bare-bones minimum of the seminal, milestone and foundational papers.

Closing thoughts

This article has described a quick and dirty, bare-bones, way of identifying the key references that you need. There’s a lot more to know about referencing, evidence, online search, academic writing and related topics.

As a starting point, though, this approach provides a simple framework that helps make sense of what you’re doing with references, and helps you to find the most important ones swiftly and efficiently.

In later articles, I’ll look at the topics above in more depth.

Links:

Some previous articles on this site cover related topics that you might find useful.

For an overview of where references fit into the academic world:

https://hydeandrugg.wordpress.com/2014/10/16/finding-the-right-references-part-1/#more-1339

For references and use of evidence:

https://hydeandrugg.wordpress.com/2014/07/05/logic-evidence-and-evidence-based-approaches/

For use and misuse of evidence and references:

https://hydeandrugg.wordpress.com/2014/06/01/cherry-picking-and-dodgy-reasoning-for-beginners/

 

Advertisement