Critically reviewing the literature, the quick and dirty way

By Gordon Rugg

I’ve blogged previously about literature reviews, and about the significant difference between a literature review and a literature report. There are links to relevant previous posts at the end of this article.

Literature reviews are an important part of research. They’re how you find out what’s been tried before, and what happened when those previous approaches were tried. They’re a good way of identifying potential problems that you might encounter in your own research, and a good way of identifying gaps in previous research, which might be your chance to achieve fame and fortune by filling one of those gaps with a brilliant new solution.

This article is about one key aspect of literature reviews, which is that literature reviews involve critical analysis of the key issues. That raises the question of just how you set about starting a critical analysis.

In case you’re wondering whether this really is a big deal, then there’s one very practical consideration that can make a difference to the outcome of your time at university. One of the criteria for getting distinction-level marks on most taught university courses is showing that you’ve done critical independent reading. This is also a major criterion for getting through a PhD viva, so all in all, it’s a big deal.

But just what is a critical review of the literature anyway, as opposed to a non-critical one, and how can you possibly do a critical review of a literature that may include tens of thousands of journal articles and thousands of books? It’s not physically possible to read all of that literature in the three years of a typical undergraduate degree or PhD, let alone a one-year MSc or MA.

This blog article is about one quick and dirty way of making a good start on a critical literature review.

If you’re doing the literature review properly, you’ll be searching the relevant specialist bibliographic databases for your field.

If you’re a fairly typical student, feeling lost and bewildered by discovering this new world of advanced assessment of a huge literature, then it’s quite possible that you’ve never heard of specialist bibliographic databases. I’ll leave those to the side for the time being, and instead use Google Scholar as a familiar, friendly place to start.

Let’s suppose that you’re doing a literature review about software build projects. You know about the waterfall model, where the project team gather the requirements at the start, then plan the project, and then build the software and show the completed product to the client for sign-off. You’ve never heard about any other models, and you don’t know the strengths and weaknesses of the waterfall model.

So, how do you start?

The method I’m describing today involves going onto Google Scholar, and typing in the word critique followed by the phrase “waterfall model” in inverted commas. The inverted commas are important, because otherwise you’ll get a lot of false positives (i.e. records that aren’t relevant, which were found because they include the word “model” somewhere or the word “waterfall” somewhere but which aren’t about the waterfall model).

What you get from that search will look something like this.

A search for critique “waterfall model”critique1

Here’s a close-up of the relevant part.

critique2

This is the first page of results from your search, and already you’ve found articles that describe problems with the waterfall model. If you’re going to do a software build, then you need to know what those problems were, and what the alternative approaches might be. The Sommerville article and the Kang & Levy article both describe widespread criticisms of the waterfall model, and the Somerville article explicitly mentions other, different, software development methods that were likely to avoid those problems.

That’s a pretty good start. You already have two articles that you can read and cite in your literature review, which are both critical of the waterfall model. These articles showing the person reading your literature review that you’re going out and finding relevant critical literature on your own.

If you’re of a cynical nature, you might have noticed that both those articles are from the 1980s, and you might be wondering if that’s a problem.

If you’re of a cynical nature, you might be pleased to learn that it isn’t necessarily a problem. If you’re cynical and lazy, you can simply write in your literature review something along the lines of: “The limitations of the waterfall model were identified as early as the 1980s” and then move swiftly on. If you’re more conscientious (and also more careful about whether those old problems might have been fixed by now) then you’ll be well advised to look for some more modern articles, to see what the situation is now.

What you often find is that there’s a lot of literature about a problem when it’s first identified, and that the literature about it diminishes as consensus is reached in the field. After that point, the topic will usually just be mentioned in passing as a throwaway sentence in the introductions of articles, or it might not be mentioned at all. (For instance, you won’t see many recent physics articles that say much about the problems with the concept of phlogiston, or many recent medical articles that say much about the problems with the “four humours” model of medicine.)

Where do you go next? One sensible next step is to start a list of the other methods that are being mentioned, and to repeat the same type of search, only with the word critique combined with the name of each other method. That will give you a quick and dirty overview of the main issues relating to software build methods. There’s still a lot more work to do before your literature review is finished, but this approach will give you a fast overview of the key points, so that you’re not drowning in a huge sea of detail.

Once you have that overview of the key points, you should know the main arguments in the field being covered, which makes it much easier for you to start critically reviewing the strengths and weaknesses of those arguments in your literature review.

One quick practical point before I close. You might be wondering why I’ve recommended the word critique rather than more familiar plain English terms like criticism. There’s a reason for this.

The word critique tends to be used mainly in the context of high-level academic criticisms, whereas the word criticism is used in a broader range of contexts. This means that a search using critique will usually return a higher proportion of relevant records. However, this doesn’t mean that you should ignore the word criticism. A search using criticism will return a slightly different set of records, some of which will be relevant records that you wouldn’t have found using critique. It’s therefore usually a good idea to do several versions of your search, using words such as critique, criticism and critiques and criticisms in the plural. These may or may not return different results from each other, depending on which search engine you’re using, and which bibliographic database you’re searching. (I never said that doing the whole of the literature review would be easy; I only said that this was a good way of making a quick and dirty start…)

So, that’s one way of approaching a critical literature review. I’ll blog about others in later articles.

Notes and links

You’re welcome to use Hyde & Rugg copyleft images for any non-commercial purpose, including lectures, provided that you state that they’re copyleft Hyde & Rugg.

There’s more about the theory behind this article in my latest book:

Blind Spot, by Gordon Rugg with Joseph D’Agnese

http://www.amazon.co.uk/Blind-Spot-Gordon-Rugg/dp/0062097903

Related articles:

https://hydeandrugg.wordpress.com/2014/07/25/literature-reviews/

https://hydeandrugg.wordpress.com/2014/06/25/the-quick-and-dirty-approach-to-meta-analysis/

https://hydeandrugg.wordpress.com/2014/10/16/finding-the-right-references-part-1/

https://hydeandrugg.wordpress.com/2014/10/20/finding-the-right-references-part-2-getting-a-first-overview/

https://hydeandrugg.wordpress.com/2014/10/30/finding-the-right-references-part-3-breadth-depth-and-the-t-model/

https://hydeandrugg.wordpress.com/2014/03/21/academic-writing-and-fairy-tales/

Overviews of the articles on this blog:

https://hydeandrugg.wordpress.com/2015/01/12/the-knowledge-modelling-book/

https://hydeandrugg.wordpress.com/2014/09/19/150-posts-and-counting/

https://hydeandrugg.wordpress.com/2014/04/28/one-hundred-hyde-rugg-articles-and-the-verifier-framework/

 

 

Advertisements

4 thoughts on “Critically reviewing the literature, the quick and dirty way

  1. Pingback: Getting an overview of the literature via review articles | hyde and rugg

  2. Pingback: Life at uni, revisited | hyde and rugg

  3. Pingback: Making the most of bad bibliographic references | hyde and rugg

  4. Pingback: Life at Uni: Some tips on exam technique | hyde and rugg

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s