The quick and dirty approach to meta-analysis

By Gordon Rugg

In an ideal world, everyone would always do everything perfectly. However, it’s not an ideal world.

So what can you do when you’re trying to make sense of a problem where there’s conflicting evidence, and you don’t have time to work through all the relevant information?

One approach is simply to decide what your conclusion is going to be, and then to ignore any evidence that doesn’t fit. This is not terribly moral or advisable.

Another is to do a meta-analysis, to assess the quality of the evidence as a whole. This sounds impressive; it also sounds like hard work, which it is, if you do a full-scale proper meta-analysis. Most academic researchers therefore use two types of meta-analysis.

  • The first is the quick and dirty type, which normally gives you a pretty good idea of whether the topic is worth spending your time on.
  • The second is the proper type, which is time-consuming, and requires sophisticated knowledge of research methods, including statistics.

This article, as the title subtly implies, is about the quick and dirty approach. It’s a flawed, imperfect approach, but it’s a good starting point in a flawed, imperfect world.

Continue reading

The studied subtexts of academic insults

By Gordon Rugg

The academic insult at its best is a highly sophisticated art form with a long, rich history.

A classic example comes from one of my heroes, Thucydides the Athenian, in his History of the Peloponnesian War. That war between Sparta and Athens took place two and a half thousand years ago. He fought in it, and he wrote its history. He was brilliant by anyone’s standards, and the studied impartiality of his writing is remarkable even by the most rigorous modern standards. Here’s what he had to say about what people’s knowledge of contemporary history.

There are many other unfounded ideas current among the rest of the Hellenes, even on matters of contemporary history, which have not been obscured by time. For instance, there is the notion that the Lacedaemonian kings have two votes each, the fact being that they have only one; and that there is a company of Pitane, there being simply no such thing. So little pains do the vulgar take in the investigation of truth, accepting readily the first story that comes to hand.

It looks like a rant from a grumpy old man just before he yells at some kids to get off his lawn. In fact, it’s an elegant, cutting, multi-level take-down that’s on a par with what the best modern academics can offer.

This article is about the serious, constructive subtexts beneath academic insults, and about what those subtexts say about the nature of research.

Long after the war: Scenes from Sparta and Athens today

header pic3

Images from Wikipedia

Continue reading