Logic, evidence, and evidence-based approaches

By Gordon Rugg

So what is “evidence-based” anyway, and why do so many people make such a fuss about it?

In this article, I’ll look at the context of “evidence-based” and at some common misconceptions and mistakes about it.

It’s a journey through the limitations of logic, through the legacy of theology on modern debate, and through the nature of evidence.

It starts with a paradox that took over two thousand years to solve, involving pointy sticks and tortoises.

The arrow of logic and the chain of evidence, plus a tortoise and a charm bracelet0header2Images adapted from Wikipedia and Wikimedia; details at end of article

Continue reading

The quick and dirty approach to meta-analysis

By Gordon Rugg

In an ideal world, everyone would always do everything perfectly. However, it’s not an ideal world.

So what can you do when you’re trying to make sense of a problem where there’s conflicting evidence, and you don’t have time to work through all the relevant information?

One approach is simply to decide what your conclusion is going to be, and then to ignore any evidence that doesn’t fit. This is not terribly moral or advisable.

Another is to do a meta-analysis, to assess the quality of the evidence as a whole. This sounds impressive; it also sounds like hard work, which it is, if you do a full-scale proper meta-analysis. Most academic researchers therefore use two types of meta-analysis.

  • The first is the quick and dirty type, which normally gives you a pretty good idea of whether the topic is worth spending your time on.
  • The second is the proper type, which is time-consuming, and requires sophisticated knowledge of research methods, including statistics.

This article, as the title subtly implies, is about the quick and dirty approach. It’s a flawed, imperfect approach, but it’s a good starting point in a flawed, imperfect world.

Continue reading