By Gordon Rugg
Note: This article is a slightly edited version of an article originally posted on our Search Visualiser blog on May 17, 2012. I’ve updated it to address recent claims about how Artificial Intelligence might revolutionise research.
So what is pattern matching, and why should anyone care about it?
First picture: Two individuals who don’t care about pattern matching (Pom’s the mainly white one, and Tiddles is the mainly black one (names have been changed to protect the innocent…)
Pattern matching is important because it’s at the heart of the digital revolution. Google made its fortune largely from the simplest form of pattern matching. Computers can’t manage the more complex forms of pattern matching yet, but humans can handle them easily. A major goal in computer science research is finding a way for computers to handle those more complex forms of pattern matching. A major challenge in information management is figuring out how to split a task between what the computer does and what the human does.
So, there are good reasons for knowing about pattern matching, and for trying to get a better understanding of it.
As for what pattern matching is: The phrase is used to refer to several concepts which look similar enough to cause confusion, but which are actually very different from each other, and which have very different implications.
We’ll start with a couple of pictures and three questions about them, to illustrate the core concepts involved in the different meanings of pattern matching.
Second picture: Unconcern about pattern matching
Question 1: Are the first and second pictures identical copies of the same image?
It’s not a trick question: the answer is obvious to us humans, and also easy for a computer to answer. Those photos aren’t identical; they’re different from each other.
That sounds pretty trivial, until you try a different pair of images. Ignoring minor issues of background shade, are these two copies of the same photo of a chess pawn, or are they photos of two different pawns from the same set?
Third picture: Just in case you thought nothing could care less than Tiddles and Pom…
That’s a much harder question for a human to answer, but an easy one for a computer to answer: they’re copies of the same photo. So why does the change in image make it so much harder for a human to answer, when the underlying question is the same? That’s what we’ll be looking at in this article.
The second and third questions involve identifying a couple of other issues that we need to keep clearly distinct.
Question 2: What type of animals are shown in the first two photos?
That’s an easy one for a human to answer, but difficult for a computer to answer; the animals are cats.
This type of pattern matching involves identifying the category to which an item belongs – in this case, the category of “cat”.
The third question brings in a different type of pattern matching.
Question 3: Tiddles and Pom are sisters from the same litter. Two of their sisters have very similar markings to them. Does Picture 2 feature Tiddles and Pom, or does it show two different cats from the ones in picture 1?
That’s a hard question for a human to answer, and a very hard question for a computer to answer. It involves identifying specific individuals, as opposed to identifying the category to which they belong. These are two very different processes.
So, just how can you, or a computer, tell whether two images are the same as each other or not?
One way of answering this question is by string matching.
A string, in this context, is a string of symbols. A sentence can be viewed as a string of symbols (in this case, letters); a number can be viewed as a string of digits. And a picture can be viewed as a string of coloured rectangles. Here’s how that works.
Picture 4: Cubist house on island
Picture 5: Cubist house on island
Question: Are Picture 4 and Picture 5 identical or not?
We can answer that question by comparing the images line by line, just the same way that we could compare two lines of text by putting one above each other. That reduces the problem to a much more manageable size.
Here’s what the top line of squares looks like in the two images, with the top line for Picture 4 above the top line for Picture 5.
Here, in contrast, is what happens when you put the lower window row from Picture 4 about the corresponding row from Picture 5.
If you look closely at the blueish window squares representing the window on the right (three squares in from the end) you realise that they’re actually different shades of blue. (Depending on your monitor, you might need to look at the image from different angles to make the difference apparent.) The upper set of squares has two window squares that are identical in colour to each other; the lower set doesn’t. Here are the two different squares from Picture 4 and Picture 5 without any distracting background.
The human eye is very easily misled by the image background, and overwhelmed by volume of detail, but a computer isn’t; it just plods methodically along, comparing one pair of squares at a time, unaffected by background or boredom.
The photos of the chess pawn above can be compared to each other using just the same process as we used to compare the pictures of the two houses. A computer treats a photo as a set of rectangles just like the ones in pictures above, only with much tinier rectangles, and with a lot more of them – tens or hundreds of thousands for a medium-size image, and millions for a large or high-detail image.
So that’s how you can compare two images to see if they’re identical, using string matching. If you’re a copyright lawyer trying to show that someone has pirated an image from your website, or a fraud investigator checking whether a document is not what it should be, that approach can be invaluable. For everyday life, though, it’s not usually a huge issue for most of us.
However, you can use exactly the same process to compare two sets of text, and that method is at the heart of Google and a huge amount of software around the world. Paradoxically, explaining how that works is more complicated than explaining how to compare two images.
The bottom line is that software is very good at doing some types of comparison, and at doing types of search based on those types of comparison. However, software is very bad at doing some other types of comparison and corresponding types of search.
Various forms of Artificial Intelligence have attempted to tackle the “are these two different pictures of the same cat?” problem and of the “do these two phrasings describe the same concept?” problem. There’s been a lot of recent hype about this, and about how these approaches could revolutionise research, under names such as “deep learning” and the more familiar “machine learning”. I’m sceptical, because of the factors above. Software probably will find some radically new and important links between findings whose relationships had previously been missed. However, it will also probable find a large number of false positives. Just what the ratio of new insights to false positives will be is an interesting question…