Why are we being examined on this?

By Gordon Rugg

It’s a fair question, if it’s being asked as a question, rather than as a complaint about the cosmic unfairness of having to study a topic that you don’t see the point of. Sometimes, it’s easy to answer. For instance, if someone wants to be a doctor, then checking their knowledge of medicine is a pretty good idea.

Other times, though, the answer takes you into deep waters that you’d really rather not get into, especially if there’s a chance of some student recording your answer and posting it onto social media…

Why do some answers take you into deep waters? That’s the topic of this article. It takes us into history, politics, proxies, and the glass bead game.

bannerv3

Political issues

As history teachers know all too well, history is inextricably intertwined with politics. Often, there’s one version of events phrased in terms of peaceful expansion of our glorious culture and another version of the same events phrased in terms of brutal aggression and subjugation by the invaders. Which version goes into the syllabus? That’s often a political decision, rather than a decision made by historians or educators. The “teach the controversy” option is also usually political, as a way of claiming legitimacy for versions of events which don’t bear much relation to the evidence. Similarly, the “the truth lies somewhere in the middle” approach is an invitation for opposing camps to stake out increasingly extreme views, to drag “the middle” in their direction.

This issue is well recognised. Its converse, the significant absence of a topic, is also well recognised. Often, topics don’t feature in a history syllabus because society would prefer not to talk about those topics.

It’s easy to see how this principle applies to history. It also applies to other topics, though not always as obviously. In English literature, there’s a whole set of value judgments about which authors should or should not feature in the syllabus. In English language, there are similar value judgments about which dialect of English is being treated as the norm. In maths, there are decisions to make about the balance between pure and applied maths, which tie in with the expectations about the types of work for which the education system is preparing the students.

Which leads on to a different issue in education, namely proxies for future achievement.

Proxies, predictors and the glass bead game

Sometimes, students are assessed in terms of something which is a proxy and/or a predictor. A classic example is SAT (Scholastic Aptitude Test) scores. The original idea behind these was that they could be used to predict how well a student would do in higher stages of the education system. So far, so good. Assessing someone’s aptitude for a particular vocation or topic has obvious and real uses.

Assessment of aptitude, however, isn’t always easy or straightforward. Often, there are very real problems in trying to measure the relevant skills directly. For instance, if you are an airline trying to assess how well potential pilots would respond to a mid-air emergency, you probably wouldn’t want to put each candidate into a real aircraft and then generate a real mid-air emergency and see what happened. Instead, you would probably use proxies, i.e. you would assess their performance in relation to things that are reasonable stand-in substitutes for that situation.

The trouble with proxies is that they can all too easily take on a life of their own. I’ll use shoe size as an example which pushes the principle to its logical, scary conclusion.

There’s a solid literature showing that height in humans correlates with social status and with salary. Leaving aside the issue of how fair that is, we can push this correlation a stage further. Since foot size correlates with height, we can predict that foot size at a given age will be a predictor of salary. Foot size, in turn correlates with shoe size. Future salary isn’t something that a school can measure directly, but shoe size is something that can easily be measured. So far, so good, more or less. So, let’s suppose that the education system decides to use students’ shoe sizes as a proxy for their future salary levels, and as a performance indicator for schools. What happens next?

What will happen next, with tragi-comic inevitability, is that schools will be strongly tempted to provide their students with bigger shoes, to improve their performance in the league tables which include shoe sizes as a measure. The bigger shoes won’t in themselves help the students’ achievement, and will probably lead to a lot of accidents and general misery. However, the schools that go down this route will do better in the league tables, and thereby get more resources from the education system, which will ironically probably help the long-suffering students to achieve more.

This particular example is intended as a satirical demonstration of a principle (though there’s the uneasy feeling that someone in central government might view it as a brilliant, cost-effective new approach to education performance indicators…)

In other cases, though, the suitability of a proxy is harder to assess. In principle, a proxy should usually be a dependent variable, where there’s only a one-way relationship between the proxy and the thing that it’s standing in for. In principle, if you change the value of the proxy, that shouldn’t have any effect on the value of the thing that it’s standing in for. Conversely, however, if you change the value of the thing that it’s standing in for, then this should change the value of the proxy. So, for instance, greyness in adult hair is a proxy for age. Dyeing someone’s hair much more grey won’t make them much more old; however, becoming much more old will usually lead to much more grey hair.

In practice, however, clear one-way causation is rare, because so many things are causally linked with each other. Finding good proxies is difficult, which produces an inherent system pressure towards using less good proxies, and towards behaviours which are geared towards those proxies, rather than the things that the proxies are supposed to be standing in for.

Which takes us to the glass bead game. In the Hermann Hesse novel of that name, the game is used in the selection and training of intellectuals. Hesse’s vision of the game is that it is based on identifying deep structure connections that help the intellectuals achieve a more powerful understanding of the world.

That’s a plausible starting point. What, though, if the game was more like Go, which is very far removed from any practical understanding of the world? If we further imagine that skill in this abstract version of the glass bead game is an excellent predictor of ability in government and administration, then performance in the game would have huge implications for an individual’s future.

Where would this lead? Because of the systems pressures explored above, schools would start teaching the glass bead game. This would create a demand for experts in the glass bead game to teach the teachers, and for courses in the glass bead game. This in turn would create a niche for university and college departments of glass bead game studies, and for conferences and journals. All of this would happen for perfectly understandable reasons, despite the initial key point that the game we’re postulating is inherently useless, with no direct applicability to the real world.

So, returning to the question that started this article, sometimes students end up being examined in something which is actually useless in its own right, for reasons which can vary between solidly sensible and cynical manipulation of the system.

As for which particular topics fall into this category of “being examined for understandable reasons although it’s actually useless”: That’s an issue outside the scope of this article…

Links and notes

There’s more about the theory behind this article in my latest book: Blind Spot, by Gordon Rugg with Joseph D’Agnese

You might also find our website useful for an overview of our work.

Overviews of the articles on this blog:

https://hydeandrugg.wordpress.com/2015/01/12/the-knowledge-modelling-book/

https://hydeandrugg.wordpress.com/2014/09/19/150-posts-and-counting/

https://hydeandrugg.wordpress.com/2014/04/28/one-hundred-hyde-rugg-articles-and-the-verifier-framework/

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s