A Chemical Orthodoxy

Please note I have written about this in a less technical way for teachwire here

Exam Technique: Is That a Thing?

No.

Well, probably not. Presumably it depends on how you define “exam technique”. In my experience people generally use it to refer to an ability to work out exactly what an exam-style question wants from you and be able to address your answer to that. This ability would generally be thought of as a “transferable skill” or a “domain general” ability; if I am good at answering questions in chemistry, it follows that I should be good at it in biology, history or English.

I’ve written before about why I’m not so keen on such ideas. In short, your ability to perform any cognitive task is in overwhelmingly large part related to your knowledge of that particular domain (or what I call here internal resources). So whilst I might be able to score 100% in a chemistry A Level paper, I would not be able to do the same in an English one, or even tests specifically designed to minimise the effect of prior knowledge. By far the most effective way to perform in those tests is to increase my knowledge of those areas. (As further reading I highly recommend this overview of the 120 year history of this debate and the consensus regarding its outcome.)

So when my students tell me that “they didn’t read the question properly” or that they “struggle to apply their knowledge in exams” or that they need to “improve their exam technique” I tell them outright that they are kidding themselves. If they dropped a mark in a test, 9 times out of 10 it is because they just didn’t know the answer well enough; they had not practised retrieving it enough times, or as Tim van der Zee put it last night on Twitter:

“The crux of educational research: To learn how to do a thing, you must repeatedly attempt to do the thing. Transfer is mostly a myth.”

In this post, I want to explore two examples which could prove illustrative. I know that there are people who read this blog that aren’t science specialists so I will try and keep it light on the technical information (1). All examples are taken from A Level chemistry. I am also wary of the teachers-playing-cognitive-scientist problem and this post is partly a plea for input from those more knowledgeable than myself.

Problem 1: Determination of structure

This question is about NMR Spectroscopy, which is a way to figure out how the atoms in a chemical are arranged. A scientist might know that a chemical has 6 carbon atoms, 14 hydrogen atoms and 1 oxygen atom, but they cannot know how in what order these atoms are joined up until they have done various experimental procedures, one of which is NMR.

My year 13s just sat a mock where a data reading from an NMR finding was presented to them. They were told that the formula of the molecule under discussion was C6H14O, so it would have had 6 carbon atoms, 14 hydrogen atoms and 1 oxygen atom. The question asked them to use the NMR (and some other information) to determine its structure (in what order the atoms were joined up). All that for 8 marks.

The answers were generally poor and confused. A lot of students had no clear method to slowly work through the problem, and were trying to look at all the data at once and formulate a solution in one go, rather than working slowly and methodically from the information provided (2).

For me, the most interesting answers came from about half the students whose final answers contained different atoms to the ones that they were explicitly told in the question the final answer would have. Students’ answers had more than 1 oxygen, fewer than 14 hydrogens, and, in some cases, atoms of nitrogen too.

So how did this happen? Did they not read the question?

Yes and no. I think that they did read the question. But between the question and the data they are provided with there is a mammoth amount of information to process. Each part of the NMR data has a number of sub-parts and it makes sense that some things that students had read simply fell out of their heads due to the burden of the cognitive load.

In response to this, the feedback with my students went as follows:

1. I pointed out to them their error
2. I pointed out to them that almost all NMR questions are of this format, and the fact that they couldn’t adequately address the format was a function of their lack of practice in advance (I had provided them with a large pack of NMR questions + answers), not because they “hadn’t read the question”
3. We discussed that if they are trying to work something out they should do that on scrap paper and then package and transfer the outcome to the exam script
4. We looked at methods to slow down the process of analysing the data, going piece by piece rather than all at once

Another interesting example is the “name and outline a mechanism” type question, where a number of students outline the mechanism (a chemical drawing) really well, but do not name it. Again, I think that this is probably because they are not adequately prepared in terms of their knowledge retrieval. If they have to think about all the different things that make up a good mechanism, then the part of the question which says “name” will simply disappear from their working memories. Ironically, one student has the habit of underlining key points in exam questions (not my idea). In this question he had underlined “outline a mechanism” but not “name”. He didn’t name the mechanism and dropped the mark.

2 Benzene

This one is a bit technical, and it’s really where my plea for help starts.

Benzene is a molecule with three double bonds:

A double bond is represented by the double lines and have four electrons in the bond, rather than the two that a single bond would have. In benzene, these bonds all “overlap” with each other and make the molecule very stable (3).

The exam question contained the following image:

The question asked students to predict which one is more stable. The correct answer is the one on the right because, just like with benzene, its double bonds are close enough to overlap and therefore increase stability. The one on the left has the same number of double bonds but they are too far away from each other to overlap.

I had not taught students this question. I had not taught them about “overlap” in terms of proximity. With one exception, all the students got it wrong. They all suggested that the one on the left would be more stable as in the diagram on the right the double bonds are close to each other which means there are a lot of electrons very close to each other, something which they know from other areas can lead to instability.

So here we have a problem of transfer (for a primer see here), where students take one piece of knowledge from a familiar domain and apply it to something a little bit further away from that domain. All the students who got this question wrong had, elsewhere in the paper, shown a very good understanding of stability in benzene. But they simply could not transfer one aspect of that understanding to a new molecule. In fact, their answer to this question flatly contradicted the understanding of benzene they had shown elsewhere in the paper!

The student who got it right had never seen this question before. Contra Tim above, he had never practised it before. He had received more or less the same instruction as everyone else. So why is it that he managed to achieve transfer? Why did he manage to avoid the incorrect reasoning which everyone else followed?

I do not know the answer to these questions. It simply lies beyond my current knowledge of how the brain works.But this is our bread and butter – how do we go about preparing our students for questions like this? No doubt if I had have emphasised this aspect, or done questions like this one, more students would have got it right. But the question of transfer still nags. Why this student and none of the others?

Tim would say to ignore the word “transfer” and use “overlap” instead, where are the areas of overlap between the two questions? I think that’s a really interesting distinction, but I don’t feel closer to the answer.

As teachers the message is clear: transfer is hard. Really hard. But teaching students “exam technique” and allowing them to excuse their lack of knowledge due to it is not the way forward.

(1) Ironically, as I have no doubt that the greater a person’s knowledge of A Level chemistry the easier this post will be to digest.

(2) the cognitive processes involved in Rummikub are my favourite analogy for this kind of approach to problem solving.

(3) it’s a bit more complicated than this