A simple test of student understanding?

This post may be obvious to some and it is probably something I should have learnt or done long ago. In setting and marking my recent set of exams I have stumbled upon an observation. I think it illustrates a few important points:

• Too many students approach university exams in the same way they have been taught to approach high school ones. They study for "the exam" not study "the subject". This means memorising solutions to problems rather than actually learning to solve the problems.
• Students actually learn and understand less than we want to admit.
• Exams should test actual understanding.

Here is what I found. Take an old exam or assignment question (which students have seen before) and change it slightly in a manner which you might consider to fairly trivial (e.g., use different parameter values, replace a 1/4 filled energy band with a 3/4 filled band, ask them to sketch a simple graph that was not in the original question, make a sub-part simpler, change the order of the question sub-parts).

Then it appears that some students have actually memorised the solution to the question that they have encountered before. They can reproduce (regurgitate?) parts of the question they have seen before but not new parts, even when the new aspects of the question may be trivial. Sometimes they will even write out the solution to part (c) on the old exam, even though on the actual exam part (c) is a different question!

I realise that my observations need to take into account the stress of exam conditions. Sometimes, students may fail read to question carefully and realise it is different. But I still think there is still something here.

I am certainly going to be more diligent about not simply recycling questions. I also need to draw a harder line with students who just want me to tell them an answer to an old question, rather than actually helping them to think through how to solve it. This is a danger of running revision tutorials.