**An idea for Maths Exams**

We’ve had plenty change within Scottish education
recently. In this blog I’m not advocating for more change, merely
airing something I’ve been thinking about, as an idea for ‘next time’.

This is a thought experiment to start a conversation. I'd enjoy hearing what you make of the ideas I will discuss.

In this blog I’ll consider 3 questions:

1.
What is mathematics?

2.
What does a mathematics qualification suggest
about a pupil’s competences?

3.
What am I proposing we do for assessment?

**What is mathematics?**

There
is no universally agreed definition of mathematics. Aristotle
described the subject as ‘

*the science of quantity.’*This and other definitions which focus on magnitude and counting fail to account for much of the content subject. Many areas of mathematics, of course, bear no obvious relation to measurement or the physical world.
There
are formalist definitions such as ‘

*Mathematics is the manipulation of the meaningless symbols of a first-order language according to explicit, syntactical rules’*(Snapper, 1979). However, formalist definitions seem to make the symbols and notation the object of study. These definitions ignore both the physical and mental meaning of mathematics.
Henri
PoincarĂ© moved beyond simplistic ideas of number and symbols stating that

*‘Mathematics is the art of giving the same name to different things.’*This quote hints at one of the essential ideas in our subject: the act of generalisation. Taking the specific, spotting patterns and relationships and extracting an abstract generalisation that is independent of the specific. Karl Freidrick Gauss took the perspective that mathematics was about ideas, famously arguing that ‘*What we need are notions, not notations*’.
A
universal definition of the subject is unlikely to be agreed upon. There is a tension which arises from the subject being viewed both as an art and science, which is perhaps responsible
for this. In Scottish education mathematics is often labelled as numeracy and
considered to be a life skill. In his
famous piece “A mathematicians lament” Paul Lockhart (2002), takes a stance
against this sort of utilitarian
perspective.

*‘It would be bad enough if the culture were merely ignorant of mathematics, but what is far worse is that people actually think they do know what math is about— and are apparently under the gross misconception that mathematics is somehow useful to society! This is already a huge difference between mathematics and the other arts. Mathematics is viewed by the culture as some sort of tool for science and technology. Everyone knows that poetry and music are for pure enjoyment and for uplifting and ennobling the human spirit (hence their virtual elimination from the public school curriculum) but no, math is important.’*

I
am of the opinion that somebody who studies the subject is learning:

·
Procedural fluency

·
Conceptual Understanding

·
Mathematical Behaviours

The
subject can be viewed as a domain with many facts, theorems, definitions, rules
and procedures to master. This perhaps relates to what Skemp (1976) would
call Instrumental Understanding. I
refer to this as Procedural Fluency.

Skemp
also discusses Relational Understanding which is about the ideas underpinning
all of these instrumental understandings. It is about the network of
connections and relationships within and between each of these understandings.
It is about the principles that govern these relationships. I refer to this as Conceptual Understanding.

However, while Skemp’s ideas of Relational and Instrumental Understanding
are necessary to capture what learning mathematics involves, I do not feel that they capture it fully. The third type of “understanding” is about the
actions and thoughts of a mathematician. To paraphrase Mark McCourt “Being
a mathematician is a way of existing in the universe.” I refer to this as Mathematical
Behaviours.

Cuco,
A., Goldenber, E. P., & Mark, J. (1995) lay out what they call Mathematical
habits of mind. They suggest that pupils should be:

·
Pattern sniffers

·
Experimenters

·
Describers

·
Tinkerers

·
Inventors

·
Visualisers

·
Conjecturers

·
Guessers

These
are the sort of behaviours of somebody who is in a state of ‘free play’ with
some piece of mathematics. I am of the opinion that a mathematics
education should result in pupils who have developed a range of mathematical
behaviours, in addition to a robust Instrumental and Relational Understanding. The
tasks we offer pupils shouldn’t be solely focused on the destination. The
journey to completion and the actions that pupils take to get there is just as
important. This goes beyond the models used, or the strategies invoked.
It is about developing a language which captures the mathematical
actions, such as those listed. It is about developing an awareness, in pupils, of these actions as
being available. A pupil who is behaving mathematically will
demonstrate these behaviours spontaneously, even when not explicitly asked to
do so by the teacher or through some prompt in the task.

Anne
Watson (2015) captures some mathematical behaviours in the table above.
This is a useful point of reference.
A question one might consider when teaching is: in addition to developing procedural fluency,
and conceptual understanding how often do we plan to develop these specific
mathematical behaviours?

Learning
to use and apply simple mathematical procedures and ideas in everyday life is,
and should be, a basic requirement of any education. For some, learning
the subject is about setting the groundwork for later study in other fields
such as physics, engineering or computer science. For these sorts of
reasons, mathematics is a useful subject. But usefulness is not what
drives or has driven most pure mathematicians through the ages. The enquiring,
conjecturing minds pursuing pure thought for its own sake, has led to most of
the great works in the subject’s history.

I
offer the following conjecture: many of our pupils spend 12 years
“studying” mathematics while rarely ever doing mathematics.

There
would be some who argue through the lens of the knowledge versus skills
dichotomy. I think this is academic. In stating Mathematical Behaviours separately
from Procedures and Concepts I hope to draw attention to them and make them explicit.

**What does a mathematics qualification suggest about a pupil’s competences?**

I
would hope that for a pupil achieving say, National 5, that they had a grasp of
a certain level of content. I’d hope
that a candidate who passes would have fluency with some fundamental processes
and understanding of relationships underpinning those. We operate in a comprehensive system and, as
such, at least a grade C pass *should* be within the grasp of the majority
during their schooling.

One
concern is that there is very little genuine problem solving in our
examinations. The last question in the
paper can generally be relied upon to be something previously unseen, but most
questions are textbook questions with slightly different contexts. When the examiners do include genuine problem
solving the pupils don’t score well.
However, this makes a lot of sense as we shall now discuss. Going back a few years to when we had Credit
level with separate Knowledge & Understanding and Reasoning grades, the
threshold percentages for Reasoning were much lower. Also, over time, the Reasoning questions
become more routine.

I
suggest pupils don’t do well on non-routine problem solving in exams for two reasons:

1.
Maturation, which I’ll discuss in the
next section. Essentially it is hard to problem solve with mathematics which you've only recently learned.

2.
That teaching is driven by past exams, which dictates that the focus is on the problems that routinely appear.

“The tasks presented by high stakes
examinations and textbooks largely determine the types to ask that are used
within classrooms.” Burkhardt & Swan (2013). It is often the case that being
able to do questions which resemble past examination questions is viewed as the
“goal state”. This can then lead to an emphasis, in teaching, on
procedures at the expense of concepts, behaviours and problem solving. If
the exam system does not value mathematical behaviours or conceptual
understanding as highly as procedures (all of which essential in learning
mathematics, as opposed to learning how to pass exams) then it is little wonder
that they can be neglected.

As an aside, I suggest, perhaps controversially, that teaching pupils to pass exams is a real
problem. The popular booklet Nix the Trix lists a whole range of “tricks”
which pupils are sometimes taught without any understanding - purely with a
focus on passing exams. While these tricks may help pupils pass the
current level, they are seldom a sound basis for further study and often create
problems for future learning. The inclusion of questions which test
conceptual understanding or require problem solving rooted can encourage
teachers to ensure pupils are mathematically proficient.

While,
as individuals, we may not have the power to change assessments at a national
level we can certainly influence the design of internal assessments in our own
schools. I firmly believe in the mantra “if you teach the mathematics and
build a mathematician, the exams take care of themselves”.

**What do I propose for assessment?**

I’m in favour of the retention of exams. I’ve yet to be convinced that coursework is
meaningfully valid or reliable as an assessment model in mathematics. In the reality of busy schools, I think that
there are serious consequences that would need to be carefully considered. Although not a coursework model, the assessment approach
for National 4 hasn’t filled many teachers with confidence.

Recently, all of our national exams were increased in
length. They are all rather long and
were extended to take account of the removal of UASPs even though many/most
schools still use them. Rather than
assessing more content in the same manner, I’d have liked a restructuring.

In advocating a retention of exams, I’d suggest we have
three sections:

1.
Concept focused ‘multi-choice’ hinge questions

2.
A range of questions of the same ilk as just
now, getting increasingly challenging through the paper.

3.
A ‘mathematical thinking’ section.

__Concept Focused Multi Choice__

Research seems to suggest that well written conceptual
questions in exams can help us to increase focus on conceptual teaching. I like multiple choice as they are short and sharp and can be precise tools in ascertaining understanding. I wouldn’t want to have 20 questions like in
the old Higher. Instead I'd rather have no more than 10 of them,
which are focused precisely on concepts and relationships. I want well written questions where deep
understanding is needed. In old Higher
the multi-choice test included a lot of basic procedural work. I’m not advocating this.

Our
exams are primarily focused on procedures, although a certain element of
conceptual understanding is clearly required to answer many of the questions. It is hard to create assessment items which
focus on concepts only, without invoking some element of procedural
fluency. However, we’ve had questions recently
which make a fair attempt at assessing concepts, including:

Cos60 =
0.5, what is Cos240?

Having the concept focused multi choice questions, I
suggest, would hopefully positively influence the range of tasks pupils were
exposed to in school. We need a
succession of layered experiences, other than drill exercises, to develop this
conceptual understanding. The tail does
wag the dog. A key reason, in my
experience, that people were uncomfortable with CfE was that there was no
indication of the what the end point would be.
The endpoint assessment, more than anything else dictates the
curriculum. As such, let’s make the
endpoint assessment different so that it makes pupil experience of the
curriculum is richer. I suggest that
this would be much more effective than spuriously including words such as
“creativity” in the course documents.

__A range of questions like just now__

The exams we have just now are useful and include a good
range of questions from basic application through to ‘working backwards’ and
‘application of two ideas’ questions. I
would suggest that the bulk of the exam continues to be focused on this range
of questions. There are certain
mathematical ideas and processes we might expect pupils to have fluency with,
for instance finding the equation of a straight line. In this section of the exam there would be
one or two genuine, unseen, problem solving questions which require National 5
maths. These would likely be considered
‘A’ questions.

__A Mathematical Thinking Section__

Previously I suggested that maturation was an important
reason for pupils performing poorly in problem solving. Essentially, problem solving using National 5
mathematics is hard for pupils who have just been learning National 5
maths. This mathematics hasn’t totally
become assimilated into the pupil’s schema.
It begins to make more sense with continued study of mathematics.

We often saw this idea of maturation come to light in
Higher. Everyone used to teach straight
line at the start of the year. Most pupils
didn’t find this trivial. They would
make mistakes, like using points which don’t lie on the line when finding equations of
medians etc. However, come the end of
the year almost every passing Higher candidate could do pretty much any
straight-line question – even those that were non-standard. The reason is that the ideas in straight line had had a
period of maturation. They might make
calculation errors, but generally the pupils ideas of what to do would be correct.

On the contrary, the topics taught later in the course would be done
badly if assessed in the exam in any sort of non-routine way. This would have been logs and exponentials,
for instance. Nevertheless, six months
later, during Advanced Higher those same pupils would routinely invoke the
solving of exponential equations with relative ease in all sorts of situations. So maturation matters.

Colin Foster writing in MT (Feb 2019) captures the phenomena we see
with the genuine National 5 problem solving questions.

*“it is perfectly possible to have all of the necessary techniques safely inside your toolbox and yet not see how they could help you solve the problem you are tackling. The teacher feels frustrated, because they think that the students ought to be able to solve the problem. They apparently know everything they need to know, but they do not mobilise it in the particular situation they are presented with.”*

This is something that I’m sure many teachers can relate to. He continues:

*“One reason for this may be that the students have met the relevant content only in a narrow range of contexts and have not seen how it might be applied more widely. Another reason may simply be that they have encountered the relevant content too recently. When learning a language, students do not spontaneously and fluently use the vocabulary they have just learned. It needs time to bed in. Similarly, if we want students to make sophisticated use of what they know, it might be better to rely on mathematical content that was learned some time ago and is quite robustly known. Content learned 2 years previously is a rough rule sometimes used at the Shell Centre in Nottingham. … It also acknowledges that if the problem-solving demands are high, other demands, such as procedures and concepts, may need to be lower.”*

With this in mind, if we value genuine problem solving and want to see
pupils demonstrate this, then we might need to pose questions in this third
section of the exam which are at a level below.

What might these questions look like?
Well, the UKMT Junior questions might be somewhere to get inspiration
from. They use maths which is

**well****established**for National 5 pupils. However, they require pupils to reason with these ideas on a sophisticated level and apply problem solving strategies. The two questions below would be quite difficult for an S2 pupil, but provide just the right amount of challenge for a National 5 candidate to apply their previously learned material in a problem solving context.
While it might seem strange to include content which is not at the
level being assessed, this is not a new idea.
Many of the marks awarded at National 5 are not for National 5 skills.
Further, in the 1990s Higher Maths papers frequently featured questions which
were Advanced Higher level! Pupils would
be given an example and then asked to complete a problem based upon their reading
of the example.

As we know the pupils have the knowledge to do this, the focus for
preparation for this part of the exam may focus around strategy. A framework such as the one laid out in
Thinking Mathematically by Mason, Burton and Stacey might well be a useful
point of reference.

The final question in the paper, would be something quite
radical. Rather than being a problem to
be solved, it would be a prompt for the pupil to explore. The marking of this might be based upon
comparative judgement, where value is put on the evidence of mathematisation
rather than on technical competency.
What is valued here is not the “answer” but the pupils mathematical
creativity, tinkering, conjecturing, visualising and experimenting.

Potential questions might be in the style of the ATM's Points of Departure. Two examples are shown below.

What we value here is the mathematical behaviour. While these tasks lend themselves better to a longer protracted engagement, allowing say 20 or so minutes to 'play' with one prompt could still be valuable in allowing pupils to demonstrate their mathematical attributes beyond "what they know and understand".

**Conclusion**

I've suggested that learning maths is about procedures, concepts and behaviours. Our exam system values procedures and, to a lesser extent, concepts. It places little stock in mathematical behaviours, despite much of the curriculum rhetoric being focused on these 'softer' skills.

The exam system does influence the enacted curriculum and the associated teaching focus. If our exams changed to reflect my idea, or some other idea, then the teaching and enacted curriculum would also change.

The idea of an exam I have laid out is far from perfect. I have no idea if it would work, but it tries to bridge the gap between a dry "regurgitate this" style exam to one where pupils have a realistic chance of showcasing their problem solving skills.

While I've focused on National 5, similar arguments could be made for Higher and Advanced Higher, where pupils problem solve and 'tinker' with ideas at the level below, for some of the marks in their exam.