Questioning Questions

I thought I might jot down a few of my thoughts having attended the Cambridge Assessment conference: Questioning Questions.

Types of Questions

There was quite a lot of focus was on formative and summative questioning. Victoria Crisp from Cambridge Assessment spoke about designing summative assessments. She shared some of the considerations that need to be made in order that the question is actually testing what it's supposed to be testing. She gave a lovely example where a question was supposed to be checking whether students could identify that Zug was German for train. There were images to choose from and the most common wrong answer was a picture of a tram. Was that because they didn't know the translation or was it because the picture wasn't clear and looked a bit like a train?

There were also thoughts shared on questions used for formative assessment, In general the consensus seemed to be that although everyone knows that it is very important, in general it is implemented very badly. I had been listening to Craig Barton's interview with Dylan Wiliam on the way in, which was very interesting and coincidently a few of the speakers, including Michael O'Sullivan and Simon Peyton Jones, referred to the project that Craig Barton has been working on with his diagnostic questions. I really like the idea of a multiple choice question where each answer suggests a different common misconception. At the conference there were some great examples shown, however I do have a few concerns that I will save for another blog post.

I personally am most interested in questions that stimulate thinking. Lucy Crehan has studied teaching in a variety of countries and talked specifically about China, Japan and Finland as examples. She had noted that China was not as didactic as we might imagine and students are often invited to reflect on their answers and justify their ideas. She also shares that in Japan, teachers collaborate on writing questions. This reminded me of a couple of blog posts that I really enjoyed by Ed Southall, about observing maths lessons in Japan: part 1 and part 2. I think part 2 indicates more the role of the teacher's questioning in developing the students thinking.

Technology

It was noted that we are not yet at a point where most exams are completed on computers. A few exceptions that I have had some experience with include the MYP eAssessments, and the numeracy professional skills tests, neither without issues. However, technology can support teachers and learning if used effectively.


Dr Evelina Galaczi, Head of Research Strategy at Cambridge Assessment English, took a refreshingly measured approach to ed tech, reminding us of things that teachers can do and technology can't, for example promoting collaboration, managing discussion, and creating a positive learning environment, as well as things that technology can do much better and at a greater scale than teachers, for example data management and immediate feedback. I have been dubious of many learning platforms that include some kind of feedback in the past, the ones I have seen have been incredibly limited in terms of feedback and scope and, as Evelina pointed out, they atomise and narrow the content that is covered. However, as technology is moving forward, I am excited to see what Evelina and her colleagues are developing.

Purposeful Practice

I was really intrigued to hear what Daisy Christodoulou had to say about reclaiming formative assessment. I agree with her that there is a problem in England with the obsession with slapping a grade on every piece of work done and trying to measure progress on finer and finer gradients on a scale. She made an analogy to compare writing an essay to running a marathon. If you were training for a marathon you wouldn't run 26 miles every week so why would you write an essay every week if you were practicing for that? You would focus on improving different skills and note the progress in each of those skills. I wonder how this works in a maths classroom though, as I am not sure whether many students can identify what this marathon is that they are training for.

Daisy also noted that the oft quoted 10 000 hours of practice required to master something usually misses out that it has to be purposeful practice and not just 10 000 hours of repeatedly doing the same thing. This made me consider the focus in maths at the moment on times tables. Clearly they are a fundamental in maths and being fluent in recalling them is helpful but it is just one aspect. Maybe when we just focus on times tables we are just practicing the sprint finish?

I thought back to Colin Fosters thoughts on Mathematical Etudes. I think these are perfect examples of purposeful practice in a maths classroom that also remind us of the bigger picture.

Assessing Creativity

Bill Lucas was the last speaker of the day and I was really fascinated to hear about his project Learn to be an Engineer. He is working on ways of teaching and assessing Habits of Mind. In his work he references Cuoco, Goldenberg and Mark, a favourite of mine. He humorously noted that a focus on developing these traits is not a 21st century idea as you can see from this book cover from 1898 about traits of character.

I will be interested to see whether the work Bill Lucas is doing for the OECD and PISA will influence how the government views the importance of these mathematical habits of mind. If so then I imagine there will be a knock on effect on what happens in classrooms and perhaps teachers may feel more empowered to spend time developing these skills.

Conclusion

The day has given me lots to think about and these are only my first thoughts. I know that I will definitely come back to the idea of diagnostic questions and reflect on their use and I am also going to reflect on how purposeful practice is planned and implemented in mathematics classrooms.

Finally I wanted to add my thoughts about a question that was presented. Apparently this question would be great to see whether children really got Pythagoras' theorem. I had some doubts:

I have rewritten the question in a way that I think captures a few more of the issues:

And finally, I really likes these excellent quotes about the need to be able to ask good questions: