ResearchED: session 7

(I was late for session 6 so I missed it out of blogging)

Amanda Spielman, HMCI, taking us through the different ways that Ofsted uses research to improve its practice.

What should be the purposes of research at Ofsted?

Giving a bird’s eye view of what works in schools, based on evidence from all the inspections. Informing Government about how well.olicies are being implemented and whether they are having an impact. Building evidence to improve inspection processes and practices. Assessing the impact inspections have on improving schools. 

So, are Ofsted fit for purpose? How do they use research to test the validity of their judgements? Amanda says that one of the nice things about her job is that almost everybody has an opinion on how to do it better. She says to keep the blogs coming, by the way, as she does find them very interesting. She works through some suggestions that have been made for improving judgement quality, such as parallel inspections – expensive and difficult to do. It would be very tricky to check for data bias, for example; and the free schools inspected with no data can be compared with similar schools that had data, to provide information on the existence of such bias. 

Lesson observations are a valuable tool but should consist of an aggregation of micro lesson observations. She repeats the message that schools should not grade individual  a lessons and says that this sort of thing has her tearing her hair out.  Book scrutiny is similar. 

What about the bird’s eye view? School poplns are often too small to understand properly the performance of special interest groups – disadvantaged, SEN etc. so aggregating the data provides a better view of this. Ofsted have been looking at how the new NC has been introduced and what non-NC schools are doing. She has a strong desire to ensure that the curriculum is fit for purpose as this has a bigger impact on social mobility than the school itself does. She explains why she is a fan of the new GCSEs: a big step in the right direction, she feels, as are new A-levels.

Understanding the impact of Ofsted. Making reports useful for parents; how grading structures affect school behaviour – both of these are currently being researched. (Note: behaviour OF schools, not IN schools).

How are Ofsted developing their next framework? (Don’t worry, not due until at least 2019). Partly by drawing together research from lots of different sources to ensure the best narrative and causal explanations. 

Advertisements
Posted in CPD Notes, researched2017 | 1 Comment

ResearchED: session 5

Leigh Ingham on young women, school and confidence: the problems and solutions, in their words (a late addition to the program). This session is a report on the impact of programs run by a group called Fearless Futures. 

We begin with a depressing list of statistics about the gender gap, from Rachel of Fearless Futures. Fearless Futures is aimed at tackling these inequalities as well as a wide variety of others, acknowledging intersectionality. They work with schools to try to do this. Leigh, giving the presentation, came in as an external researcher to look at the quality of Fearless Futures’ work.

Leigh’s talk:

The participants all said that participating in the program helped to improve their academic self esteem. They were almost all BME; a majority went on to study STEM subjects at high performing universities. They talked through episodes of discrimination they had experienced and also how to tackle discrimination, which helped them to be confident enough to aim high. 

It also helped their personal self esteem, particularly through the ‘I am special because…’ activity. Participants said that it helped them to recognise their qualities and what they could achieve.

The importance of all-girls sessions. It provides a safe space that made it possible for them to be very open and avoid being shut down. It created trust among the members, allowing them to feel supported and solidarity. One, who didn’t speak in school before the program, went on to successfully lobby for more diversity in the A-level Music spec, speaking on Channel 4. 

It gave the girls the ability to critically analyse the world and recognise/tackle inequalities, whilst giving them the confidence and tools to change society. 

Fearless Futures are running a conference in November called the Gender Assembly. Sounds like it would be worth a look. 

Posted in CPD Notes, researched2017 | Leave a comment

ResearchED: session 4

Dr Niki Kaiser on threshold concepts. 

Niki is a chemistry teacher and network research lead. She relates a lesson on ionic bonding where she identified a misconception she was unaware of and how happy the subsequent student lightbulb moment made her. 

Niki explains that a threshold concept is described as a portal to transformed learning. You move through it to greater understanding and this is irreversible, making accessible previously out of reach concepts and ideas. To model this, Niki breaks down the different concepts students need to understand ionic bonding. She says Meyer and Land (2003) differentiate between core and threshold concepts and explains her identified bottleneck in understanding ionic bonding. It is tricky for us as experts to put ourselves in the shoes of novices and identify the trouble spots. 

There isn’t a great deal of research on threshold concepts, but Twitter suggested looking at the misconceptions literature and extrapolating from that. Planning a unit on structure and bonding, Niki used a tracker to help her to identify student trouble spots. 

Niki then spoke about her own research, modelling that there are many different ways of going about researching. I lost the thread a bit here (sorry Niki. Sorry readers)

Having identified tricky spots in the learning, Niki revisited them, a la spaced practice, and used Google forms to gather confidence scores from her pupils, to see which ideas they were finding most difficult. She compared the confidence scores with marks from tests on different topics within the course, and this data combined to helpfully show which the most troublesome areas were. 

I like threshold concepts (you might remember) and this was a really good session about how to research among your classes to identify new ones. This is much more scientific than my method, ‘Thinking about it on public transport’. It is also interesting to think about threshold concepts, not just in general historical understanding, but in individual topics. 

Niki’s ppt – http://bit.ly/rED17NikiKaiserLinks

Posted in CPD Notes, researched2017 | Leave a comment

ResearchEd: session 3

Daisy Christodoulou on improving assessmen: the key to education reform. I’m clearly going heavy on the assessment today. Delighted to get my first seat of the conference: it’s either very busy or I’m picking the most popular sessions. There are a lot of people sitting on floors, in the gallery, on the steps…

Daisy says, it’s a big claim, but ultimately, not just in education but in a whole range of fields, better measurement leads to improvements in innovations, while bad measurement leads to distortion and unintended consequences. Here are four assessment practices that are being influenced by theory that isn’t quite right. 

One. Using prose descriptors to grade work.

These are so entrenched that people think this is assessment. It can be difficult to think outside of this structure. But it is important, because they are neither accurate nor helpful. They don’t give an accurate summative judgment. Daisy demonstrates this with a fraction problem that also appears in her book, showing that ‘can identify which fraction is larger’ is not as simple as it sounds: questions have different difficulties and even small changes have an impact on this. Prose descriptors are not precise enough. 

Are they still a useful structure, eg for formative feedback? Daisy thinks no. They are not helpful for that either. Written comments become generic. They can be accurate, but not helpful. 

Daisy talks about well-designed multiple choice questions that are designed to tease out common misconceptions. Unambiguously wrong, but plausible distractors. This is a better way of giving feedback. 

Took this picture too early: she also recommends Michael Polanyi.

Two. Marking essays using absolute judgment.

Daisy uses the same crimes activity she includes in her book, from Mozer et al: poisoning a barking dog appears on two lists, once with lesser crimes and one with greater crimes, and people asked to score them on a scale of 1-10. 

The alternative to absolute judgment is comparative judgment. This will provide a system for this using an algorithm created by teachers rating pairs of essays. Technology is used to enhance human judgment. Daisy mentions that No More Marking will do this. I missed her summary slide. 

Three. Thinking of grades as discrete categories.

Someone at the top of a grade will be closer to someone at the bottom of the next grade than they are to someone else at the bottom of their own grade, but this is not reflected in the letter on the page. Daniel Koretz has attacked this in ‘Measuring Up’. 

Four. Thinking test scores matter. 

In fact, it is the inferences we can make from then that matter. What can we say from the evidence about what the student can do in other contexts? (Wiliam). Test scores are only samples of a wider domain. When the link between the test and the domain breaks, the inferences made are no longer valid. Any exam is only as good as its link to its domain, but this has become distorted over time, in large part due to the high stakes attached to the grades. ‘When a measure becomes a target, it ceases to become a good measure’: Goodhart’s Law. 

This lecture was a good overview of Daisy’s book, ‘Making Good Progress?’ so I can recommend reading that if you liked this. 

Posted in CPD Notes, researched2017 | 1 Comment

ResearchED: session 2 

Tricia Taylor on improving peer assessment (nice dovetailing with the previous season). 

About 80% of verbal feedback comes from peers and much of it is wrong (Hattie). Feedback is highly variable, but highly effective. Being able to take feedback is important for a growth mindset. It should be about the task, not the person (ie, not self). 

Tricia shares some of the Austin’s butterfly video, Berger’s case study of using peer feedback to improve a student’s work. She also shares his advice for students giving feedback and reminds us, hard on content but soft on the person. 

It’s important to create the right culture for this. Students need to be able to take feedback and be eager to learn from their mistakes. They need to have good formative feedback modelled and be given a chance to practise, eg with the teacher doing the task first and then asking for feedback from the students to improve it. At the end of this training they need to reflect on it: how they gave feedback and how they were able to take it. “I learnt to be patient and keep trying.”

Student feedback is improving. Students value having several chances and they can see the improvements. Feedback was appreciated and rarely personal, or taken personally. It really helped teachers to get better at success criteria: more specific language when setting them. 

Staff introduced prompt cards to help shape how students five feedback. “I like the way you….have you considered….?” This was less successful with older students (Y8) where it felt a little contrived, but when they used them to work with a reception class they understood the point and it worked really well. 

Next stage: apply to literacy. An example: an English teacher working through a piece of writing might pull out good examples from it to share with students, but students find it more difficult to recognise this. Hence: rubrics. Tricia shares a rubric for applause to help us to improve clapping her: advice on volume and tempo, for example. A rubric helps students to recognise what is good and give advice for improvement. 

Students focused on spag when they give feedback, and teachers are also guilty of this: it is easy and accessible. I often find that when I’m marking a long essay, the spag corrections come thick and fast at the end when I have lost sight of the content of the essay, so I try to mark once for spag and once for task. Anyway…they made the rubric more specific to try to tackle this problem. 

Tricia used gallery critique to give students opportunities to comment on several pieces of work. 

She summarises her learning on making peer assessment work. If kids know there’s an audience at the end, it motivates them. Improving work reinforces the idea that change for the better is possible. Students don’t have to act on all feedback but can choose what they want. 

Posted in CPD Notes, researched2017 | Leave a comment

ResearchED: session 1

Nik Booth on formative assessment. 

What is formative assessment?

It’s using judgment about quality of student work that is used to shape student competencies: moving learning forward, but not only the teacher’s job (Sadler). Nik talks us through some other definitions, since there is no single agreed one. The key thing is that is has to be used to improve teaching/learning. It doesn’t involve giving out marks, levels or grades (though marks can be formative for the teacher, who can then plan based on who scored what). It doesn’t compare students. It just focuses on what should happen next in the learning process. It is responsive teaching, and responsive learning. 

Why hasn’t it worked in schools? Probably because people don’t completely understand what should happen. Confusion between formative and summative. 

Nik shares a 1988 study which showed that comments had an impact of 30% improvement in achievement where grades alone had none. This same study showed that comments and grades also had no impact, because students ignore the comments and instead focus on ranking themselves against other students. They are more interested in the consequences of the grade, says Kohn (1994), rather than making improvements. 

How can we make formative assessment better in our schools? 

Learning intentions are important and can be powerful, but they can be tricky to write. Shirley Clarke (2005) says it probably shouldn’t include context of task, as this makes it less transferable. So, “To write a letter” rather than “To write a letter to the local council”. But also – it should be the learning, not the activity.  The above then becomes, “To be able to write persuasively” for example. 

Success criteria. Product and process, says Clarke, but process is the more important. Students need to know how to move forward and this should be explicitly clear. 

Hinge questioning helps provide quality feedback to shape the next stage in the learning. Using multiple choice questions where the other choices tackle common misconceptions (the 3 truths and a lie quizzes I wrote for the old GCSE often did this) – just one in the middle of the lesson, with a response from every student, eg using flashcards, is enough. Nik talks us through some examples from other subjects. 

Feedback – common on task, process, self-regulation and self. Whatever feedback you give must cause thinking. Peer, self – can be powerful because students become insiders in the learning process. Problem: it is often done at the end of the lesson, making it less useful. It should happen within the lesson if we want to grow self-regulated learners. 

Posted in CPD Notes, researched2017 | Leave a comment

#newyeargoals

The very vivid and very full back to school nightmare last night has prompted me to spend much of today working, which has been something of a guilty pleasure. My school has a slightly earlier summer finish so I have now already been off for six weeks; it’s not surprising that my brain is a little confused.

I’ve alternated school books with personal reads this summer and that, combined with an excellent Sutton Trust/Durham University conference last week on independent learning, has prompted me to make some plans. Here they are in various states of development:

Assessing Sixth Form

The lecture on transition issues at Durham, led by the enthusiastic Dr Jacquie Robson, made it clear that independent research leading to better understanding as opposed to a shorter-term high grade was something that first year undergrads struggle with. This chimed in with a conversation I had with a history lecturer back in July, who said in passing that they spend about 18 months unpicking school conditioning in their undergrads; and with a conversation I had with a social policy lecturer years ago who expressed incredulity that her students were using her past papers to inform their revision (“Why?! They can’t do those past papers because we never covered those topics, but those topics are not going to come up because I write the paper!”). It also fell neatly alongside Christodoulou’s comments in Making Good Progress? about not playing a whole baseball match to practise hitting one type of pitch (I paraphrase, in the manner of a butcher) and the session I attended at the WLFS conference about using narrative. I had a nice tumblers-falling-into-place epiphany halfway through the last Durham lecture, which was inconvenient because I really struggled to focus on it.

So, my intention with year 12 next year is to have them research two small projects alongside their studies – one on each Henry, which will work well with the Tudors course. I will attempt to run these as small-scale NEAs. I think what I’m going to do is provide a bit of scaffolding and maybe even topics/questions for H7, but less for H8. Each project will require at least five readings. Their performance will not receive an individual grade, but will go towards their end of y12/UCAS predicted grade. This hopefully will, in no particular order:

  • Make them do some wider reading, thus improving their knowledge, thus leading to a better grade at the end
  • Give them a better understanding of how to use a library
  • Prepare them for the NEA
  • Provide me with some insight into how well they understand the course
  • Provide me with significant body of work that I can include in my UCAS references
  • Remind them that they really love History (or indicate to them in a timely fashion that, well, maybe they don’t)

On top of that, I’m going to require an ongoing narrative account of the Tudors, written in the style described by Vartan Tamizian at WLFS. This is mainly to help me see that they understand the basic facts of the course.

I’m still batting this about. I am toying with the idea of also requiring the narrative from Y13 but making the projects optional, since they will be doing their real NEA. I think I might have to wind back from my rule of at least one exam question a fortnight for Y12 in order to give them the space to complete this, but I think that is OK, especially with the new linear course. I think I’m gutsy enough to be able to defend against the inevitable outcry at doing work that isn’t marked to an exam board scheme. I do need to discuss it with my A-level co-teacher, though.

GCSE

I’ve got two year 10 classes next year. I’m going to teach one Medicine thematically and one chronologically, to see if it makes a difference in their understanding. I would also like to assess each one differently, to see if that helps their progress, once again inspired by Christodoulou, but it doesn’t seem wise to do both of these things at once. Maybe I will do that for the later year 10 unit.

Meanwhile year 11 will benefit from my speed-reading of Willingham’s Why don’t children like school? – I’ve planned something for them on chunking and automating processes that will hopefully be useful for their revision and may lead to an adjustment in how I teach them to remainder of the course. There wasn’t much in Willingham that was news to me but chapter 5 was a real stand-out. My notes on chapters 1-4 are 2-3 lines per chapter. Chapter 5 runs to 6 pages, including sketches of lesson slides.

KS3

I continue to plug away at thematic units for KS3. I have three that work reasonably well and I am hoping to trial two more this year.

CPD

I am aiming to attend 15 CPD things across the year. I have tickets to ResearchED in London and TLT, for term 1, and there will be pizza group again. There were also some great local insets run last year by Bristol Museums so hopefully there will be something similar I can get involved in. Also I need to update my First Aid – that counts, right?!

Other

I’m hoping for a PGCE student after Christmas. There will be more exam board work, which only becomes more interesting. I’m working on another book, with a March deadline. A few more trips would be nice – little local ones seem so achievable.

I’m interested to hear what other people are hoping to do: please share.

Posted in Assessment, Planning | Leave a comment