Methodology

ERO visited schools involved in Year 9 Plus in Terms 1 and 2, 2016, and again in Term 4, 2016. Appendix 1 includes a list of the both nominating and participating schools and when ERO visited them for this evaluation.

Evaluation framework for Terms 1 and 2

ERO's evaluative question for Terms 1 and 2, 2016 was:

  • How well has the Year 9 Plus concept been understood and implemented by schools to the end of May 2016?

To investigate this question ERO used four investigative themes when visiting schools, with two lines of inquiry used to analyse data under each theme. These are shown in Figure 1.

Figure 1: Investigative questions and lines of inquiry

Investigative theme Lines of inquiry
Student nomination and selection

How did participating schools find the Year 9 Plus nominating and selecting processes?
What criteria did they find helpful for their need to identify the most 'at risk' students exiting Year 8?

The role of the champion What roles have champions played so far in supporting the learning of Year 9 Plus students?
How well have schools used the champion as an added resource to support at risk learners?
Transition from Year 8 to Year 9 How well were the selected students transitioned from Year 8 to Year 9?
What factors contributed to this being more or less successful?
Student engagement in learning How well have Year 9 Plus students been engaged in Year 9 up to the middle of 2016?
What factors have contributed to or lessened the chances of positive early engagement?

 

ERO teams gathered data in April-June 2016, as shown in Figure 2:

Figure 2: Data gathering

Visit 1 (2-6 April)

Participants met by ERO:

Interviews with all nominating school tumuaki/principals, senior staff, key teachers, Year 9 Plus students.

Discussions centred on their knowledge of, and expectations for, the Year 9 Plus initiative and the basis on which they nominated students to participate in the initiative.

Visit 2 (23-26 May)

Participants met by ERO:

Interviews with each participating tumaki/principal and/or teacher in charge of Year 9 Plus.

Discussions centred on school expectations for the Year 9 Plus trial, the champion's role and students' transition and initial engagement in Year 9.

Visit 3 (31 May - 3 June)

Participants met by ERO:

Interviews with selected Year 9 Plus students - this included up to three students from each participating school.

We gathered broad findings about their personal/previous educational backgrounds and their perception of the effectiveness of support provided so far by the champion and the school.

 

Evaluation framework for Term 4

ERO's two evaluative questions for Term 4, 2016 were:

  • How well are Year 9 Plus participants (students, champions, parents/whānau and participating schools) working together to frame and support initial student success over 2016?
  • What factors are contributing most strongly to this success, and what challenges have arisen over the year which if not addressed might constrain future success?

We used the following investigative questions and lines of inquiry:[2]

Figure 3: Investigative questions and lines of inquiry

Investigative theme Lines of inquiry

Student Outcomes:
How well are Year 9 Plus students placed to
succeed in schooling in 2017 and beyond, and
what factors have helped or constrained this in
the first year of the trial?

What did the Year 9 Plus students themselves see as
success in Year 9, and what did they see as helping them
with this?
How strong was student agency and sense of
engagement in different schools overall in 2016, and
what factors supported this?
School Contribution:
How well are participating schools responding
to the learning needs of their Year 9 Plus
students in 2016, and what factors helped or
constrained this in the first year of the trial?
How well have participating schools contributed to Year 9
Plus students’ success in 2016?
What were the factors that helped success in 2016 where
school contribution was significant; and what factors
restricted success where contribution was more
constrained?

Champion Contribution:
How valuable have the Year 9 Plus champions
been as a resource for change to students and
whānau and in participating schools in 2016,
and what factors have helped, or constrained,
this value?

How did the Year 9 Plus champions create productive
relationships with students, whānau and schools?
What approaches have worked well for them over 2016,
and what have their key challenges been? What might
help in the future with the latter?

 

ERO teams gathered data in November 2016, as shown in Figure 4:

Figure 4: Data gathering

Visit 1 (7-11 November)

Participants met by ERO:

Meetings with tumuaki/principals, senior staff,
key teachers, Year 9 Plus students.

A meeting with Lead Champion and
Programme Coordinator, Ministry of Education
Visit 2 (14-18 November)

Participants
met by ERO:

Meetings with key teachers and Year 9 Plus student
groups. A meeting with the champions as a team.



 

 

 

 

 


2 Conceptual frameworks developed to guide the evaluation are included as Appendix 2 and 3.