Methodology

ERO’s evaluation focus

The Achievement 2013-2017 initiative is a flexible programme involving Ministry staff and schools working together to identify areas where they can improve student achievement. Typically Ministry personnel work with schools to discuss how NCEA Level 2 achievement can be improved. Schools are asked to focus on a cohort of students who would be unlikely to gain NCEA Level 2 without further assistance. The Ministry personnel do not prescribe particular solutions for schools. In most cases schools have at least some initiatives that are already focused on improving achievement.

While the intention of Achievement 2013-2017 is to have advisors working with schools from the beginning of each academic year, in 2013 the specialist advisors were not appointed until mid-year. As a result most advisors did not start their work with schools until well into Term 3 (or Term 4 in some cases).

A key focus for ERO’s evaluation of Achievement 2013-2017 in 2013 has been to inform the Ministry’s ongoing development of this initiative. In this regard ERO has had ongoing discussions with the Ministry about its findings and these have been used to inform developments during 2014. The focus for ERO’s evaluation was:

  • the specific actions each school took to identify and respond to their target cohort
  • the impacts achieved for students, including changes in their achievement, attendance and retention
  • what was learnt by the school
  • changes that were sustained in the school’s operations in 2014
  • practices and/or processes used by the Ministry staff that were successful for supporting the school to identify and respond to their target cohort.

Sample selection

During Term 1 2014, teams of ERO reviewers visited 30 of the 129 schools that had been part of Achievement 2013-2017. ERO selected a range of schools to be part of this evaluation to get a cross-section of schools. The schools visited as part of this project ranged from decile 1 to 9. They were located in Auckland, Waikato, Rotorua, Taranaki, Manawatu, Wellington, Nelson, Marlborough, Christchurch and Invercargill. [1] During each visit ERO:

  • spoke with the principal and other leaders with oversight of this work
  • interviewed groups of students involved in the initiative
  • met with staff who have continued this work in 2014
  • discussed with a few family or whānau members any benefits they had observed for their teenager
  • examined documentation, including any self-review material, the school had prepared related to this initiative
  • considered any other evidence or examples of ongoing good practice, benefits or developments linked to this work.