Appendix 1: Methodology

Data collection took place as part of scheduled education reviews.

The project’s overarching evaluation question asked:

What does the school know about Pacific student achievement since the last ERO review?

The six key evaluation questions supported this:

Since the last ERO review -

  • to what extent has Pacific student presence improved?
  • to what extent has Pacific student engagement improved?
  • to what extent has the board of trustees’ knowledge and understanding of Pacific issues improved?
  • to what extent has the school’s engagement with its Pacific community improved?
  • to what extent has student achievement in literacy and numeracy improved?
  • to what extent has the quality of achievement information improved?

Evidence was triangulated through discussions and interviews with a range of school personnel, as well as from observations and relevant documentation.

Based on evidence gathered, reviewers formed overall judgements about the progress made in each of these six areas. They also identified and reported examples of good practice.

Analysis included the identification of a group of high performing schools and a group of schools that were least effective in their provision of Pacific students. ERO’s judgements on aspects of the six areas were assigned a numeric value (highest quality level given highest number) and these numbers were added together to give an overall total score. With a possible grade aggregate of 63, 12 schools that scored 50 or more were identified as the most effective, and qualitative data on these schools was examined to develop broader judgements about effective practice.

Schools whose aggregate was 14 or less were identified as the least effective.

Evaluation Services evaluators collated and analysed reviewers’ synthesis sheets to identify and report on national trends and patterns