Te Tino Uaratanga provides a centre of focus for the education for students in Te Aho Matua kura. This is also the main focus for an ERO review.
In the example below, the kura whānau has chosen aspects of Te Tino Uaratanga as its focus for self review: celebrating success, student success and professional development for staff.
Figure 3: Self review and external evaluation focusing on Te Tino Uaratanga - an example
The kura whānau will then develop an evaluation plan. This will be based on an evaluative question to guide the external evaluation process. The evaluative question is informed by self review. When constructing an evaluative question whānau could begin with the following:
The kura documents its own investigative pathway with its whānau so that they are well informed and can all identify what happens, when, by who, how and the impact on students. Some whānau may select one or more of the wāhanga while others may choose all wāhanga with only certain parts for focus.
Depending on its findings, ERO makes a decision at the end of each review on the type and timing of the next ERO review. This could be a development evaluation carried out over the next one‑to‑two years (Te Pupuketanga). It could be an expansion evaluation after three years (Te Rākeitanga). Or, in the case of an exceptionally high performing kura, ERO could decide to return in four‑to‑five years for an enrichment evaluation (Te Manakotanga).
Kura that are performing well will continue to be reviewed under the Te Rākeitanga methodology.
Factors in the timing decisions are the amount of support needed, the improvements that can be made and the capability of each kura for self review and future action.
When making a decision on the next review, ERO considers the whānau evaluation focus, the information provided in the Whānau Assurance Statement (WHAS), and ERO’s Te Aho Matua criteria for review timing.
ERO’s formal decision on the timing of the next review is set out in the confirmed review report, which is printed and published on ERO’s website, www.ero.govt.nz.
The nature of development from ERO’s Te Pupuketanga (over the course of one-to-two years) review, to Te Rākeitanga (three year) review, through to Te Manakotanga (four-to-five year) review is designed to be progressive and aspirational.
The criteria for the timing of reviews are outlined in this Framework under each type of review.
Figure 4: Evaluation plan *See Appendix 1 for a bigger version to print off and use
The kura whānau gives its evaluation plan to ERO as a part of the first self review presentation hui at the onsite stage of the review. In some instances whānau can provide a draft for ERO to consider before ERO comes to the kura. This plan provides relevant detail for ERO so that review officers can develop their onsite evaluation design, to guide investigations at the kura.
As the external evaluator, ERO will develop the evaluation design.
The evaluation design will include key areas of focus, investigative pathways and questions to prompt the review process and cover the information provided by whānau. In conjunction with whānau, review officers will also create a timetable that includes additional hui, classroom observations and time for team analysis and synthesis. This also allows for ongoing consideration to the review design, ensuring flexibility where necessary.
Figure 5: ERO Evaluation design
Reviewers discuss the design with whānau to ensure that the process of evaluation is transparent and reflects the intentions of whānau self review. Where possible ERO will provide additional guidance with the evaluative question to ensure the review is purposeful.
The kura whānau makes evidence available to ERO during the review and there will be additional opportunities for hui and discussions as required.
ERO uses the evaluation design to focus its investigations. This allows time to be specifically allocated to the ‘thinking’ role of an evaluator. The evaluation framework focuses on high quality analysis to inform clear and concise evaluative judgments.
Throughout Te Rākeitanga there are three distinct review process phases - Whanaungatanga, Whakaaetanga and Whakataunga (see Figure 1). These are the formal stages of the process and involve interactions, communication, investigation, collection and collation, analysis, synthesis, decision‑making, recommendations and reporting.
A balanced approach to evaluation takes into account evidence of both outcomes for students and the processes and philosophies implemented in the kura to achieve these outcomes. ERO looks for this balance of evidence across both outcomes and processes.
Outcome indicators describe the knowledge, skills, attitudes and higher-level competence directly associated with what Te Aho Matua kura kaupapa Māori are trying to achieve for their students. They include evidence of the academic, physical, emotional and spiritual outcomes that are the direct indicators of success.
Process indicators describe the practices, processes and policies of the kura that, when implemented, are intended to achieve the desired results. The presence of process indicators does not, however, necessarily guarantee positive outcomes for students.
Evidence of both types of indicators needs to be considered if the reviewers are to reach genuinely well-informed conclusions. Considering outcomes alone can distort the evaluation by emphasising results without considering whether the results are what could reasonably be expected. A sole focus on process and philosophy can give an impression of success without actual evidence that students are progressing and achieving as expected and desired.
Consider how the potential evaluation focus contributes to student outcomes. What is the magnitude of the effect in terms of:
In general, priority areas for review will be those where there is likely to be a high impact on the achievement of some students, or where high numbers of students are affected.
Figure 6: Matrix to determine evaluation focus
Figure 7: Analytical framework for the evaluation
The analytical framework used by ERO is included to help whānau expand their self‑review practice.
The evaluation design contributes to the content of ERO’s investigations and the content section of its report.
At each stage, ERO considers the answers to the evaluative questions. What is the evidence of effective process? What is the evidence of positive outcomes?
Considering each of the evaluative questions being addressed, how do they rate?
Is this practice excellent/very good, good, adequate, fair, or poor?
Are these outcomes excellent/very good, good, adequate, or poor?
The following key 1 provides a guide to these terms.
Figure 8: Terms of evaluation judgements
The key decision for teams to reach is a judgement about whether or not the quality of the programme, and the outcomes for students, are acceptable as a minimum.