ERO is updating its approach to review and evaluation, initially in centre-based early childhood services. Ngā Ara Whai Hua: Quality Framework for Evaluation and Improvement in Centre-based Early Childhood Services outlines our approach to review and evaluation for accountability and improvement. This framework is very strongly focused on improvement whilst maintaining a focus on compliance with regulatory standards.
Akanuku | Assurance Reviews assess whether services are meeting regulatory standards. We have been implementing Akanuku since July 2019 following a trial in the first half of 2019.
Akarangi | Quality Evaluations evaluate the extent to which services have the learning and organisational conditions to support children’s learning.
Akamatua | Evaluating Excellence and Innovation identify, investigate and evaluate cases of excellence and innovation. We are developing this case study approach and we will be undertaking some initial case studies in selected services later in 2020 to inform the design of this approach prior to implementation in 2021.
ERO is responsible for reporting publicly on the quality of education provided in New Zealand schools and early childhood services. Its role encompasses accountability (including compliance with legal requirements), educational improvement, and knowledge generation.
ERO undertakes an Akanuku | Assurance Review process in any centre-based service:
An ERO Akanuku | Assurance Review assesses whether a service meets regulatory standards/licensing requirements. As part of an Akanuku | Assurance Review process, ERO makes one of the following judgements:
ERO will recommend that the next review of the service is likely to be an Akarangi | Quality Evaluation; or that the next ERO review of the service will be in consultation with the Ministry of Education.
Akarangi | Quality Evaluations evaluate the extent to which an early childhood service has the learning and organisational conditions to support equitable and excellent outcomes for all learners. Te Ara Poutama- indicators of quality for early childhood education: what matters most are the basis for making judgements about the effectiveness of the service in achieving equity and excellence for all learners. The Akarangi Quality Evaluation Judgement Rubric derived from the indicators, is used to inform the ERO team’s judgement about this service’s performance in promoting equity and excellence.
In an Akarangi | Quality Evaluation, judgements are made in relation to outcomes for learners for te Akatoro | each domain in Te Ara Poutama- indicators of quality for early childhood education: what matters most. The Akarangi Quality Evaluation Judgement Rubric is used to evaluate where this service is on a quality improvement continuum. ERO is encouraging early childhood services to use the same judgements in their internal evaluation supported by evidence about where they are on the continuum and what they need to do to improve.
|
Judgement |
Description |
|
Whakatō Emerging |
The seed is sown, a place of potential Emerging means that the service is at an early stage of developing the learning and organisational conditions (systems, processes and practices) to improve quality and realise equitable outcomes for children. |
|
Whāngai Establishing |
Nourishment is provided Establishing means that the service is establishing the learning and organisational conditions (systems, processes and practices) and is building shared understandings to improve quality towards realising equitable outcomes for children. |
|
Whakaū Embedding |
Consolidation of all elements has occurred Embedding means that the service has established and is embedding coherent learning and organisational conditions (systems, processes and practices) and is consistently implementing practices and processes to enable improvement in quality towards realising equitable outcomes for children. |
|
Whakawhanake Sustaining |
Appropriate conditions provide strong and ongoing support Sustaining means that the service has evidence of its capability and capacity to sustain ongoing improvement to the learning and organisational conditions (systems, processes and practices) and the impact of this improvement in realising equitable outcomes for children. |
|
Kia rangatira ai te tipu Excelling |
A combination of favourable conditions and a nourishing environment produces ongoing quality of results Excelling means that the service is sector leading and demonstrates exemplary practice resulting in equitable outcomes for children. |
Te Ara Poutama- indicators of quality for early childhood education: what matters most
Akarangi Quality Evaluation Judgement Rubric
Ngā Rāpupuku Indicators – poster
ERO begins the process with a notification email that details:
the indicators
Before the on-site phase information is shared between ERO and the service. This communication gives you an opportunity to:
The exchange of information is essential in preparing your service and the ERO team for the evaluation.
In a Quality Evaluation ERO will begin with checking some compliance items working with a nominated person from the service. The following items are checked because they have a potentially high impact on children’s learning, wellbeing and health and safety:
If this checking process identifies non-compliance that is an ‘unacceptable risk’ to children (related to any of the items above), ERO will shift the focus of the evaluation to that of an Akanuku | Assurance Review and do a full assessment of the regulatory standards and licensing criteria.
ERO also checks to see that areas of non-compliance in the previous ERO report (if applicable) have been addressed.
ERO considers your service’s internal evaluation information about what you know about your effectiveness in achieving equitable and excellent outcomes for learners and the extent to which your service’s learning and organisational conditions support or hinder improvement.
The ERO team uses all available information, including discussions with you, to design the evaluation.
The onsite phase of the evaluation begins with an initial discussion between your service (key personnel) and the ERO team.
The discussion is an opportunity for the ERO team to:
The ERO team gathers information in a range of ways. Review officers will read documents, meet with key people and observe practice including some aspects of your curriculum in action. At the end of the onsite phase the ERO team will meet with nominated service personnel to share and clarify emerging findings of the evaluation.
Akarangi | Quality Evaluation reports are concise. They make clear ERO’s evaluative judgements in relation to the outcome and process indicators.
The Akarangi | Quality Evaluation report will include:
The unconfirmed report is emailed to your service 20 working days from the last day the ERO team was at your service. You will be invited to respond noting any errors of fact, significant omissions or comment on any other matters related to the findings, including providing additional evidence to support this. Your service has 15 working days to respond.
The ERO coordinator considers the response from your service including the supporting documentation supplied. The ERO coordinator may want to discuss the response with others in ERO. The review coordinator then decides whether to amend the report and, if appropriate, makes appropriate amendments.
ERO will publish the Akarangi | Quality Evaluation report on its website two weeks after it is sent to your service.
ERO’s intent is to ensure that all learners in our system are successful. We are interested in supporting your service’s improvement journey. Before our Quality Evaluation starts on site, we encourage you to reflect on the questions below. These questions provide the broad context for discussion likely to occur during your external evaluation. In reflecting on these questions, you are likely to refer to a range of existing information you already have in your service.
We are particularly interested in discussing with you: