Part 7: Process guidelines for reviews

Notification

ERO begins the process with a notification email that gives the hospital-based service time to prepare for the external review.

In the notification email ERO includes links to electronic versions of He Pou Tātaki (this document), the Hospital-based Assurance Statement and Self-Audit Checklists, the Self Report and links to electronic versions of other relevant documents.

Self report

The self report plays a significant part in the ERO review. ERO’s evaluation builds on the information that the service provides in the self report about what is going well and where improvement may be needed. It helps ERO to design a review that is responsive to each context.

The self report is largely structured on Ngā Pou Here. Completing the self report is an opportunity for the service to reflect on its practice in relation to contributing to children’s learning and promoting their wellbeing

Information exchange

Before the on-site stage of the review, information is shared between ERO and the service.

The review coordinator gives the service an opportunity to clarify information about the review process through telephone or email contact, and explores with the service who will take responsibility for being involved in the review.

The review coordinator specifies the date for the hospital-based service to provide ERO with the completed Self Report and Assurance Statement and other requested documentation.

Designing the review

Review design involves ERO deciding, often in collaboration with the service, where it will place its evaluation emphasis during the ERO review. A significant determinant of review design is the quality of internal evaluation within a service.

The organisation and nature of the ERO review is determined in response to context. Context includes the service’s philosophy, vision and goals and its capacity, as indicated through documentation and other sources of information.

ERO uses information provided by the service, including the completed relevant Assurance Statement and Self Report, to scope the review. ERO also draws on internal information it has about the service including the last ERO report and reporting history. The scoping process helps ERO to determine what needs to be further explored during the review.

ERO uses Ngā Pou Here as a framework for organising the information it has and the investigative questions it wants to answer on the review. Consideration is given to the interrelationship between the Pou and where the emphasis will be placed.

The review team plans the review process taking into account the resources available for the review. The review design is shared with the service.

 

The focus of the review

Decisions about the balance between the Pou and where to focus reviews depend on information in the completed Self Report and Assurance Statement.

The time allocated to gathering information within each Pou, the processes used and the reporting of findings varies between reviews. ERO uses a ‘one-size-fits-one’ approach so that the process is tailored to be as responsive and contextual as possible. Each review will look different.

The Ngā Pou Here framework shows relationships between each Pou and children’s learning and wellbeing. Areas of strength and areas for review and development that are identified by ERO and the service should lead to (or help create the conditions for) improvements in how the service promotes children’s learning and supports their wellbeing.

The national evaluation topics (NETs) provide a way for ERO to investigate key aspects of the hospital-based service’s performance in relation to the Government’s education priorities, where relevant. Each topic is explored through the review framework, Ngā Pou Here, and is evaluated in this context.

In some services, reviews need to focus on compliance, because of risks to the safety and wellbeing of children. This is likely to be the case where, despite attestation made by the service in the Assurance Statement, there do not appear to be adequate systems for the internal checking of compliance.

Investigation and synthesis

During its time on-site the review team:

  • uses evaluative questions, investigative questions and evaluation indicators as a basis to gather and document evidence
  • reads the service’s documentation, talks to leaders, HPS and others as appropriate, and determines the purpose and focus of any observations
  • synthesises the review findings.

The service delegates appropriate personnel to be involved in the review and negotiates the level of their involvement with the review team. Ongoing interaction between service personnel and the review team will be a feature of the process.

The scoping process helps review teams to plan with the service who else ERO should talk to during the review. Participants may include:

  • hospital play specialists (HPS)
  • the contact person/service provider
  • hospital leaders/management
  • parents and families/whānau

From the service’s self-review information ERO identifies if the hospital-based service is already performing well in a specific area or if there is a need for improvement.

For areas where the service is performing well, the priority is on validating the results of internal evaluation. For areas where review or development is needed ERO uses its processes to build the service’s capability to evaluate and improve its own practice.

 

Consultation

ERO encourages each hospital-based service to share information about the consultation it has undertaken. ERO is particularly interested in any consultation that shows:

  • how the service works in partnership with parents and families/whānau
  • how HPS contribute to review and development in the service.

Discussion of findings

Towards the end of its time at the service the review team will discuss the review findings with the personnel nominated by the service. In situations where significant development is needed ERO will indicate the likelihood of an early return review.

This discussion of findings could include services leaders and/or staff who will have the responsibility for taking action as a result of the external review.

It may be that findings are shared throughout the review process, in which case a discussion of findings may not be necessary at the end of the onsite stage of the review.

Reporting

The audience for ERO reports includes the government and the public, as well as those in the early childhood sector.

Hospital-based service reports will start with an overall judgement. The report will include the material findings that answer the overarching evaluation question. The report is sent to the hospital-based service as an unconfirmed report within 20 working days of the completion of the on-site part of the review. A service that is identified as Not Well Placed will receive the unconfirmed report within 10 days of the last day on site.

The management of the hospital-based service has 15 working days from the date ERO sends the report in which to query the evidential basis for reported judgements; and/or advise ERO in writing of any errors of fact and provide supporting documentation. A service that is identified as Not Well Placed has 10 days to respond.

ERO considers any response from the service and, where justified, makes amendments to the report. The report is confirmed and a copy is sent to the service provider.

The confirmed report is released publicly on ERO’s website two weeks after it is sent to the service provider.

Differentiated Return Times

The timing of the next ERO review will depend on how well placed the service is to contribute to children’s learning and promote their wellbeing. There are four options:

Very well placed – the next ERO review in four years

ERO will next review the service in four years when it finds that the service is consistently effective in contributing to children’s learning and promoting their wellbeing. High quality performance in relation to ERO’s evaluation indicators for Ngā Pou Here will be evident.

Well placed – the next ERO review in three years

ERO will next review the service in three years when it finds that the service is effective in contributing to children’s learning and promoting their wellbeing. Good performance in relation to Ngā Pou Here will be evident.

 

Requires further development – the next ERO review within two years

This option is used when many of the factors that contribute to children’s learning and promote their wellbeing are not evident or require significant development. ERO will have some confidence that the service can improve with support.

After receiving the confirmed report, the service will be expected to participate in a meeting with ERO and the Ministry of Education to begin developing a plan for improvement.

The Ministry of Education will oversee the support that the service needs to become Well placed to contribute to children’s learning and promote their wellbeing.

Approximately six to nine months after the ERO review, the service will provide ERO with an update about its progress. The Ministry of Education will also update ERO on the progress made. This information will be used to determine the appropriate ERO return time within the two year period.

Should ERO find that there has not been sufficient improvement the service will be identified as Not well placed. ERO will recommend that the service’s licence is reassessed by the Ministry of Education.

The next ERO review in consultation with the Ministry of Education

This option will be used when a service is not performing adequately, is not meeting legal requirements and does not have the capacity to make improvements without support or Ministry intervention.

The service will be expected to address concerns and prevent a continuation of poor performance. There will be licensing consequences for continued poor performance.

ERO will not review the service again until the Ministry of Education is satisfied that the service meets licensing requirements.

The four different review return times provide ERO with greater flexibility in how it responds to  a hospital-based service’s performance. ERO and the Ministry of Education will work closely with services that need to improve and help them build capability. ERO will make less frequent visits to those services that are performing very well.

Most services are likely to be considered Well placed and will continue to be reviewed every three years.

ERO has developed criteria to support the overall judgements and their corresponding return times. These are on ERO’s website (www.ero.govt.nz) under Review Process/Early Childhood. Ngā Pou Here and the evaluation indicators support the criteria and provide a deeper insight into what ERO considers to be high quality early childhood education and care in hospital-based services.

Diagram 7: Connections between Ngā Pou Here, the Evaluation Indicators, and the Criteria

Diagram showing connections between Nga pou here, indicators and criteria