ERO’s external evaluation process is both proportional and responsive to the service’s internal evaluation. It responds to the hospital-based service’s overall capacity and capability to evaluate and improve its own performance. ERO’s external evaluation also has a role to play in building the evaluation capacity of each service according to its context.
Diagram 1: Responsive approach to evaluation
An evaluation approach that balances external ERO evaluation with service's internal evaluation according to each service's circumstances.
ERO uses its external evaluation process to increase capability within hospital-based services to undertake internal evaluation as a routine activity for both accountability and improvement purposes. The intention is for evaluation to become embedded in the day-to-day practice of leaders and hospital play specialists.
ERO builds the evaluation capability of the hospital-based service through:
The term internal evaluation is often used synonymously with other terms such as self evaluation, self review, assessment, monitoring and appraisal.
ERO defines internal evaluation as the use of robust processes to systematically inquire into and evaluate the effectiveness of policies, programmes and practices. Evaluation findings are used to inform decision-making, improve the quality of practice and contribute to children’s learning and wellbeing.
Hospital-based services are required to undertake self review as part of their licensing requirements. The criteria to assess the governance management and administration standard specifies that an ongoing process of self-review helps the service maintain and improve the quality of its education and care.19
The licensing criteria require hospital-based services to document:
Internal evaluation is an integral part of professional practice in the governance, management and administration of the hospital-based service.
The Ministry of Education resources such as Quality in Action: Te Mahi Whai Hua20 and The Quality Journey: He Haerenga Whai Hua: Improving quality in early childhood services21 can help hospital-based services understand the concept of internal evaluation.
The Ministry has also published guidelines for self review in early childhood services:
Ngā Arohaehae Whai Hua.22 These guidelines set out a process for services to use to undertake internal evaluation. This process is one that requires a systematic process of preparation, data gathering, analysis and decision-making. It implies an evaluative approach.
Internal evaluation processes help the hospital-based service know how well it is contributing to children’s learning and promoting their wellbeing. As a result of information from internal evaluation, the service is able to identify how to improve practice.
There are different ways of describing the scope, depth and focus of internal evaluation.
The Ministry of Education’s guidelines refer to a framework of planned and spontaneous review/evaluation. The following diagram expands on that framework by including strategic evaluation. Strategic evaluation is the process by which the hospital-based service evaluates how well it is achieving its vision, goals or philosophy.
Diagram 2: Scope, depth and focus of internal evaluation
Strategic evaluation is long term, and focused on key goals related to the hospital-based service’s vision, goals or philosophy.
Regular (or planned) internal evaluations are about ‘business as usual’. They are smaller, focused and ongoing, feeding regular information into the strategic evaluation.
Emergent (or spontaneous) evaluations are in response to unplanned events or issues as they arise. They are one-off spontaneous evaluations but should fit with overall goals and link to other evaluations.
All evaluations involve gathering information which is used as evidence to support judgements and make decisions about service direction and priorities.
Evaluation is a developmental activity that is best understood as operating on a continuum of understanding and practice. The features or characteristics of each of the identified stages are a guide to determining the level at which the hospital-based service understands and undertakes internal evaluation. See diagram 3 below.
Diagram 3: Internal evaluation continuum
19 Ministry of Education. (2015). Licensing criteria for hospital based ECE services. Retrieved from www.education.govt.nz/ early-childhood/running-an-ece-service/the-regulatory-framework-for-ece/licensing-criteria/hospital-based-ece-services/ governance-management-and-administration/professional-practices/gma6-self-review/.
20 Ministry of Education. (1998). Quality in Action/Te Mahi Whai Hua: Implementing the Revised Statement of Desirable Objectives and Practices in New Zealand Early Childhood Services. Wellington: Learning Media.