Part 3: Complementary approach to evaluation in hospital-based service reviews

ERO’s external evaluation process is both proportional and responsive to the service’s internal evaluation. It responds to the hospital-based service’s overall capacity and capability to evaluate and improve its own performance. ERO’s external evaluation also has a role to play in building the evaluation capacity of each service according to its context.

Diagram 1: Responsive approach to evaluation

An evaluation approach that balances external ERO evaluation with service's internal evaluation according to each service's circumstances.

Continuum showing approach to evaluation - internal and external

The relationship between external and internal evaluation

External evaluation can:

  • stimulate internal evaluation
  • expand the scope of internal evaluation
  • validate the results of internal evaluation
  • provide an additional perspective
  • include a capacity building role as part of the external evaluation.

Internal evaluation can:

  • deepen the scope of external evaluation
  • give a context to the external evaluation
  • provide important insights
  • improve the interpretation of external evaluation findings
  • contribute to increased use of external evaluation findings.

Building evaluation capability

ERO uses its external evaluation process to increase capability within hospital-based services to undertake internal evaluation as a routine activity for both accountability and improvement purposes. The intention is for evaluation to become embedded in the day-to-day practice of leaders and hospital play specialists.

ERO builds the evaluation capability of the hospital-based service through:

  • making its own external evaluation processes transparent
  • modelling evaluation practice
  • encouraging participation in ERO’s evaluation process
  • having discussions about the service’s internal evaluation processes
  • involving nominated service personnel in evaluation design, analysis and synthesis processes
  • providing tools (e.g. examples of questions and indicators) that services can use in their internal evaluation
  • discussing resources that services can use to help them with internal evaluation.

Internal evaluation in hospital-based services

The term internal evaluation is often used synonymously with other terms such as self evaluation, self review, assessment, monitoring and appraisal.

ERO defines internal evaluation as the use of robust processes to systematically inquire into and evaluate the effectiveness of policies, programmes and practices. Evaluation findings are used to inform decision-making, improve the quality of practice and contribute to children’s learning and wellbeing.

Regulatory requirements for internal evaluation

Hospital-based services are required to undertake self review as part of their licensing requirements. The criteria to assess the governance management and administration standard specifies that an ongoing process of self-review helps the service maintain and improve the quality of its education and care.19

The licensing criteria require hospital-based services to document:

  • a process for evaluating their operation
  • a schedule showing timelines for planned evaluation of different areas of operation
  • recorded outcomes from the evaluation process.

Internal evaluation is an integral part of professional practice in the governance, management and administration of the hospital-based service.

Guidance and support to help services with internal evaluation

The Ministry of Education resources such as Quality in Action: Te Mahi Whai Hua20 and The Quality Journey: He Haerenga Whai Hua: Improving quality in early childhood services21 can help hospital-based services understand the concept of internal evaluation.

The Ministry has also published guidelines for self review in early childhood services:

Ngā Arohaehae Whai Hua.22 These guidelines set out a process for services to use to undertake internal evaluation. This process is one that requires a systematic process of preparation, data gathering, analysis and decision-making. It implies an evaluative approach.

Internal evaluation: scope, depth and focus

Internal evaluation processes help the hospital-based service know how well it is contributing to children’s learning and promoting their wellbeing. As a result of information from internal evaluation, the service is able to identify how to improve practice.

There are different ways of describing the scope, depth and focus of internal evaluation.

The Ministry of Education’s guidelines refer to a framework of planned and spontaneous review/evaluation. The following diagram expands on that framework by including strategic evaluation. Strategic evaluation is the process by which the hospital-based service evaluates how well it is achieving its vision, goals or philosophy.

Diagram 2: Scope, depth and focus of internal evaluation

strategic evaluation - regular evaluation, emergent evaluation

Strategic evaluation is long term, and focused on key goals related to the hospital-based service’s vision, goals or philosophy.

Regular (or planned) internal evaluations are about ‘business as usual’. They are smaller, focused and ongoing, feeding regular information into the strategic evaluation.

Emergent (or spontaneous) evaluations are in response to unplanned events or issues as they arise. They are one-off spontaneous evaluations but should fit with overall goals and link to other evaluations.

All evaluations involve gathering information which is used as evidence to support judgements and make decisions about service direction and priorities.

Internal Evaluation: a continuum of understanding and practice

Evaluation is a developmental activity that is best understood as operating on a continuum of understanding and practice. The features or characteristics of each of the identified stages are a guide to determining the level at which the hospital-based service understands and undertakes internal evaluation. See diagram 3 below. 

Diagram 3: Internal evaluation continuum

A continuum of capacity showing stages of internal evaluation

19   Ministry of Education. (2015). Licensing criteria for hospital based ECE services. Retrieved from early-childhood/running-an-ece-service/the-regulatory-framework-for-ece/licensing-criteria/hospital-based-ece-services/ governance-management-and-administration/professional-practices/gma6-self-review/.

20   Ministry of Education. (1998). Quality in Action/Te Mahi Whai Hua: Implementing the Revised Statement of Desirable Objectives and Practices in New Zealand Early Childhood Services. Wellington: Learning Media.

21   Ministry of Education. (1999). The Quality Journey/He Haerenga Whai Hua: Improving quality in early childhood services. Wellington: Learning Media

22   Ministry of Education.(2006). Ngā Arohaehae Whai Hua/Self-review guidelines for early childhood education. Wellington: Learning Media.