Complementary evaluation is the purposeful interaction between internal and external evaluation. It implies a mutually beneficial relationship that recognises the distinct purposes of each and where the overlap lies.
ERO’s external evaluation process is both proportional and responsive to the service’s self review. It responds to the early childhood service’s overall capacity and capability to evaluate its own performance. ERO’s external evaluation also has a role to play in building the evaluation capacity of the service by strengthening self review according to each service’s context.
Diagram 1: Complementary evaluation
An evaluation approach that balances external ERO review with service self review according to each service's circumstances.
External evaluation can:
Internal evaluation can:
ERO uses its external evaluation process to increase the capability within early childhood services to undertake internal evaluation (self review) as a routine activity for both accountability and improvement purposes. The intention is for evaluation to become embedded in the day-to-day practice of managers and educators.
ERO builds the evaluation capability of the early childhood service through:
The term self review is often used synonymously with other terms such as self evaluation, internal evaluation, evaluation, assessment, monitoring and appraisal. In early childhood services in Aotearoa New Zealand self review has its roots in terms such as programme review, internal review and quality review.
ERO defines self review as the use of robust processes to systematically inquire into and evaluate the effectiveness of policies, programmes and practices. Self-review findings are used to inform decision-making, improve the quality of practice and promote positive outcomes for all children.
Review implies a looking back or taking stock of some activity, practices or processes. However, self review is also evaluative in nature.
Definitions of evaluation vary according to the underpinning theory or theories that they are based on. All evaluation involves reaching judgements or producing knowledge about what is being evaluated. The purpose of each evaluation differs: it can be for accountability purposes, to help with decision making, to contribute to improved effectiveness, to increase understanding or to advance a principle such as equity. This is referred to as evaluation use.
Early childhood services are required to undertake self review as part of their licensing requirements. The criteria to assess the governance management and administration standard specifies that an ongoing process of self-review helps the service maintain and improve the quality of its education and care. 1
The licensing criteria require early childhood services to document:
Self review is an integral part of professional practice in the governance, management and administration of an early childhood service.
The Ministry of Education resources such as Quality in Action: Te Mahi Whai Hua 2 and The Quality Journey: He Haerenga Whai Hua: Improving quality in early childhood services 3 help early childhood services understand the concept of self review.
The Ministry has also published guidelines for self review in early childhood services: Ngā Arohaehae Whai Hua 4. These guidelines set out a process for services to use to undertake self review. This process is one that requires a systematic process of preparation, data gathering, analysis and decision-making. It implies an evaluative approach.
In 2008 ERO undertook a national evaluation 5 that focused on how well self review was understood, supported and implemented in early childhood services. ERO found that where self review was highly effective:
The factors that emerged as common to all services where self review was well understood and implemented included: