Operating Intentions

Intervention logic

Alongside its refocused approach, ERO is developing an intervention logic, which will demonstrate the relationships and links between ERO’s evaluations and the desired results. Impacts 1 and 2 (below) and their associated measures should be considered in the context of the developing intervention logic.

ERO is working with Victoria University of Wellington to develop the intervention logic and to strengthen the associated measures. ERO will also consider research undertaken by the New Zealand Institute of Economic Research (NZIER) in the development of this work.

Impact 1

What are we seeking to achieve?

ERO’s system-wide evaluations will influence and advise on the development and implementation of education policy and practice.

The aggregation and analysis of national review data gives ERO an opportunity to provide a broader picture of the topical issues, initiatives or policies of interest to the sector and Government.

How will we demonstrate success?

ERO’s national evaluations will promote debate and influence change to the education system. National evaluations and reports of good practice will add value to the sector’s policy decision-making processes when:

• there is evidence that they are used by the Ministry of Education and other government agencies to establish priorities for the education sector
• there is evidence that they help the Government to target resources and interventions to areas of poor performance
• there is evidence that they are used by schools and early childhood services to inform their own practice
• they are topical, timely, and consistent with the Government’s priorities for education.

What will we do to achieve this?

In the next three years ERO will evaluate and report on the effectiveness of specific education strategies intended to advance national education priorities, in particular:

• how well schools are implementing National Standards in literacy and numeracy, and using assessment for educational improvement
• literacy and numeracy in secondary schools and across the curriculum
• sector strategies aimed at enhancing students’ attainment of skills and qualifications
• schools support for target groups of students to achieve, particularly Māori and Pacific, and those with special needs
• the senior secondary school-tertiary interface, focusing on programmes developed to support young people in their transition from school to tertiary education and employment
• the quality of education received by international students enrolled in New Zealand schools
• the participation of target groups in early childhood education, and how well services are building strong foundations for future learning.

ERO will work with the Ministry of Education and government agencies to identify the topics for national evaluations that will be useful and add value to the education sector.

ERO will also inform policy development in these government agencies by:

• providing contractual evaluation services under agreed terms of reference
• producing timely policy advice, to ERO’s quality standards
• producing reports of good practice and self-review tools.

ERO’s national evaluations and reports of good practice will also be made available to parents, whānau, iwi, communities and education professionals – to raise their understanding and awareness of education issues and practice.

Measures and Trends

To assess the effectiveness of its evaluations, ERO will collect feedback on:

• the use of ERO’s evaluations and findings by the Ministry of Education
• the use of ERO’s evaluations and findings by schools and early childhood services.

Use of ERO’s evaluations by the Ministry of Education

A study of the use of national evaluation reports, undertaken in 2009, found that they provided high quality, relevant and useful information for the Ministry of Education. Ministry of Education officials used ERO’s evaluations and advice for policy development and briefings, planning, and the design of professional development for the education sector.

ERO will schedule a further study in 2011-12.

Use of ERO’s evaluation reports and findings by schools and early childhood services

ERO collected data on the use of ERO’s evaluation reports and findings by schools and early childhood services in 2009-10. A further study will be undertaken in 2012-13.

Impact 2

What are we seeking to achieve?

ERO will provide reviews of schools and early childhood services that will support them to improve student achievement, the quality of teaching and learning, governance, self review and leadership.

The evaluations provide quality assurance to the Government and the community.

How will we demonstrate success?

ERO’s evaluations and reports will have contributed to improvements in the education system when:

• there is evidence that schools and early childhood services have used the evaluation process to review and improve their practice - with specific emphasis on the quality of teaching and learning
• there is evidence that schools and early childhood services have used the evaluation process to improve education and promote success for Māori learners
• there is evidence that schools and early childhood services use self review to improve achievement.

What will we do to achieve this?

ERO expects to review approximately a third of all schools and early childhood services each year. In doing so ERO will take into account that:

• for the review process and reports to have the desired impact, schools and early childhood services must use them to make improvements
• review resources should be increasingly targeted to schools that need more assistance and not those schools where ERO is confident in their ability to continue to improve their service
• ERO has a reduced budget and this is not expected to increase in the forecast period – further savings will have to be made while at the same time maintaining review levels

ERO will make its reports on schools and early childhood services available to parents, whānau, iwi and communities. Through these reports ERO encourages their interest and participation in their children’s schools and early childhood services.

ERO will also make changes to its format for review reports to increase usability for parents, whānau, iwi and communities.

Measures and Trends

ERO’s aim to provide a cost-effective service is reflected in the above measures and trends. ERO aims to maintain its review frequency (one-third of schools and early childhood services reviewed annually) and high effectiveness rating (80 percent), while maintaining and/or reducing its service cost and input levels over the medium term.

To assess the impact of its reviews, ERO will seek feedback from schools and early childhood services about what actions have been taken on its recommendations and how the evaluation process has contributed to improvements in education delivery.

Table 1: Measures and trends

ERO has reduced review activity in Christchurch as a result of the earthquake in February 2011. This may continue to have an impact on the number and frequency of reviews undertaken in the Christchurch region.