2017 - Briefing to the incoming Minister

Ko te Tamaiti te Pūtake o te Kaupapa
The Child – the Heart of the Matter

Part One - About the Education Review Office

The Education Review Office (ERO) is responsible for:

  • evaluating the implementation of government education priorities, programmes and policies across the system
  • evaluating the quality of education and care in schools and early learning services and across Kāhui Ako | Communities of Learning
  • supporting improvement in the performance and operation of our early learning services, schools, and Kāhui Ako | Communities of Learning
  • contributing to the evidence base about what works in support of decision makers and practitioners.

Our review information is used to promote better educational practice, inform debate about national education issues and to provide assurance to you and the Government about the quality of education provision in New Zealand. Our reviews enable parents to make informed choices and to engage in their children’s learning pathway. We equally attempt to target teachers to share insights and evidence around effective practice.

ERO was established in 1989 as an independent government department under the structural reforms of the New Zealand Education System, Tomorrow’s Schools.

Under Part 28 of the Education Act 1989, Review of Educational Services, the Chief Review Officer has the power to administer reviews (either general or relating to particular matters) of the performance of applicable [pre-tertiary] organisations in relation to the applicable services they provide and to prepare reports on the undertaking and results of such reviews.

As Minister, you can also ask the Chief Review Officer to initiate a review into a particular issue or element of the early childhood or school sectors.

ERO’s independence from schools and early childhood services, as well as from the agencies that set the policies and standards, enables ERO to provide assurance to you as Minister, the Government, parents and the broader community on the quality of our system and of educational provision within New Zealand.

Review Services

Each year, our review officers undertake approximately 1260 external evaluations of early learning services and 700 external evaluations of schools. This is equivalent to approximately 18 school reviews and 25 reviews of early learning services each week. We are your eyes and ears on the ground and in a unique position to be able to provide you with real time information about how the system is performing.   Our 152 review officers are professional evaluators who, in the main, come from senior leadership positions in schools and early learning services. Our review approach aims to provide entities with an opportunity to reflect on their performance, what they are attempting to achieve and to draw attention to those areas where things are not going well. Depending on our findings early childhood services and schools are on a three year review cycle, with this period extended to between four and five years for those services which we assess as being on a sustainable self-improvement path. Those services where there are only limited conditions for success present will be placed on a more intensive review programme over 1-2 years.

During the course of an early learning or school review, in addition to our core focus on learner outcomes, we audit both the quality of teacher appraisal systems (on behalf of the Education Council) and as part of an agreement with NZQA, we check the compliance of the school with their commitments under the Education (Pastoral Care of International Students) Code of Practice 2016.

ERO also undertakes reviews in respect of providing assurances to you as Minister and the Ministry of Education (Ministry) for new schools and Partnership Schools | Kura Hourua. Reviews of the quality of provision for those choosing to home school their child(ren) are carried out at the request of the Ministry.


Te Uepū ā-Motu

ERO is a world leader of indigenous education evaluation, with a dedicated immersion review team, Te Uepū ā-Motu. With skills and experience in education and evaluation, this team of reviewers has specific knowledge in te reo and tikanga Māori and has developed specialist (Te Ao Māori) methodologies to undertake reviews in Te Aho Matua Kura Kaupapa Māori, Kura-ā-Iwi and Kōhanga Reo immersion settings.

National Evaluation Services

ERO’s legislative mandate and its independence enables it to provide independent and frank advice to you, the Government and wider sector on a range of national education issues.

ERO produces a programme of national education evaluations that are designed to influence and inform the national education debate. The purpose of these evaluations is to act as change levers in education, to raise systemic issues, to enhance policy design and implementation, and to assist with improvements in education and raising of student achievement. ERO’s national evaluations are used to inform and support you as Minister, and to inform the work of the Ministry of Education and other government agencies. Schools and early learning services also use our research and evaluation findings to develop and enhance their own practice. Our approach focuses on two key areas:


  • National Evaluations – system-level issues including sector performance, policy implementation and pre-tertiary educational practice.
  • Special Reviews – where a matter needs to be reviewed and reported on outside of a regular review cycle.
  • Effective practice reviews and resources – where we identify and share innovation and effective practice.

A list of recent publications associated with this programme of work and our current 2017/18 work programme is contained in Annex 1 to this briefing.


How the Sector Views our Work

Post-evaluation follow-up shows that over 90% of early learning services regard their ERO review as useful or very useful and 94% said the review identified and confirmed for them their direction forward. Over 87% of schools reviewed felt that their ERO external evaluation contributed to decisions about how to improve learning outcomes for students and 89% said that ERO’s approach helped them to identify or confirm opportunities for more effective practice. Further to this, NZCER’s (New Zealand Council for Educational Research) regular national survey of schools shows that ERO rates highly among trustees as a source of advice.

Part Two - Making a difference for every learner

A Deliberate Improvement Agenda

ERO aims to ensure that every learner in New Zealand has access to high-quality teaching and learning, and that our early learning and school systems continue to adapt and evolve to the changing needs of New Zealand as a nation. While ERO has a relatively narrow focus in a complex system, and limited resources, we believe that through our work we have significant capacity to influence what takes place in every learning environment. Evidence shows that highly effective evaluative practice that is carried out by highly-skilled evaluators will influence performance.

Since its establishment in 1989, ERO’s approach has shifted from an emphasis on compliance and externally driven accountability to one which is driven by improvement and balancing external and internal institutional level evaluation and accountability.

The step change we have made is from asking whether things are being done right, to evaluating whether providers are doing the right thing and understanding the impact of their actions. We do this by focusing on those things that we know through research make the biggest difference in lifting educational performance, and those things that are important to a high performing education system. Our approach places an emphasis on the impact of learner outcomes and the policies, programmes and practices which contribute to these.

Fundamental to this change is the premise that improvement in our system is derived from what happens in every classroom and within every institution and from the actions which occur on a daily basis within that setting. Ultimately our starting principle is that an entity’s management and leadership, and more recently a Kāhui Ako | Communities of Learning, should know their organisation(s), their community, the needs of their learners and the capabilities of their workforce.

We maintain that a high performing system is one in which providers have strong internal systems of accountability to their learners and parents, between practitioners and to their community. As part of this approach, internal evaluation systems are matched with periodic external review and assessment. Our external evaluation process promotes and complements internal evaluation, consistent with international trends in schools taking primary responsibility for their own improvement.

Characteristics of Good Providers

At the heart of what  the  evidence  and  our  insights over the past 27 years tells  us about those providers that produce good outcomes, is that they have a clear improvement agenda, and that the entity has a culture of high expectations for all learners. The have both a long-term commitment to acceleration and a planned approach to improvement that provides wrap-around support for learners and teachers. They benchmark their results internally, over time and against others and focus on accelerating learner outcomes and addressing areas of weakness. These are manifested in agreed targets and goals. Appropriate strategies, grounded in evidence and practice, are in place to reach and improve on these targets. Ongoing monitoring and evaluation of how they are progressing  and where  they  need to  adjust  their course  to continue to be successful are also  present. Our own assessments, underpinned by a significant body of research show us that leadership capability is essential in delivering on these conditions.


The Role of ERO in Early Learning Services

Early learning services provide a critical foundation to ensuring that New Zealanders grow up as competent and confident learners, strong in their identity, language and culture. New Zealand should be rightly proud of its strong early learning system and policy settings. The early childhood sector has experienced significant growth in recent years reflecting priorities for increased participation, population growth and growing demands for early learning services by working parents. This has led to an increase in the number of licensed services, and equally, a demand for higher quality education and care. Our work in the early childhood sector has highlighted considerable variability in the quality of curriculum implementation, assessment and evaluation in early learning services. Our focus will seek to address these inconsistencies in quality and provide support for improvement in provision.

Based on our research and evaluation findings, ERO has recently contributed to the Ministry of Education’s work on the refresh of the early childhood curriculum, Te Whāriki (2017). This year we are reviewing our approach to evaluations in early learning services, with a view to adopting a new framework that builds on the recently released Te Whāriki (2017) and Te Whāriki a te Kōhanga Reo, along with our increased emphasis on accelerating learner achievement through addressing Māori, Pacific and other learners at risk in our system.

Our Framework for School Improvement

In the school sector, our approach has seen us develop a school improvement framework aimed at assisting schools in their own review and assessment processes, but is equally used to guide our own focus during a school review.

Our improvement framework, School Evaluation Indicators, finalised in July 2016, reflects our understanding of the elements which are important to ensure effective organisational performance and school improvement. Our approach focuses schools and ERO evaluators on the things that matter most in improving student outcomes. The indicators draw on current research and evidence about what matters most in improving outcomes from schooling. The New Zealand Best Evidence Programme provided a strong foundation in terms of the research base underpinning our approach. The considerable evidence base generated through ERO’s national evaluation programme was also a significant resource.

Key elements which we define as hallmarks of an effective school include:

  • effective stewardship which involves planning for, and acting in, the school’s medium and long-term interests
  • focussed and capable leadership and leadership teams that promote a culture of improvement, ensure that the systems, practice and effort of the entity are focussed on improving outcomes for every learner, and that staff and resources are deployed in ways that focus on lifting and accelerating learner outcomes
  • strong relationships with parents, whānau and others outside of the entity in support of improvement and learner success
  • a responsive curriculum which is comprehensive across the year levels, engages learners, is adaptable to their needs and preferences, articulates what is expected of the learner, underpinned by comprehensive diagnostic and summative assessment and reporting systems. Key cross curricular competencies are fostered in all learning opportunities
  • effective teaching in which learning environments ensure learners are challenged and engaged in deep learning, they feel safe and are supported to take risks. Effective teachers understand and use data and continually adjust and differentiate their teaching in response to the progress that individuals are making, provide regular and timely feedback to learners (and their parents) on what they can do to make further progress, and ensure learners are engaged in setting their own learning objectives. In particular we see teachers in these schools support students to accelerate their progress in reading, writing and mathematics
  • professional capability building and communities of practice and development where staff work together as a team, supporting each other and with a clear focus on improving the quality of teaching and learning. Peer feedback is welcomed and individual appraisal and development linked to the individual and whole school development needs
  • the use of data, assessment and evaluation inquiry. Effective schools are also those that systematically collect a range of data on student outcomes, to monitor school wide achievement and performance across multiple domains, this is used to maintain a culture of self-evaluation and reflection across the school, to drive school-level decisions and initiatives to target the learning needs of individual learners and as a foundation in the school’s accountability to parents and the broader community.

Driving for Equity in Outcomes: Accelerating Student Achievement

New Zealand’s highly devolved system is characterised by increasing diversity and persistent disparities in achievement. Within the same school, young people can experience widely divergent opportunities to learn. This within-school inequity is amongst the highest found anywhere in the OECD, and it is strongly related to disparities in achievement. Furthermore, inequities in student outcomes are stratified by ethnicity with outcomes for Māori and Pacific Islands students falling significantly behind those of other groups.

The impacts of poor education outcomes are well known. Young people who have poorer educational attainment are more likely to have poor health, and over their life course have a greater reliance on welfare, are more vulnerable in the labour market, and have lower overall lifetime earnings.

The workforce of the future will demand different skills from those presently in our early childhood and school systems. Already we see the impact on the way we live our lives of the digital revolution, advances in ICT, the Internet of Things, cloud computing, big data, robotics and automation, the ubiquity of mobile devices, the increasing level of computerisation embedded within everyday appliances, the use of 3D printing and manufacturing, nano technologies, and advances in the exploitation of bio chemistry and genetics. The world of work inherited by those presently in our education system will be more skills intensive, and make many jobs obsolete, in particular low and semi-skilled jobs. Alongside growing demands for personal services, the skills demanded by our labour market in the future will require highly educated, analytical and design-oriented skills, underpinned by multi-lingual skills, skills of teamwork, creativity, organisational skills and entrepreneurship. This future labour market is also likely to be one subjected to continued disruption  and  change,  and  work  places  which  are  increasingly  global.  The  demands  on our education system are to ensure that those presently in our system have the skills to access and contribute to this future, that they have the ability to adapt and remain resilient in the face of rapid and continual change, and that as a nation we have a skill base which allows us to exploit the opportunities provided as a consequence of these technological advancements.

Preparing for this future relies on us ensuring that as a nation we turn around trends in key curriculum areas like Mathematics and Science. At the same time, we need to address the substantial risks of large parts of the New Zealand community becoming further marginalised in this future.

Through its individual provider review programme and through its national studies (see Part Three below), ERO has a role in ensuring that our system continues to improve and advance, that it remains relevant to the workforce demands of the future and that we identify and challenge those areas in our system where inequality and inequity persist.

Accelerating Student Achievement

In term one of 2016 we revised our main evaluative question in primary schools to focus on accelerating student achievement, focusing first on Māori, then Pacific learners and then on other groups of learners at risk of not achieving. The objective of this revised focus is to evaluate how effectively teachers and leaders in each school identify, and respond to, those whose learning needs to be accelerated. For primary schools, accelerated achievement has been defined as a student making more than one year’s learning progress, over the course of a year, and that they are on a trajectory that indicates they will be achieving at or above the required standard at the end of year 8 or sooner.

In term one of 2017 this approach was implemented in area, middle and intermediate schools and will launch in secondary schools in term 3 of 2017. From ERO’s perspective acceleration for students from Year 9 onwards should seek to ensure that all students gain at least NCEA Level 2 as part of a meaningful pathway to further education, training and success beyond secondary school.

Students at Greatest Risk in our System

While focusing our institutional review programme on acceleration of learners at risk, we will also undertake complementary national evaluations on provision for our most disadvantaged learners.

Our evaluative work informs not only the education sector, but also social sector agencies and community organisations, to help them understand the impact of their programmes and make decisions about future investment. Our recent work in the evaluation of a key initiative in the Prime Minister’s Youth Mental Health programme is an example of the contribution that we can make.

Many at risk learners are currently side-lined out of mainstream education into various types of remedial and alternative education programmes. Others have learning needs that mean they require additional support for inclusion and success. This work stream in our National Evaluations focuses on different aspects of provision for these learners in a variety of settings (including special school provision, Teen Parent Units or Alternative Education provision) as well as in mainstream settings, and on the arrangements which exist to provide additional support for these students (such as early intervention supports, teacher aides, counselling, wellbeing programmes, RT:LB services etc.). This work will enable us to draw deeper insights about what works for these students, both in 

terms of educational support and their wider social needs. In this area, a recent evaluation of education provision in Child Youth and Family youth justice facilities has highlighted some major concerns both with the nature of provision in these centres and with the model of provision operated for these learners nationally.

A New Approach for Schools in Need of Priority Support

We want to place our resources where there is the greatest need and where we can have the greatest impact. Approximately 9% of all New Zealand schools (180-190) that we review annually are identified as having sufficient concerns that they will be placed on a 1 to 2 year review cycle. The factors which characterise this group of schools include those that are small, in rural areas, those with new principals or recent leadership changes, and those schools serving lower socio- economic communities. Most have high numbers of priority learners whose educational needs are profound. Five percent of New Zealand children are in a school we have concerns about. While this is a relatively small group, no child or young person should be disadvantaged by a chronically poorly performing school.

Figure 1: Return times for all New Zealand schools, as at May 2017

Schools - Next Return Times (%)

 Pie graph showing school return times

In these cases, we work with the school and the Ministry to define the support, needs and strategies that are required to turn around this situation. Seventy percent of schools that are categorised in this way go on to improve within a two year time frame, however 30% remain on our “watch list”. At the extreme end, we have a small number of schools that have been in the poor performing category for more than a decade. This is not good enough.

We are taking a new approach with schools that we identify as struggling and that we believe do not have the capacity on their own to deliver improved student outcomes. Through a newly formed, dedicated team our approach commences with a deeper diagnostic or forensic assessment of the factors that we see as needing to change. We will identify the supports and interventions that are required to turn the situation around, and will work alongside the Ministry of Education and other appropriate agencies to ensure that sufficient interventions are put in place to address these situations. Our approach insists on the establishment of a documented recovery strategy and programme which ensures momentum, and deliberate focussed effort to address key areas of deficiency. We will regularly review improvement against the plan and our baseline assessments, and expect progress to be regularly discussed amongst the board, school leaders, staff, parents and students.

Kāhui Ako | Communities of Learning

The philosophy behind Kāhui Ako | Communities of Learning, supports the notion that with collaborative efforts of early learning services, schools and tertiary providers, student outcomes are more likely to be improved. This occurs through the pooling of the best resources available across the network to those areas in greatest need and where the greatest impact can be had in a student’s life course, better supporting learners through key transition points and by creating more seamless, deliberate and richer pathways, options and choices for learners. They have the potential to deliver economies of scale, agree common solutions and address cross sectorial problems which providers are unlikely to effect when working in isolation.

Evidence is increasingly telling us that a way to address the challenge of achievement variability is to shift towards a collaborative effort to maximise student progression. Kāhui Ako | Communities of Learning provide an important opportunity to improve teaching and learning through collaboration.

Ultimately such networks aim to have collective impacts across a defined geographic community not merely limited to a single institution’s enrolments. Success of Kāhui Ako | Communities of Learning clearly rests on effective cooperation, knowledge exchange, trust, the pooling of resources, agreed action and the significant use of data, analysis and evaluation.

ERO has recently published a framework document and a snapshot of where we see this transformation is presently.

We are interested in building the evidence, with the Ministry of Education, about how Kāhui Ako | Communities of Learning are establishing, developing and embedding ways of working as they move towards becoming fully functioning learning communities. We want to build the picture with schools and participants about what’s working well and what might require further support. We are currently generating the indicators and methodology that we will use to evaluate the collective impact of Kāhui Ako | Communities of Learning and intend to pilot approaches this year. Ultimately our approach will shift to assessing the impact and effectiveness that Kāhui Ako | Communities of Learning are having on an area and network basis as opposed to merely focussing at an individual institutional level. We intend to create individually tailored reports for each community, bespoke reports for schools receiving the Principal Recruitment Allowance, and national reports on trends and issues.

Part Three – Looking ahead

We intend to contribute to a measurable improvement in learner achievement by implementing targeted and bespoke evaluations that are a catalyst for change. We want to construct tailored evaluative judgements, accompanied by tangible recommendations for improvement that can be articulated and discussed by highly skilled ERO evaluators, education professionals and policy makers.

The availability of ever improving data from the Ministry of Education provides the opportunity to more clearly identify where the greatest need for attention lies. This information will enable us to be more strategic in our approach. In this context, we will consider how and when we review schools and early learning services.

Our Current National Evaluation Programme

Currently studies underway include:

  • RT:LB Service - an evaluation of all 40 RT:LB clusters
  • Sexuality Education – an evaluation of sexuality education curriculum in schools, including a focus on effective practice regarding LGBT students
  • Teen Parent Units – an evaluation of 24 teen parents units
  • Update of Te Whāriki – awareness and adoption in early learning services
  • Quality of Education in Activity Centres – an evaluation of 14 Activity Centres
  • Evaluation of the Te Kura pilot programme

Priorities for 2017 and Beyond

As described earlier, each year we produce a programme of national evaluations that are designed to influence and inform the national education debate. The purpose of these evaluations is to act as change levers in education, to raise systemic issues, to enhance education policy design and implementation, and to assist key players to improve the quality of education and the level of student achievement.

Priority areas for our programme going forward, contingent on securing appropriate resourcing and levels of interest will include:

  • continuing attention on effective leadership and support systems particularly for new leaders
  • maintaining an ongoing focus on issues of student wellbeing
  • investments and impacts of Professional Learning and Developmen
  • implementation of Te Whāriki (2017) and Te Whāriki a Te Kōhanga Reo
  • the effectiveness of behaviour management support systems and education for those with learning difficulties
  • effectiveness of early intervention services
  • effective practice in key curriculum domains
  • coverage and quality (of teaching and learning) in key STEM curriculum areas
  • skills for the future world of work including digital literacy and cross-curricular skills of collaboration and teamwork, creativity and entrepreneurial skills
  • decision making associated with the allocation of personnel and financial resources within education schools and Kāhui Ako | Communities of Learning.

Part Four – Matters for the incoming Minister

As part of our initial engagement with you, we will seek your input into our strategic direction, our key strategic projects to advance the system, as well as our national evaluations due for release. We will discuss with you where our effort and resources will have the most effective impact, and where there are opportunities to divest and re-invest in some activities.

ERO reviewers are in the field every day providing evaluative services that influence change and promote improvement. We are in a unique position to provide evidence-based information and insights to you.

We will alert you to any schools or early learning services that are at risk of under-performing and we will provide advice on interventions which we judge will turn the situation around. We will also draw your attention to new and emerging evidence from our research and evaluations and also examples of best practice that we identify through our daily operations and national evaluation topics.

At our first meeting, you may wish to discuss areas that you want ERO to prioritise (please see Part Three our programme going forward above).

Annex 1: Recent publications and National Evaluation work programme 2017 to 2018

Recently ERO has completed the following publications:



Partners in Learning: Helping your Child do Well at School

ERO website August 2016

An Evaluation of Stand Children’s Services: Children’s Villages

ERO website September 2016

Early Learning Curriculum

ERO website October 2016

School Leadership that Works

ERO website November 2016

Child Youth and Family Residential Schools National Summary Report

ERO Website November 2016

Appraisal as a Catalyst for Improved Learner Outcomes: One Year On

ERO website December 2016

Communities of Learning | Kāhui Ako: Collaboration to Improve Learner Outcomes

ERO Website January 2017

Communities of Learning | Kāhui Ako: Working Towards Collaborative Practice

ERO Website January 2017

Communities of Learning | Kāhui Ako in Action

ERO Website January 2017

School Trustees Booklet: Helping You to Ask the Right Questions

ERO Website February 2017

Extending their Language – Expanding their World: Children’s Oral Language Development (birth-8 years)

ERO Website February 2017

Food, Nutrition and Physical Activity in New Zealand Schools and Early Learning Services: Key Findings Report

ERO Website April 2017

Food, Nutrition and Physical Activity in New Zealand Schools and Early Learning Services: Effective Practice Report

ERO Website April 2017

MLP Glossary Māori Medium

To be released – mid 2017

The remaining publications planned for the 2017/18 year include:

  • Year 9 Plus – a series of reports on the concept trial from 2016-2019.
  • Teaching and Learning in the 21st Century – effective modern teaching practice
  • Teaching Strategies that Make a Difference – in schools with Years 4 to 8
  • Newly Graduated Teachers – in early learning services and schools
  • Responding to Auckland Language Diversity – in early learning services and schools, including agency, parent and student voice
  • Improvement for Learner Success - an On-line Tool to Guide School Improvement

Annex 2: ERO's structure & leadership team

Review services, other than for Kōhanga Reo and Kura Kaupapa Māori, are grouped into four regions and managed by four Deputy Chief Review Officers. Site offices are located in Auckland, Hamilton, Whanganui, Napier, Wellington, Christchurch and Dunedin.

Reporting services for Kōhanga Reo and Kura Kaupapa Māori are managed by the Deputy Chief Review Officer: Māori and carried out by Te Uepū-ā-Motu, a team of review officers with specific knowledge and skills in tīkanga and te reo Māori. ERO is a world leader of indigenous education evaluation, with five different methodologies and a specific immersion team.

ERO's leadership team

 Nick PoleNicholas Pole

Acting Chief Executive/Chief Review Officer

The Chief Executive/Chief Review Officer is responsible for the effective delivery of ERO’s functions.

Nicholas is on secondment to ERO from the Ministry of Social Development.


 Di AndersonDiana Anderson

Deputy Chief Executive: Evaluation and Policy

Diana leads the Evaluation Services Unit that produces ERO’s national evaluation programme, and provides evaluation and analytical services for ERO projects.



 Jeremy FranceJeremy France

Deputy Chief Executive: Corporate Services

Jeremy oversees ERO’s Corporate Services. He is responsible for the development and maintenance of an efficient and modern infrastructure that supports ERO in achieving its goals.


 Lesley PattersonLesley Patterson

Deputy Chief Review Officer: Southern/Te Waipounamu

Lesley has responsibility for ERO’s southern region.




 Ro ParsonsRo Parsons

Chief of Profession: Evaluation

Ro provides leadership and advice to our leadership team regarding ERO’s professional practice. She provides theoretical oversight of methodology and special projects as required. Ro works to advance ERO’s professional evaluation practice and methodology in relation to international and national research and best practice and to enhance ERO’s profile nationally and internationally as an evaluation agency.


 Lynda Pura-WatsonLynda Pura-Watson

Deputy Chief Review Officer: Waikato and Bay of Plenty/Te Tai Miringa and Te Uepū ā-Motu

Lynda is responsible for Te Uepū ā-Motu, ERO’s Māori review services, as well as the Waikato region.


 Graham RandellGraham Randell

Deputy Chief Review Officer: Northern/Te Tai Raki

Graham manages the northern part of the country.


Deputy Chief Review Officer: Central/Te Tai Pokapū

This role is responsible for the central part of New Zealand.

This position is currently vacant.