Briefing to the incoming Minister

The education system is a major contributor to New Zealanders’ social and cultural participation and wellbeing, as well as our economic prosperity and growth. As a nation going forward, we need an increasingly skilled population. Participation in the global workforce of the future will require an education system that nurtures adaptability, collaboration, problem solving, curiosity, digital fluency and creativity. Education is critical in supporting the aspirations of Māori and supporting the revitalisation of the Māori language. To achieve these goals, all learners must experience high quality teaching in continuously improving early learning services and schools.

Ko te Tamaiti te Pūtake o te Kaupapa
The Child – the Heart of the Matter

Section 1.0 – Who we are

The Education Review Office | Te Tari Arotake Mātauranga (ERO) is responsible for:

  • evaluating the implementation of government education priorities, programmes and policies across the system
  • evaluating the quality of education and care in schools and early learning services and across Communities of Learning | Kāhui Ako
  • supporting improvement in the performance and operation of our early learning services, schools, kura, Te Kōhanga Reo and Communities of Learning | Kāhui Ako
  • contributing to the evidence base about what works in support of decision makers and practitioners.

The driving force behind ERO is the belief that quality education is a right for every New Zealand child and young person.

Our review information is used to promote better educational practice, inform debate about national education issues and to provide assurance to you and New Zealanders about the quality of education provision in New Zealand. Our reviews enable parents to make informed choices and engage in their children’s learning pathway. Equally, we attempt to target teachers, leaders and trustees to share insights and evidence around effective practice.

ERO was established in 1989 as an independent government department under the structural reforms of the New Zealand education system, Tomorrow's Schools. It was created as a core mechanism in support of educational quality and accountability in a highly devolved model of schooling and early childhood education.

The structure and nature of our education policy settings, in particular the high level of autonomy afforded to schools and early childhood services, necessitates a robust accountability and assurance mechanism. ERO is established to perform this function at both the institutional and national level.

Under Part 28 of the Education Act 1989, Review of Educational Services, the Chief Review Officer has the power to administer reviews (either general or relating to particular matters) of the performance of early learning services and schools in relation to the quality of their programmes and to prepare reports on the results of such reviews.

As Minister, you can ask the Chief Review Officer to initiate a review or investigation into a particular issue or element of the early childhood or school sectors.

ERO’s independence from schools and early childhood services, as well as from agencies that set policy, funding and standards, enables us to provide assurance to you as Minister, the Government, parents, whānau and the broader community on the quality of our system and of education provision within New Zealand.

Section 2.0 – What we do

ERO’s core focus is ensuring we have a system that places the learner at the centre of everything we do. ERO’s work shines a torch on the quality of learner outcomes and on the conditions that are present to support these, at an individual, provider and system level.

Learner outcomes for ERO are not narrowly confined to achievement in specific subject areas, although ensuring our children and young people can demonstrate strong literacy and mathematics understanding, and develop the skills across the learning areas of the New Zealand Curriculum is vital. ERO aims to ensure all learners in our system have access to and can engage in quality learning, that learning is personalised, learners are supported to be confident in their language, identity and culture and that they leave school socially and emotionally competent and resilient.

Our work is underpinned by sound evidence of best practice. In the schooling sector our evaluation framework is based on a deep understanding of how schools improve and the role evaluation plays in that process. Similarly, we have developed research-based and contextually focused methodologies that promote improvement in the early learning sector and in Māori medium settings.

2.1 – National education evaluations

ERO’s legislative mandate and its independence enables us to provide frank advice to you, the Government and wider sector on a range of national education issues.

ERO undertakes a programme of evaluations designed to influence and inform the national education debate. The purpose of these evaluations is to act as change levers in education, raise systemic issues, enhance policy design and implementation, and drive improvements in education to raise student achievement. ERO’s national evaluations are used to inform and support you as Minister and to inform the work of the Ministry of Education and other government agencies.

We work with the Ministry of Education and other education and social sector agencies to ensure our evaluations provide useful and actionable information to support policy decision makers. Schools and early learning services also use our research and evaluation findings to develop and enhance their own practice. Our approach focuses on three key areas:

  • national evaluations–analyses system-level issues including sector performance, policy implementation and pre-tertiary educational practice
  • special reviews–where a matter needs to be reviewed and reported on outside of a regular review cycle
  • effective practice reviews and resources–where we identify and share innovation about effective practice.

National evaluation projects currently underway include:

  • Resource Teacher: Learning and Behaviour (RT:LB) service (an evaluation of all 40 RT:LB clusters)
  • Newly graduated teachers: preparation and confidence to teach
  • Response to language and cultural diversity - Auckland (an effective practice report)
  • Sexuality education (an evaluation of sexuality education curriculum in schools, including a focus on effective practice regarding gender and sexuality diverse students)
  • Quality of education in Teen Parent Units (an evaluation of 24 teen parent units)
  • Update of Te Whāriki (2017) (an evaluation of awareness and adoption in early learning services)
  • Quality of education in activity centres (an evaluation of 14 activity centres)
  • Te Kura Big Picture Learning pilot programme
  • Communities of Learning | Kāhui Ako case studies and evaluation programme
  • Leading innovative learning in New Zealand schools
  • Teaching approaches and strategies that work (series of school case studies about effective teaching practice)
  • Tikanga Ako Moroki o Niu Tīreni: Papa Kupu (a Māori dictionary to define modern learning practice in schools)
  • Tikanga Ako Moroki o Aotearoa: He Rārangi Kupu (compilation of modern learning practice, categorised in themes that reflect the environment connections)
  • Hauhaketia ngā taonga tuku iho kia puāwai ai (identifies what works well in a small evaluation sample, a joint initiative with the Te Kōhanga Reo National Trust).

2.2 – Institutional evaluations

Each year, our review officers undertake external evaluations in approximately 1,260 early learning services and 700 schools. This is equivalent to approximately 18 school and 25 early learning service reviews every week.

We are your eyes and ears on the ground and placed in a unique position to be able to provide you with real time information on how the system is performing. Our 152 review officers are professional evaluators who are mostly recruited from senior leadership positions in schools and early learning services. Our review approach aims to provide institutions with an opportunity to reflect on their performance, what they are attempting to achieve and draw attention to areas where things are not going well.

We place increasing emphasis on progress as well as point-in-time achievement, both for learners within a year, and from year to year. We are interested in progress across the breadth of the learning areas in the curriculum, and in the foundational key competencies; thinking, using language symbols and texts, managing self, relating to others, participating and contributing.

Depending on our findings, early childhood services and schools are usually on a three year review cycle, with this period extended to between four and five years for those services and schools we assess as being on a sustainable self-improvement path. Those services and schools showing limited conditions for success, will be placed on a more intensive review programme over one or two years.

In the 2016/17 year, 77 percent of schools were placed on a three year review cycle, 10 percent on a four to five year cycle, and 13 percent on a one to two year cycle. For early childhood services the general shape of the distribution was similar, with 81 percent on a three year cycle, 9 percent on a four year cycle and 10 percent on a one or two year cycle. While our data shows that some services and schools move from the three year cycle to a four or five year cycle, we would see this happening more frequently if the system collectively was striving for excellence.

In addition to our core focus on learner outcomes, during the course of an early learning or school review we audit both the quality of teacher appraisal systems (on behalf of the Education Council) and check the compliance of the school with their commitments under the Education (Pastoral Care of International Students) Code of Practice 2016 (as part of an agreement with NZQA). Where there is a school hostel we also review this provision.

ERO also undertakes reviews to provide assurance to you as Minister and the Ministry of Education (Ministry) for new schools and Partnership Schools | Kura Hourua. Reviews of the quality of provision for those choosing to home school their child(ren) are carried out at the request of the Ministry.

2.3 – Indigenous evaluation - Te Uepū ā-Motu

ERO is a world leader of indigenous education evaluation, with a dedicated immersion review team, Te Uepū ā-Motu. With skills and experience in education and evaluation, this team of reviewers has deep knowledge of te reo and tikanga Māori. ERO has developed specialist (Te Ao Māori) methodologies to undertake reviews in Te Aho Matua Kura Kaupapa Māori, Kura-ā-Iwi, Kōhanga Reo and other Māori medium settings.

2.4 – How the sector views our work

ERO’s role is to challenge the system, but in a way that adds value and supports improvement. Post-evaluation follow-up shows over 90 percent of early learning services regard their ERO review as useful or very useful and 94 percent said the review identified and confirmed their direction forward. Over 87 percent of schools reviewed felt their ERO external evaluation contributed to decisions on how to improve learning outcomes for students and 89 percent said ERO’s approach helped them to identify or confirm opportunities for more effective practice.

NZCER’s (New Zealand Council for Educational Research) regular national survey of schools shows ERO rates highly among trustees as a source of advice. The Kiwis Count survey of public satisfaction is carried out by the State Services Commission (SSC) across a range of government services. It assesses the level of public satisfaction with ERO’s school and ECE evaluation reports. ERO’s results for the 2015 and 2016 calendar years were 73 and 74 (out of 100) respectively[i].

Section 3.0 – Strengthening and modernising how we work

Since its establishment in 1989, ERO’s approach has shifted from an emphasis on compliance and externally driven accountability to one driven by improvement, learner outcomes, and complementary external and internal institutional level evaluation and accountability.

3.1 – Matching effort to need and impact

We want to place our resources where there is the greatest need and where we can have the biggest impact.

3.1.1 Transforming our approach to working with early learning services

Our effort is currently split 50-50 between reviews of early learning services and the school sector[ii]. Rapid growth in early learning services requires vigilance and our concerns remain with regard to the quality of provision in home based services and individual providers.

In the coming year, we propose to redevelop our early learning services review methodology in response to the newly revised curriculum statement Te Whāriki (2017).

We also want to explore how we might shift services from being adequate in terms of ERO’s quality expectations to being more strongly positioned. In order to support this shift we propose to redevelop our approach to working with the sector, with a specific review approach applied to larger employers operating multiple, licenced services.

3.1.2 Adopting a risk-based focus in our review cycles

Approximately 9 percent of all New Zealand schools (180-190) reviewed annually are identified as having sufficient concerns to justify being placed on a 1 to 2 year review cycle. Characteristics of this group of schools include those that are small, in rural areas, with new principals or recent leadership changes and schools serving lower socioeconomic communities. Most have high numbers of priority learners whose educational needs are profound.

The availability of ever improving data and tools gives us the opportunity to more clearly identify where the greatest need for attention lies. This information enables us to be more strategic in our approach. In this context, we consider how and when we review schools. We are currently developing a range of quantitative indicators of risk which will inform our review scheduling. This is an extension of the notion of earned autonomy that underpins the differentiated review cycle, but with greater specificity and informed by a wider range of more regularly collected data.

3.1.3 Capturing the voice of stakeholders in our review process

Better capturing the voice of stakeholders (parents, whānau, students, staff and the entity’s leadership) in our review process offers us a greater capacity to triangulate our judgements about the quality and performance of a school or early learning service. We propose to explore new approaches to better capture the voice of stakeholders through our review methodologies and explore opportunities for greater involvement by leaders in the review process.

3.1.4 Improving the way we report to parents, whānau and the community

ERO’s review reports fulfil several objectives. This includes reporting to a school/early learning service on how we judge their performance and areas for improvement. We also report to parents and the community on the overall quality of education. Researchers gather information from each review that contributes to system wide reporting on trends and issues. We are currently exploring how we can improve our reporting to parents on the quality of their child’s school or early learning service to ensure information is more accessible, and that our judgements provide greater clarity.

3.2 – Promoting collaboration

Across various international jurisdictions, there is increasing recognition of the power of educational networks and collaboration within and across institutions to drive improvement. Currently in New Zealand, the most visible element of this is the establishment of Communities of Learning | Kāhui Ako.

The potential benefits of effective community partnerships and networks include:

  • a collective focus on specific common performance and achievement targets
  • opportunities to address learning needs within a geographic catchment earlier in the life cycle of the learner
  • greater sharing of professional practice and expertise and opportunities for teachers to be part of a wider professional learning community, especially when they are employed in small to medium sized schools
  • a greater capacity to access specialist knowledge and capability that is distributed across a community or network
  • more coherent and individualised pathways for learners as they transition between institutional contexts
  • greater opportunities to coordinate specialist resources and adopt greater consistency in practice and approaches to working with specialist services (such as RT:LB or other learning support services)
  • fewer and clearer points of interface across a geographic community in terms of their interactions with other service providers
  • network efficiency benefits from economies of scale and reduction of redundancies in purchasing resources.

Leaders of collaborative networks have a critical role in the development of a compelling, collective vision and priority goals/targets that represent the perspectives and aspirations of all community participants, particularly students, parents and whānau. It is a collective commitment to the community of what matters in teaching, learning and student outcomes. All members of the community need to ‘buy in’ to this commitment for it to be successful.

As a country, there are lessons we can learn and insights we can gain from what is working and what is not. ERO has developed an initial resource Communities of Learning | Kāhui Ako: collaboration to improve learner outcomes that brings together international research findings about effective collaboration in action. We intend to build the New Zealand evidence base about what works, through case studies and evaluation work. We will continue to develop our methodology to address the particular challenges and opportunities of evaluating Communities of Learning | Kāhui Ako in a way that goes beyond simply evaluating the sum of the various institutional parts.

3.3 – Addressing persistent poor performance

ERO is currently piloting the School Turnaround methodology to address schools with repeated poor performance, or in rapid decline. This is a new rapid-cycle evaluation methodology designed to build evaluation capability where the school lacks the capacity to improve their performance without intensive planning, monitoring and targeted support. We are implementing the pilot in six schools, and if successful, we hope to scale up, using the approach in around 30 to 40 schools at any one time.

The new methodology requires an increased level of coordination between ERO and the Ministry of Education to ensure targeted and specific resources to support school improvement are timely and relevant. ERO’s expertise in diagnostic assessment, organisational performance and school improvement processes will support the development of a sophisticated and place-specific response in each instance.

Turnaround schools are required, with support from the Ministry, to develop a recovery plan and set termly goals and targets. An ERO national team evaluates the quality and sufficiency of the recovery plan and conducts ongoing, regular evaluation work to inform the next stages of school improvement. This is an intensive approach designed to increase the effectiveness of school planning and create actions to improve the quality of teaching and increase student’s educational outcomes. If improvement is not sufficient or sustained within a particular school, further measures and escalation will be considered.

As this is a pilot, we are incorporating robust, formative evaluation processes to ensure we continue to refine and improve the methodology-considering emerging successes and challenges. This work will also assist in identifying what additional support and resources are needed to adequately turn around a school that has struggled to perform over a long period.

We will keep you informed of progress both in terms of approach, but more importantly, with respect to the impact of this approach on these schools.

3.4 – Supporting quality Māori language teaching in English-medium settings

Māori language in English-medium schools involves students who are learning te reo Māori as a language subject, or taught the curriculum in the Māori language for up to 50 percent of the time (Māori Language Immersion levels 3-5). At 1 July 2016 there were 1,074 English medium schools providing Māori language to 161,381 students. Increasing numbers of students are coming into bilingual or Māori language immersion settings within English-medium schools. There is also an increasing movement of students between Māori‑medium kura and Kōhanga, and English-medium schools. These trends are forecast to continue.

However, there is minimal information about outcomes for learners of te reo Māori available to teachers, learners, parents and leaders. There is variability in the way we measure student outcomes, and existing student outcome measures do not necessarily align well with the important aspects of practice. Therefore it is a challenge for schools to be able respond to whānau aspirations and evaluate their own practice.

As such, we are undertaking a project to assist with the evaluation of Māori language in English-medium settings and develop an ‘outcomes framework’ to judge the quality of te reo Māori in these schools.

This project will begin by developing evaluation indicators specifically designed for use in English-medium settings where te reo Māori teaching and learning programmes are delivered. The evaluation indicators will support us to make decisions about a new evaluation methodology for Māori language in English-medium. This work is expected to occur mid-late 2018.

This work has the potential to improve the quality of te reo Māori instruction in English-medium settings, which is a crucial element in the ongoing revitalisation of te reo Māori. To be successful, we will need to extend our own internal capability and capacity to evaluate this area effectively in all school reviews.

Section 4.0 – Immediate priorities

During our initial meetings, you may wish to discuss how we can support your work programme and the Government’s commitments and priorities. We note that the government’s intentions for education are wide ranging. ERO can provide you with an independent perspective on many of the issues that you have highlighted as priorities.

We equally welcome the opportunity to discuss how we might support you in the evaluation of policy reform and change, particularly in areas where ERO has a strong mandate and/or insights.

As part of our early engagement with you, we will seek your input into our strategic direction, key strategic projects to advance the system, and national evaluations due for release.

Your input on our proposed programme of national evaluations for the 2018-20 period is critical.

In particular we would like to discuss how we can support the government to:

  • develop more effective ways of evaluating the performance of schools
  • review the learning support system
  • review the use of the Special Education Grant in each school and the adequacy of learning support provision relative to need
  • review home based early education
  • review the funding system for guidance counsellors in schools
  • support the role you may wish the Education Review Office to play in initial teacher education.

 We will also discuss progress on changes to our methodology of the Schools Turnaround pilot, and the Māori language in English-Medium project.

On an ongoing basis, we will alert you to any schools or early learning services that are at risk of under-performing and provide advice on interventions which we judge will turn the situation around. We will also draw your attention to new and emerging evidence from our research and evaluations which may have implications for further system improvement. 

Annex 1: Recent publications

ERO has released the following publications on its website:

Publication

Date

Early learning curriculum

October 2016

School leadership that works

November 2016

Child Youth and Family residential schools national summary report

November 2016

Appraisal as a catalyst for improved learner outcomes: one year on

December 2016

Communities of Learning | Kāhui Ako: collaboration to improve learner outcomes

January 2017

Communities of Learning | Kāhui Ako: working towards collaborative practice

January 2017

Communities of Learning | Kāhui Ako in action

January 2017

School trustees booklet: helping you to ask the right questions

February 2017

Extending their language – expanding their world: children’s oral language development (birth-8 years)

February 2017

Food, nutrition and physical activity in New Zealand schools and early learning services: key findings report

April 2017

Food, nutrition and physical activity in New Zealand schools and early learning services: effective practice report

April 2017

He Pou Tātaki: how ERO reviews hospital-based education and care services

June 2017

Year 9 plus 2016 – the first year (Year 9)

July 2017

Annex 2: ERO’s structure and leadership team

Review services, other than for Kōhanga Reo and Kura Kaupapa Māori, are grouped into four regions and managed by four Deputy Chief Review Officers. Site offices are located in Auckland, Hamilton, Whanganui, Napier, Wellington, Christchurch and Dunedin.

 Nicholas Pole - Chief Executive/Chief Review Officer

Nicholas Pole

Chief Executive/Chief Review Officer

 The Chief Executive/Chief Review Officer is responsible for the effective delivery of ERO’s functions.

 

 Diana Anderson -Deputy Chief Executive: Evaluation and Policy

Diana Anderson

Deputy Chief Executive: Evaluation and Policy

Diana leads the Evaluation Services Unit that produces ERO’s national evaluation programme and provides evaluation and analytical services for ERO projects.

 

Dr Ro Parsons-Chief of Profession: Evaluation

Dr Ro Parsons

Chief of Profession: Evaluation

 Ro provides leadership and advice to our leadership team regarding ERO’s professional practice.

 

Jeremy France-Deputy Chief Executive: Corporate Services

Jeremy France

Deputy Chief Executive: Corporate Services

Jeremy is responsible for the development and maintenance of an efficient and modern infrastructure.

Graham Randell-Deputy Chief Review Officer: Northern/Te Tai Raki

Graham Randell

Deputy Chief Review Officer: Northern/Te Tai Raki

Graham is responsible for the northern part of the country.

Lynda Pura-Watson-Deputy Chief Review Officer: Waikato and Bay of Plenty/Te Tai

Lynda Pura-Watson

Deputy Chief Review Officer: Waikato and Bay of Plenty/Te Tai Miringa and Te Uepū ā-Motu

Lynda is responsible for Te Uepū ā-Motu, ERO’s Māori review services, as well as the Waikato region.

Dr Lesley Patterson -Deputy Chief Review Officer: Southern/Te Waipounamu

Dr Lesley Patterson

Deputy Chief Review Officer: Southern/Te Waipounamu

Lesley is responsible for ERO’s southern region.

Currently vacant- Deputy Chief Review Officer: Central/Te Tai Pokapū

Currently vacant

Deputy Chief Review Officer: Central/Te Tai Pokapū

This role is responsible for the central part of New Zealand.

Endnotes

i During 2016/17, the SSC changed its survey reporting period from a fiscal to a calendar year base.

ii Enrolments in the 5,300 ECE services equal approximately 200,000, while there are around 2,530 schools in our system enrolling approximately 800,000 students.