In 2015-16, we delivered all evaluation and review outputs at expected levels. Our change and improvement focus as outlined in our Strategic Intentions document for 2015-19 was to:
Strengthen system performance
Support the development of Communities of Learning
Put the learner at the centre of our work
Provide more information to engage parents and whānau
Provide information and services for the profession
Build ERO's internal capability
Our progress against these focus areas is detailed in the following sections.
ERO aims to contribute to a measurable improvement in system performance by implementing evaluation methodologies that focus on the needs of learners 0-18 years. We have evolved our approach to external evaluation to highlight the importance of equity and excellence.
ERO's approaches, methodologies and indicators must remain fit for purpose. They must reflect developments in the system and be focused on those matters that are most important to support improved achievement from the early learning years right through to young adulthood.
In 2015/16, key achievements were to:
During 2015/16, ERO trialled the School Evaluation Indicators: Effective Practice for Improvement and Learner Success in schools across the country. We also received considerable, largely positive, feedback through our website and feedback surveys. As a result of the trial, the sector feedback and further review by the expert panel we have refined the indicators.
The indicators provide a common language for the interaction and dialogue between a school and ERO about the school's current state development, and future direction.
In the context of both internal and external evaluation, the indicators support evaluative thinking, reasoning, processes and decision making about the quality and effectiveness of education provision.
MoE and ERO jointly developed and published School Evaluation: How to Do and Use Internal Evaluation for Improvement. This tool replaces and improves upon previous guidance about how to do self-review. ERO also published School Evaluation for Improvement: Good Practice, during 2015/16. This resource demonstrates how successful schools do and use evaluation and inquiry to improve student outcomes.
These three publications bring together the outcomes valued for New Zealand students in schooling, what matters most in improving those outcomes, and how to do and use school evaluation for school improvement and student success. The evaluation indicators for schools were finalised in July 2016 and will be used from the third term in 2016. They provide the basis for each school's ongoing internal evaluation and ERO's external evaluation.
To generate more alignment across the system we want to embed the evaluation indicators to the extent that they inform initial teacher education and professional standards. This will help to create more capability and capacity in internal evaluation practice in early learning services, schools and communities.
In January 2016, ERO revised its evaluative question to sharpen schools' focus on accelerating student achievement. The revised question asks how well each primary school is accelerating achievement for Māori students. ERO's experience in external evaluation shows that when schools are accelerating achievement for their Māori students they do so for all their students.
The objective of the revised evaluative question is to evaluate how teachers and leaders in each school identify all children whose learning needs to be accelerated. We expect to see the development and implementation of a learning plan for each child to meet the standards required at Year 8. If a child's learning is not accelerated at this point it is very much harder for them to catch up later.
We began implementing this approach in primary and contributing schools in early 2016. In the period to 30 June 2016, reports were completed for 135 evaluations. In 36 (27%) of cases the school was asked to tell us how they intend to raise achievement for those children at the school whose learning needed to be accelerated. In 13 of these schools a workshop was scheduled to assist with the development of a plan.
Accelerated achievement has been defined as a student making more than one year's learning progress, over the course of a year, on a trajectory that indicates they will be achieving at or above the required standard at the end of year 8 or sooner.
We have moderated our evaluation findings for primary schools to increase consistency in our judgements, and this has led to delays in completing some evaluation reports. This is reflected in our timeliness results for 2015/16.
We have also changed our report format for the primary school reviews, with the objective of clearly explaining to parents and whānau the basis of our judgments. Many Board chairs and principals recognised the need for changes to the report structure and content. Their feedback has been used to make further improvements.
Our plan is to roll out the approach to intermediate and area schools in 2016/17 and to secondary schools in 2017/18.
Our 2015/16, national evaluation programme included several national evaluation projects focused on schools' identification of, and acceleration of progress and achievement for, learners at the highest risk of inequitable outcomes. These resources are provided to schools to support them to improve their practice.
In this evaluation of 41 secondary and 310 primary schools we found that two thirds of the schools set effective targets that focused on underachievement. About 45 percent of the schools took effective actions that satisfactorily accelerated learning for target students. Secondary schools were generally less effective in both setting targets and taking actions than primary schools.
ERO made recommendations to boards, school leaders, teachers and the Ministry of Education, intended to improve student achievement through targeted actions. Two key conditions were required for effective target setting. The first was goal setting: having optimum challenge in the targets to stretch expectations for success; and the second was having maximum visibility of targets so that those needing to take actions shared responsibility for the actions. Coherence and alignment were two key qualities that distinguished the actions of the successful schools from those of the less successful schools. Leadership was critical to creating this coherence and alignment to raise achievement. Coherence meant plans made sense to those implementing them in practice. Alignment meant the actions of a range of people had a common purpose.
This resource brings together findings from ERO's recent national reports to outline what works to accelerate progress for students at risk of underachieving in primary schools. We share approaches schools have taken where progress was accelerated and schools were able to extend their practices to help more students succeed. Innovative schools focus on inequity within their student population, resulting in improved outcomes for all students.
As part of the Prime Minister's Youth Mental Health Project ERO published an effective practice report describing practices that highlight the variety of ways a group of primary and secondary schools promoted and responded to student wellbeing.
In these schools programmes were planned as part of the curriculum and students were expected and supported to show leadership. They knew that their opinions were heard and were used in decision making.
School leaders recognised that enhancing student wellbeing was a shared responsibility and that partnerships with whānau, the community and relevant support services were vital.
In addition, the inclusion of agreed values, integrated into everything school leaders and teachers did, provided cohesion across systems, attitudes and actions and resulted in clear and consistent expectations for behaviour.
This resource was developed as a companion resource to the Wellbeing effective practice report. It highlights the importance of schools promoting the wellbeing of all students as well as the need for systems, people and initiatives to respond to wellbeing concerns for students who need additional support.
It encourages school leaders and teachers to think about how they promote the wellbeing of all students in their school community and the way in which they respond to emerging wellbeing concerns.
During 2015/16, ERO developed and published a revised methodology for reviewing home-based early childhood education and care services. The new methodology reflects a collaborative effort between ERO, representatives from the home-based early learning sector, and researchers with an interest in home-based education and care.
In October 2015, a new methodology was released for hospital-based education and care services. It reflects the unique characteristics of hospital-based services, and ERO's focus on a complementary approach to evaluation.
The draft methodology draws on He Pou Tātaki, the review methodology for centre-based services, and Ngā Pou Here, the review framework in He Pou Tātaki.
The revised approach will support improved service performance and accountability, and focuses on how well placed hospital-based services are to contribute to children's learning, and promote their wellbeing. The draft methodology was trialled in hospital-based education and care services scheduled for an ERO education review in Term 4, 2015.
ERO knows that highly effective leaders and teachers need to collaborate and support each other to achieve outcomes. We will increasingly share information about the contribution Communities of Learning are making to enhance pathways and achievement for all learners.
In 2015/16, key achievements were to:
Communities of Learning are the cornerstone of the IES initiative. The initiative requires new ways for boards, leaders and teachers to work together. ERO's work is intended to share effective practice in a way that helps new Communities of Learning to be effective as quickly as possible.
ERO has contributed to the development of the evaluative capability within newly formed Communities by providing contextual reports. These reports summarise Public Achievement Information for schools within each community and the findings in recent ERO reports. They are used by each community to facilitate the development of its achievement challenges.
As at May 2016, there were 117 Communities of Learning established across the country involving 1,006 schools - about 40% of the national total.
During 2015/16, ERO has completed reports for 84 Communities and provided 14 workshops to help a number of them build capacity to evaluate their own effectiveness. This new work was allowed for in ERO's programme for the year and was offset by a reduction in the number of completed school reviews.
ERO also supports schools approved to offer the Principal Recruitment Allowance by writing a summary report that details the performance areas to be addressed and the actions undertaken, maintaining a liaison role with the board and principal and determining the most appropriate timing for the next review.
The education system must be configured around the needs of the learner. With the learner at the centre of a coherent system every child from 0-18 years will be engaged and progressing well.
In 2015/16, key achievements were to:
The government has supported the development of Innovative Learning Environments through investment in the technology infrastructure and flexible learning spaces in schools. In order to establish a true Innovative Learning Environment every school must now focus on developing the pedagogy to support a future-focused curriculum; one that brings to life the New Zealand Curriculum and the vision for our young people to be confident, connected, actively involved lifelong learners.
The glossary provides a common basis for conversations about 21st century best practice in teaching and learning. Feedback on the glossary has been sought from a wide range of audiences, including the Ministry of Education, academic experts, and learning communities that have demonstrated modern learning best practice.
The trial version of the glossary is being used within ERO as a reference document to support professional learning around modern learning practice. While this version is not intended for parents, it forms the foundation for future work.
This guide to using internal evaluation for improvement purposes is published jointly by ERO and the Ministry of Education as a companion to School Evaluation Indicators: Effective Practice for Improvement and Learner Success and is supported by Internal Evaluation: Good Practice. It describes what effective internal evaluation is, what it involves, and how to go about it in ways that will enhance educational outcomes for students.
The guide draws on current knowledge about internal evaluation, and on case studies of New Zealand schools that have used internal evaluation to inform the development of strategies that have been successful in raising achievement, particularly that of Māori and Pacific learners.
The key purpose of internal evaluation is to make evidence based changes to processes and practices that will positively impact on children's learning.
This good practice report shows how thirteen high performing schools carried out internal evaluations to improve teaching and learning and raise students achievement. ERO identified the 13 schools through an analysis of their achievement data and the findings from their regular ERO review.
In all of the 13 schools, internal evaluation was valued by leaders and trustees, who made sufficient time and resources available for genuine, improvement-focused inquiry into the areas that mattered most for their learners.
This guide acknowledges and describes effective practice in services where the curriculum integrates the breadth and depth of mathematics.
Children in these services experience many and varied mathematical learning experiences that respond to and extend their strengths and interests.
A balance of child-initiated learning experiences and deliberately planned activities provide a platform for teachers to extend children's developing of mathematical concepts that will be critical for their future learning success.
ERO found two critical factors underpin this effective practice - pedagogical leadership and teacher knowledge.
Effective pedagogical leaders understand how to integrate mathematics into the curriculum to best support children's learning. Skilled teachers intentionally plan experiences to extend children's developing mathematical understandings.
ERO investigated how well services ensured they met current and future legal requirements for safety and wellbeing.
We found that most services did a good job of managing requirements. A small number of services were not up to date and some were not able to identify the person responsible for monitoring changes.
In light of the increasing use of digital technology ERO investigated how early learning services considered the privacy principles when collecting, storing and sharing information. Most services did a good job of managing requirements for children's privacy. Many services remedied isolated areas of non-compliance by the end of their review, for example by locking away children's health records where they hadn't previously.
Vocational Pathways are part of the Government's Youth Guarantee scheme, which is about improving the transition from school to work by providing a wider range of learning opportunities, making better use of the education network, and creating clear pathways.
In Terms 3 and 4, 2015, the Education Review Office evaluated how well 35 secondary schools were using Vocational Pathways to provide students with a responsive, relevant curriculum. A few schools were using Vocational Pathways as a way of moving towards curriculum change. We found that leaders in these schools had found that implementing Vocational Pathways had necessitated some changes to school structures in order to better support their implementation of a more relevant, responsive curriculum. Other schools had made a philosophical shift towards a pathways approach, but at the time of this evaluation, were still considering how best to move ahead with the changes they wanted to make.
ERO provides high quality evaluation for Māori medium education settings. ERO has worked with Ngā kura-ā-iwi to complete the development of culturally responsive evaluation methodology and indicators for kura who affiliate to this organisation.
The completion of this work acknowledges that both partners have contributed their different and equally comprehensive knowledge and alternative perspectives, and share a common focus on successful outcomes for children.
The more informed parents and whānau are about quality learning outcomes, the more they will demand of the system and the better it will perform.
During 2015/16, we:
Educationally powerful connections are relationships between schools, parents, whānau and communities that improve education outcomes for students. Research studies have shown that educationally powerful connections are critical to learner success.
This evaluation was undertaken in 256 schools reviewed in Terms 3 and 4, 2014. ERO evaluated how well schools worked with parents, families and wha-nau in their response to students at risk of underachievement.
We looked for stories about successful working relationships where schools, with parents, had helped students move onto a successful pathway from a less successful one. We found that the most important factors for success with educationally powerful relationships were:
In 2014, ERO undertook a cluster review of five Puna Whakatupu as part of their scheduled education reviews. We found that all five shared similar kaupapa Māori philosophies and were highly effective in working with children and their whānau. Where these elements were strong, children were confident and happy, had a strong sense of place and were enthusiastic learners.
The following elements strongly influenced positive outcomes for tamariki:
This booklet is a resource for all parents and caregivers who have infants and toddlers in an early childhood education (early learning) service, or who are thinking about enrolling their child in one. It sets out:
The booklet is designed to help parents and caregivers to ask the right questions of their current and prospective early learning services and make more informed choices.
In July 2015, the Niuean Director General of Social Services, with the approval of the Minister of Social Services, approached ERO to undertake an evaluation of the provision of education in Niue.
The Director General's request to ERO had the support of the New Zealand High Commission and the New Zealand Government under the Education Sector Support Programme.
The draft ERO report was written following a discussion of emerging findings on-site in Niue and off-site in New Zealand. At the Director General's behest, ERO's draft recommendations were shared with the contractor charged with drafting the new Education Support Plan. The report was finalised after discussion with the Director General during a visit to New Zealand in June 2016. A follow up visit to provide the school with next steps is planned for September 2016.
High quality leadership and learner-centred teaching are central to modern learning practice and critical to equity and excellence in learning outcomes.
To generate more alignment across the system we want to embed the evaluation indicators to the extent that they inform initial teacher education and professional standards.
Given that our review officers are in schools and early learning services all year round we are uniquely placed to assist out education agencies.
During 2015/16 we:
In 2015, the Education Council contracted ERO to undertake an audit and moderate at least 10 percent of practising certificates. The underlying reason for the audits is to improve teaching performance by strengthening appraisal. Appraisal is a tool to improve the quality of teaching and leader capability, and strengthen the school/service focus on improving outcomes for students.
During the 2015-16 year, ERO audited just over 4,000 practising certificates across 1,682 schools and early learning services. Audit outcomes and trends were reported to the Education Council monthly and ERO has met regularly with the Education Council during the year to discuss the findings in depth.
During the year we have continued to build upon the professionalism of our staff and the high levels of engagement and motivation to do the very best for the children of New Zealand. The success of ERO will continue to be based on the skill of our evaluators working with schools and early learning services, supported by great leadership and highly competent advisory and support staff.
We finalised our operating model design principles and continue to use these as a framework that helps guide the development of policies, processes and practices in a consistent way.
High quality, professional evaluation practice is critical in enacting ERO's Purpose Statement: Our evaluation insights are a catalyst for change so that every child achieves success as a lifelong learner.
Our Professional Practice Strategy guides ERO's capability development. The strategy outlines a coherent multi-faceted approach to developing leadership and evaluation practice in the field.
ERO's school evaluation indicators (finalised July 2016) and accompanying resources have been well received by the sector and are providing a valuable framework for defining educational quality and effectiveness. ERO has designed and delivered a programme of workshops internally and to the sector, including the Ministry of Education, to support this new suite of evaluation resources.
Key actions over the last twelve months have included:
Post-graduate programme in evaluation
ERO has established a professional relationship the University Of Melbourne Graduate School Of Education. The first cohort of ERO evaluators is being supported to undertake post-graduate study in evaluation.
Leadership and Talent Management
Our operating model design principle is that “We will lead by example"
Over the past twelve months, ERO has actively participated in the Social and Justice careers boards and the Education Sector career board. Through this we have continued to seek opportunities for our staff.
ERO uses the State Services Leadership Success Profile as a basis for thinking about leadership and incorporating these aspects into the work done in this area. During the year 13 of our tier 2 and 3 staff had the opportunity to participate in the Leadership Success assessments process with follow-up coaching. This forms the basis of their development plans over the forthcoming periods.
We have continued to use project and national report activities to allow staff opportunities to be involved in different parts of the business and broaden their experiences.
ERO's evaluation expertise continues to be recognised through opportunities and invitations to provide presentations in national and international contexts. There is a significant demand for presentations that focus on ERO's approach to evaluation as well as evaluation findings related to specific policy areas.
In September ERO held a professional two-day forum in Auckland.
This was a first for ERO in a decade and the focus for the Forum was “ERO a Catalyst for Change" opened by our Minister and including speakers from the various parts of the sector. It gave staff an opportunity to focus on the needs of the people we serve and how our work can be best carried out to meet those needs.
There was a practical workshop for Evaluators led by Dr Alton-Lee and a complementary session for non-evaluators focussing on their support for the business of the Office.
The response was terrific and it will be repeated.
ERO has worked successfully within fixed baseline funding for a number of years and continues to manage all aspects of expenditure in a careful manner. Key initiatives include:
In January ERO reorganised its regions to create a further region based around the Waikato /Bay of Plenty (Te Tai Miringa). This new region was split out from the previous Northern region and recognises both the growth in complexity and population in the upper half of the North Island. No new position were created as the opportunity was taken to colocate both the Waikato/BOP region with ERO Māori Immersion Unit and have one manager for both.
ERO launched its new-look website in May 2016 in collaboration with the Ministry of Education (MoE). Sharing resources with the Ministry has enabled us to manage the change at low cost and within our baseline. The benefits to ERO and to users of the website include:
The new website is being managed by the Ministry of Education web team, who will also manage ongoing maintenance and future development. ERO will remain responsible for day-to-day maintenance ofthe site and has full autonomy over content
ERO participates proactively in All-of-Government (AoG) initiatives, and ensures that:
In addition ERO is subject to Cabinet Circular (16) 3 issued on 25 May 2016, which sets out requirements for the implementation of the New Zealand Business Number (NZBN) by the end of December 2018. ERO has established a project team to ensure these requirements are met and is currently identifying the inventory of business-facing systems, processes and forms to incorporate NZBN.
ERO's Executive Leadership Team is responsible for identifying, assessing and managing key strategic and operational risks by reference to an organisational risk management framework.
Our aim is to be recognised as a professional evaluation agency, effectively promoting equity and excellence in the New Zealand Education System. We recognise the need to act collaboratively and positively to influence education sector priorities, programmes and outcomes over the long term.
In 2015/16, our Risk and Assurance Framework categorised assurance needs in terms of four major risk areas:
The integration of national quality assurance findings into the national professional learning and development programme is contributing to the quality and consistency of evaluation practice. ERO intends to continue this programme of regular internal evaluations to identify and guide further improvements.
In line with the Public Service Equal Employment Opportunities Policy, ERO strives to:
ERO continues to take action to ensure equality and diversity in all aspects of its business.
In late 2015, ERO was named joint Silver Distinguished winner of the YWCA's Equal Pay Awards. The award was a tribute to the collaborative work between ERO and the PSA in addressing the pay inequity for employees.
ERO and the PSA continue to meet regularly at national and local levels as part of the Working Relationship Agreement. Both parties find these meetings useful.