ERO Insights - Term 3 2019

Head shot of Nicholas Pole, Chief Review Officer

Kia ora tātou,

Indicators of quality for early childhood education: what matters most

The case for high quality early childhood education is well proven. Children learn at a faster rate during the first five years of life than at any other time. It is at this stage that critical cognitive, social and emotional competencies are developed which contribute to future learning and lifetime outcomes. Support of family and whanau as key contributors to positive early experiences, along with access to high quality early learning services are therefore critical foundations to New Zealand’s social, cultural and economic future.

As a system New Zealand’s early learning sector has many positive features. These include high levels of participation in early childhood education, providing a common direction for quality early learning through the ECE curriculum, Te Whāriki, and a system which supports diversity of provision in response to different needs and contexts.

The recently released OECD report Education at a Glance (2019) shows that with 50 percent of children aged under three in an early learning service, New Zealand sits 8th out 30 countries for participation of young children in early learning services. At 95% participation for those aged 3 to 5, we are 11th out of 42 countries in terms of participation.  In relative terms New Zealand staffing and funding levels for early learning services also sit in the top group of nations for whom the OECD collects data. These trends reflect a rapid expansion in participation over the past decade. Along with these increases has been a substantial change in the profile of early learning provision.

Despite significant participation increases and associated increases in investment over the past decade, ERO continues to identify only one in 10 early learning services as “Very Well Placed”, and around 5% of services of concern. We also see later and lower participation for those from poorer communities and a disproportionately higher proportion of services who are of concern serving these communities. As a consequence, we may not necessarily be gaining the full benefits of early learning investment described above due to both quality and equity concerns. These issues have been centre stage in terms of the deliberations of the Ministerial Advisory Group’s work on an Early Learning Strategic Plan for New Zealand.

In contributing to greater clarity around our understanding of quality early childhood education, ERO has released for consultation a set of draft indicators as part of  a framework for evaluation and improvement in centre-based early childhood services: Ngā Ara Whai Hua. The indicators of quality focus on valued learning outcomes for all children as articulated in Te Whāriki, foreground the relationships required to enact Te Tiriti o Waitangi/ the Treaty of Waitangi, and focus on the conditions that are most likely to provide equitable opportunities to learn and promote children’s learning. The indicators are underpinned by current research and evaluation evidence about the factors that we know make the greatest difference in learner outcomes. Our intention is that these indicators become the basis by which we as a system judge service quality. Our purpose in developing these is not only to support ERO’s judgements in its reviews of centre-based early childhood services, but also for them to be used by centres in their internal evaluation, planning and priorities for ongoing improvement. We are seeking feedback (here) on our draft indicators from now through to 30 November 2019.   

Leadership Partnership Programme Launched

ERO’s Leadership Partnership Programme will provide successful school leaders with the opportunity to train and join review teams as partners in ERO’s external school evaluation process. They will be partners in ERO’s external school evaluation process.

This programme is intended to build strong enduring partnerships and networks between ERO and the sector, build school leaders’ understanding of evaluation for improvement through upskilling practitioners in ERO’s work, and enhance the review process for both schools and ERO. It also creates the potential for leaders to develop insights into their own schools through the opportunity of contributing to the review of others. We see the programme as bringing more diverse expertise to the evaluative process, and allowing ERO to benefit from the valuable insights and expertise of current practitioners.

Through Term 4, we will call for expressions of interest from senior school leaders who may wish to join with ERO in the school review process. ERO will trial the initiative over the next 12 months. We intend to start with a small number of senior leaders and grow sector involvement over time based on what we learn through the initial phases of the programme. Those selected to work with us will be provided with professional learning and development in ERO’s methodologies and procedures prior to working as part of an ERO school review team. We believe this initiative is a great opportunity to bring the sector and ERO closer together, to strengthen understanding about ERO’s frameworks and methodologies, and to provide leaders with an opportunity to observe the practice of other leaders and practitioners across the sector and importantly contributing to the review process with the lens of a current practitioner.

Further detail about the expression of interest process will come out to schools early in Term 4.

If you are interested in becoming a Leadership Partner please email Shelley Booysen, ERO’s Manager Sector Initiatives.

Overall Findings and Judgement for ERO’s external school evaluations

ERO’s Overall Findings and Judgements for schools has been updated.  The newly updated framework focuses more strongly on what contributes to positive and valued outcomes for children and young people. This aligns strongly to ERO’s Evaluation Indicators and provides greater coherence between each of the judgement criteria. Reviewer Officers will begin using this framework with schools from the beginning of Term 4, 2019.

Recent ERO work with the School Sector

In this issue of INSIGHTS we also profile recent research into schools that are achieving great outcomes for learners from low socio-economic communities, what we have found in the provision of Pacific bilingual education in New Zealand schools, where we are at as a system in the effective deployment of Professional Learning and Development (PLD) and our observations of NCEA.  

Ngā mihi, 

Nicholas Pole
Chief Review Officer 

Professional Learning and Development in schools

As schools move into Term 4, many will be completing their planning for the 2020 year, and in particular determining the areas they will wish to invest in for Professional Learning and Development (PLD) through the 2020 year.

ERO recently conducted an evaluation of PLD in schools, following up on two previous studies (conducted in 2009) that found that schools needed to strengthen their strategic decision making about PLD and better evaluate its effectiveness (PLD in Primary Schools and PLD in Secondary schools). Recent changes to the way schools apply for centrally-funded PLD respond to some of what was found in that work.

PLD at its heart is a change process, aimed at lifting teacher capability and through this extending and improving learner outcomes. While PLD can be undertaken in many different ways, at the centre of every effort should be improving teaching and learning. There are generally four distinct phases in PLD:

  1. Initiation and selection
  2. Programme delivery
  3. Embedding practice
  4. Assessing impacts

In our recent work, ERO visited 242 schools in Terms 3 and 4 2018, speaking with school leaders responsible for planning PLD and examining related documents.

In comparison to our 2009 findings, most school leaders were using data to determine their priorities for PLD, and were generally strategic in their PLD choices, linking it to wider priorities for the school. We found that PLD choices were based on a mix of quantitative and qualitative evidence and balanced with external advice when data was poor.

A number of the schools we studied lacked strategies (e.g. teacher learning groups or observation and feedback sessions) to ensure that what was learnt through PLD became truly embedded into teaching practice. Equally strategies supporting the continuity and sustainability of PLD in the face of staffing changes (such as refresher programmes, embedding PLD elements into new staff induction) were areas many schools needed to consider.

Where schools were generally weak was in understanding the impact of PLD on changes or shifts in teaching practices and whether the PLD was leading to improvements in student learning outcomes. Around one third of schools had not considered the impact of their PLD investment at all. 

Schools who were assessing the impacts of their PLD investments tended to focus merely on teachers’ confidence and knowledge of curriculum content, not its application or impact.

ERO noted in particular that small and rural schools struggled to gain access to appropriate PLD. 

Pacific bilingual education

In August, ERO released a report about the Current Provision of Pacific Bilingual Education.

There is growing evidence, both within ERO and internationally, that culturally located learning results in accelerated learning and higher achievement not only for the target group, but for all learners in schools. New Zealand’s Pacific population is projected to be 480,000 by 2026, mostly in Auckland, though regional Pacific populations are growing.

Our report into 30 schools with Pacific bilingual education programmes found that these programmes tend to be developed locally and funded through schools’ baseline funding. While a locally based curriculum has been shown to be beneficial to learning, there was a lack of appropriate resources, particularly for assessment.

Schools found it difficult to identify and create meaningful bilingual pathways through into the senior secondary school. The level of Pacific language immersion tended to decrease alongside this challenge. We also found that continuity and sustainability of programmes was highly dependent on the availability of teachers who were confident in teaching a Pacific language and, in the case of immersion programmes, teaching across curriculum areas through the language. 

ERO has identified a need for further support to develop a strategic and consistent approach across schools and fit-for-purpose resources, reflecting current research literature and best practice for pacific bilingual provision. A particular area for focus as a system is the need for comprehensive assessment tools and strengthening of assessment practices in support Pacific bilingual provision.

Given New Zealand’s high and growing Pacific population, supporting Pacific students to realise their potential will benefit all. 

Schools that beat the odds

Most low decile schools have lower NCEA achievement compared to higher decile schools, which serve a wealthier catchment. But some schools in poor communities are bucking the trend and achieving at least as well, sometimes better, when compared to schools with children from more advantaged backgrounds. What are these schools doing to lift and sustain achievement?

ERO recently conducted desktop analysis into this question, as a preliminary to further study. We found that decile 1-3 schools that were bucking the trend, were more likely to be Kaupapa Māori or state-integrated religious schools, in other words schools with a particular philosophical underpinning or approach.

We examined NCEA level 2 or above achievement in 115 decile 1-3 secondary schools of varying sizes and locations in 2011/12 and 2016/17:

  • Nine of these schools had persistently high achievement in 2011/12 and 2016/17. These schools were in the top 25 percent of all secondary schools (including high decile schools) in terms of achievement in NCEA level 2 (91 percent of school leavers achieved NCEA level 2 in these schools in 2016/17). All nine of these schools were Kaupapa Māori or state integrated religious schools.
  • Another group of schools (32) had started the period in 2011/12 with relatively low NCEA level 2 achievement (60 percent of school leavers) and then experienced significant improvement in achievement by 2016/17 (78 percent of school leavers). Among these schools, one in three were Kaupapa Māori or state-integrated religious schools.
  • Among the group of low decile schools with the lowest NCEA level 2 achievement in 2011/12 and 2016/17, 90 percent of the schools were state schools.

What are these kura and state-integrated schools doing that allows them to beat the odds for NCEA achievement? Why are low decile state schools not doing as well as low decile Kaupapa Māori or integrated schools?

Firstly evidence suggests that they appear to better cater for Māori and Pacific students, have strong links to the community and provide for greater opportunities for parents and teachers to meet regularly. School leaders also appear to know their students’ families better and vice versa, which appears to strengthen the relationships between the school and the community. In the case of Whare Kura it may also reflect that these learning environments enable Māori learners to enjoy teaching which strongly affirms language, culture and identity.

These schools have a greater proportion of their roll compared to non-integrated schools where enrolment is a deliberate choice and as such may be able to attract parents/whānau who are more focused and more supportive of their child’s education.

ERO is planning to look more closely at low decile schools with accelerated achievement in the coming year, and hopefully identify common approaches which can be replicated across the system.

An Example

As part of ERO’s Improvement in Action Te Ahu Whakamua series we visited Invercargill Middle School to investigate how they were connecting with their families by sharing what is going on at school and getting the children to show and teach their parents what they have been learning in the classroom. Read more

Graph comparing deciles against high and low performers. It is titled Percentage of school leavers with NCEA level 2 or above. It is a stacked bar chart with two bars, teal green fro High performers and Orange for Low performers.  the x- axis runs from 0 - 100 ins steps of 20.  From the top which is Decile 1-3 the high performers are 83% and the low performers 18%, Decile 4-7 is high performers 83% and low performers 31% and finally Decile 8-10 is high performers 97% and low performers 40%.

Proportion of school leavers with NCEA level 2 or above Decile 1-3 schools. This is a scatter chart divided into four quarters. The Y axis runs from 0 at the bottom to 100 at the top in the other steps are 20, 40, 55, and 80.  The x axis runs from left to right and its steps are 0, 20,46,60,80 and 100. The four quarters are from top left clockwise Accelerating achievemnt, Persistent high achievement, falling achievement and Persistent low achievement. The scatter point are a solid orange square representing Intergrated, Orange square outline represnting Kaupapa Maori and a gray circle for Non intergrated.  In this chart most Intergrated and Kauapapa Maori are in the Accelerting and High achievement with a few scattered thru low and falling achievement.  The overwhelming majority of Non intergrated falls in the Persistent low achievement quarte.

NCEA Observational Studies

Last year, ERO partnered with the Ministry of Education to research how NCEA impacted on curriculum design, pedagogy, assessment strategies and student wellbeing in schools and Tertiary Education Organisations (TEOs). The study provided input to the review of NCEA as part of the government’s Education Conversation – Kōrero Mātauranga.

ERO’s recently released report NCEA Observational Studies provides generalised profiles of nine different school types and three TEOs, with the intention of identifying the range of innovative practices, triumphs and challenges they faced around the implementation of NCEA. Data to inform the report was collected from visits to different types of schools, together with evidential files of similar schools reviewed as part of ERO’s regular review cycle. TEOs were visited separately.

The report describes themes common to all schools and some for the TEOs. While the challenges tended to be similar across education providers in widely different communities, the strategies they used to address them were many and varied.

These themes and profiles provide an opportunity for deeper understanding of the impact of the assessment system on schools and TEOs, and the complexity of challenges experienced by schools and TEOs when using NCEA.