Findings

In identifying secondary schools that were effective in raising achievement, the key features considered were:

  • the use of evidence to inform the school’s analysis of its patterns of success or failure
  • the extent to which school personnel had a well-informed understanding of why both individuals and groups of students have previously succeeded or failed
  • the effective implementation of curriculum, pastoral care and/or careers initiatives targeted at improving achievement
  • changes in student achievement, especially in NCEA Level 2, but also changes across other NCEA levels and improvements made in Years 7 to 10 (or 9 to 10, where appropriate). In particular, emphasis was put on changes in the achievement of Māori and Pacific students.

The overall results show that 10 of the 40 schools had inquiry and improvement processes that made a significant difference to student achievement. A culture of inquiry and responsiveness successfully operated across each of these schools. The efforts of staff at these schools led to improvements in student achievement, including that of targeted Māori and Pacific students.

ERO identified that 14 of the 40 schools had inquiry and improvement approaches that were ‘somewhat effective’. These schools showed some evidence of improvement, although these changes did not consistently or significantly affect most students, especially priority learners. While some staff at these schools had developed useful approaches, these schools had yet to develop a consistently effective school-wide approach to investigating and responding to student achievement information.

Sixteen of the 40 schools were identified as having inquiry and improvement processes of limited effectiveness. ERO found little evidence that their efforts were changing the school’s achievement patterns. Staff often lacked urgency and focus in responding to student achievement issues. School-wide guidelines or processes to support staff to understand and respond to student achievement issues tended to be weak.

Schools with effective inquiry and improvement processes

Ten of the 40 secondary schools in this evaluation demonstrated effective inquiry and improvement processes. These schools were a variety of sizes and from both rural and urban settings. Four were high decile schools, five middle decile schools and one was low decile.

These schools each had a culture of inquiry and a focus on improving achievement. Staff demonstrated an urgency to both identify achievement issues and develop responses that would improve student learning and engagement. They were focused on identifying ‘who’ were the students underachieving, ‘what’ needed to change, and ‘how well’ new school initiatives had contributed to any gains in achievement. The leaders and teachers were confident that they could work with students and their families to change things for the better. This belief was supported by good levels of teamwork and coordination across the school.

In addition to the above qualities, these schools had:

  • levels of academic achievement that were rising and/or above schools of a similar decile, and Māori student achievement in NCEA Levels 1 and 2 (but typically not University Entrance) was comparable to that of all leavers [12]
  • leadership that underlined the importance of investigating and responding to achievement information, including evidence of student underachievement
  • a shared commitment from staff to improve the status quo and a relentless focus on improving student achievement
  • a focus on engaging individual students at risk of not achieving and helping them succeed
  • made a difference for targeted groups of students, including Māori and Pacific students
  • trustees who received information about student achievement and used this to inform school strategies and operations
  • made some modifications to the curriculum to respond to the identified needs of students
  • well-developed school-wide frameworks, guidelines and expectations for inquiring and responding to student achievement information
  • professional development and support for teachers to develop skills in analysing data and to manage teaching as inquiry projects
  • an integrated approach to achievement information that was not limited to academic achievement but also included a range of student aspirations (e.g. vocational, sporting and cultural)
  • coordinated pastoral care and careers systems that were effective at identifying and responding to the needs of students, including developing support structures for student learning and vocational pathways
  • an increasing focus on involving families/whānau in strategies to improve student achievement
  • an emphasis on students as ‘self-managing’ students who can take responsibility for their achievement.

These features, along with examples from the effective schools, are discussed in more detail under the three sub-headings below.

Inquiry and improvement process for schools

This diagram includes key self-review questions for schools as part of an overall strategy which schools can use to improve achievement. The process below and the associated questions are based on the findings of this report. The key message for schools from this diagram is the importance of schools using data to build their understanding of where they can improve teaching and learning. The key reason schools should be collecting and analysing data is so that they can identify improvements.

this diagram this the montioring effectiveness framework it includes the key questions for self review for schools. It revolves around four principles being annual student achievement, analysing results, developing solutions and initiatives. Off initiatives four questions arise they are curriculum innovation/change? professional learning and development? Families/whanau engagement? and new careers programmes pastoral care and support? Off analysing results three questions arise they are, Who is/isnt succeeding and why/why not? What are the key issues affecting achievement? and what is occuring for key groups? Maori, Pacific boys?  Off developing soloutions one question arises it is do we need to re-examine curriculum structures, options, the timetable?

Depending on the issues identified schools should develop curriculum, pastoral or careers initiatives to better respond to students’ strengths, interests and aspirations. Innovation across these areas may be needed to improve learning for students who have traditionally not been well-served. In many schools this includes many Māori and Pacific students. The relationships that schools develop with families and whānau can help staff learn more about their students and develop their understanding of what can work to improve achievement.

High levels of coordination and responsiveness

As noted in other recent ERO evaluations of secondary schooling, student achievement at the effective schools was reinforced when curriculum, pastoral and careers systems worked together. [13] This coordination was an important aspect of the culture of the school. Staff shared a belief in the benefits offered by restorative and/or supportive approaches for students who were struggling. The schools with more effective approaches had less emphasis on punishing students, and more emphasis on understanding and responding to the factors that prevented students from reaching their potential.

Staff at the schools with effective approaches often differentiated their responses for individual students at risk of not achieving. School personnel used different strategies depending on the situation of each student. They also differentiated their approach with families so that specific support strategies could be developed for each student and their situation. In the example below the school’s involvement of families and flexible curriculum approach had a significant effect on the NCEA achievement patterns.

Making the curriculum work for all students - especially Māori

One small, low decile area school, with a predominantly Māori roll, had identified that their NCEA achievement needed to improve. In 2011, the school introduced a new programme into their senior school which provided a high level of flexibility to their curriculum based around student strengths, needs, interests and aspirations.

The programme sees Years 11 to 13 students take part in a core programme involving mathematics, English, te reo Māori and science. The students also have two timetable lines (approximately 10 hours) dedicated to a particular curriculum theme. The themes are broad areas of student interest and include contexts such as multi-media studies, performing arts, academic studies and trades-based learning.

Students take part in an interview with staff where they discuss and decide their area of interest. Families and whānau are fully involved in this process. Teachers, students and family members then set specific learning goals which become the focus when they meet together twice each term. The goals inform the specific activities undertaken in class and teachers link achievement and unit standards to these projects. In this way the student interests shape the curriculum, which is then assessed. Significantly, this is in contrast to most senior school settings where NCEA assessment still operates as the default curriculum. In 2013 a student from the multi-media programme won an award for her digital project. Students from the performing arts area received credits on the basis of a production they developed for the school.

The use of students’ goals to shape the curriculum and assessment focus of these schools has considerably improved student engagement. Other initiatives the school has introduced have also supported engagement. For example, students also have access to STAR courses, which they do as block courses during the year (this helps to minimise clashes with the normal timetable). Students can also take part in Trades courses at a nearby provincial centre one day a week and some of the Year 13 students also take part in Gateway.

The benefit of the school’s approach is most evident in the increase in NCEA achievement. From 2011 to 2012 there was a 25 percent increase in the number of leavers with NCEA Level 1, NCEA Level 2 and University Entrance. [14]

In contrast to the efforts of the school above, most of the other schools with effective approaches had not made the same level of change to their curriculum in an attempt to improve student achievement. Most schools were focused on using pastoral care approaches to keep students engaged, rather than make significant changes to their teaching and learning. Pastoral care approaches included effective student mentoring and support as well as the tracking of individual students and responding to specific issues affecting their performance. [15] Some of these schools already had some effective curriculum innovations in place but most had maintained somewhat traditional learning opportunities, including the use of conventional subjects and typical timetable structures. For most of the effective schools the high levels of NCEA success they had already achieved may have outweighed the need to make significant changes to school-wide curricula.

Other significant curriculum changes made by schools with effective approaches, in response to student achievement information, included one school introducing three additional vocational options to its senior programme following a survey of students. This school had identified that 20 percent of its leavers were not well catered for. Leaders supplemented what they had learnt through an analysis of achievement information with a survey of the students who were not well served, and asked them what other options they would like added to the timetable. The additional vocational options were added and these were linked to small increases the school saw in its (already high) achievement rates for NCEA Level 2.

At another school, its review of student achievement information resulted in them moving away from Cambridge examinations. When the school decided to develop Cambridge classes for its top students, the rationale was that it would be more motivating and help more able students to gain endorsements in NCEA. It was hoped that this would flow on to NCEA Level 3 and scholarship results. The school also identified a risk in that other schools in their community were offering Cambridge examinations and they did not want to lose students to these schools.

After several years of Cambridge examinations the school identified that its top students had not noticeably benefitted in terms of their NCEA endorsements. There were also logistical difficulties in having students enrolled in two assessment programmes. In some cases the school’s involvement in Cambridge examinations limited the NCEA opportunities for students. For these reasons the school opted to drop the Cambridge examinations in favour of an all NCEA assessment programme.

Making a difference through effective inquiry and improvement

Effective schools had levels of achievement that were either rising and/or above that of schools of a similar decile. Across most of these schools, Māori and Pacific student achievement was comparable to that of all students at NCEA Levels 1 and 2. However, the numbers of Māori and Pacific students gaining University Entrance tended to be below that of other students. [16]

ERO found evidence that the inquiry and improvement processes at these schools had made a difference for Māori and Pacific students. In particular, ERO found considerable evidence that these schools had effectively identified issues affecting student achievement, explored some of the possible solutions, implemented a strategy and seen an improvement in the performance of most or all of these students. This is seen, for instance, in the example below.

Improved results for a targeted group of students at a small secondary school

One small secondary school uses a targeted approach with students it identifies through its assessment processes. For example, in 2010 the school identified a group of low achieving Year 9 students, four of whom were Māori.

In 2010 these students took part in an English class where their specific literacy needs were the focus of the school’s special education needs coordinator (SENCO). Two teacher aides were also trained to support these students in class. This included training about behaviour, dyslexia and autism.

In 2011, when these students had entered the senior school, a learning plan was developed for each student. Each student’s strengths and needs were considered, along with their intended pathway through school and beyond. These students were then provided with courses relevant to their individual pathways. Two students were given access to work experience, life skills programmes as well as community and STAR courses. Another student was placed in a Trades Academy.

As of 2013, eight of these nine students had progressed to NCEA Level 2 courses with just one having left school (without NCEA Level 2). Four were on track to achieve NCEA Level 2 in 2013, while another three were likely to achieve this milestone in 2014. Another student was unlikely to achieve NCEA Level 2 due to unresolved truancy issues. Two of the students had become mentors for younger students at the school.

Leadership at schools with effective inquiry and improvement approaches

Schools with effective inquiry and improvement approaches showed a focused and coordinated commitment to improving student achievement. This had typically been developed over several years in most cases and reflected the efforts of well organised and strategic leadership working with capable staff. The success of these schools reinforced the widespread belief of staff that they could make a difference for all students. Overall, there was a sense of urgency and agency from staff to ensure that they made a difference, and that their efforts, along with those of their colleagues, would contribute to improved student outcomes.

Leaders were an important aspect of the urgency and agency displayed by staff. They modelled the importance of ‘finding a way’ when it came to dealing with the challenges that limited student achievement. Leaders had a role in developing effective school-wide processes for the collection, analysis and use of student information. This was not limited to achievement data, but also included information gathered from the pastoral team as well as day-to-day observations made by form teachers, mentors and other staff.

School leaders also helped develop useful school-wide processes for monitoring and responding to student achievement issues. They established the expectations and guidelines for how departments analysed, reported and responded to achievement information. Leaders ensured that detailed analyses of student achievement information led to well-considered plans for improving student learning.

Similarly, the boards at the effective schools supported the values and approach of school leaders and teaching staff. Trustees in these schools were, in most cases, effective governors who received good information about student achievement and resourced suitable strategies to improve learning.

School-wide professional learning and development (PLD) helped give staff the skills to carry out inquiry and improvement processes. This PLD included training in analysing achievement data as well as undertaking ‘teaching as inquiry’ projects. In many cases, staff were trained as mentors (academic counsellors) so they could give one-on-one advice and guidance to students, and their families and whānau, about their progress at school, including each student’s academic goals and career pathways.

Effective mentoring by staff helped students achieve success across a range of academic, sporting, artistic and social domains. Staff developed their knowledge and understanding of students, and often emphasised the importance of ‘knowing the student’ as a significant aspect of developing processes that support each student to succeed. ERO observed that the mentoring and support given to students at the effective schools helped to motivate them to achieve and acquire the skills to manage their own learning.

Striving to make a difference for Māori students

One large, mid-decile school was judged to have effective inquiry and improvement processes because of the efforts it had gone to in analysing and responding to the educational achievement of its Māori students. A third of the students at this school identified as Māori. The school had an ongoing involvement in Te Kotahitanga. [17] In 2011 this involvement contributed to a significant shift in Māori student achievement at NCEA Level 2.

In 2012 the school saw a much smaller gain in Māori student achievement at NCEA Level 2. The school’s leaders identified that they needed greater levels of coordination between their existing initiatives to improve student achievement, especially Māori student achievement. They reviewed and modified their systems designed to support student achievement. They also considerably improved their connections to whānau and families. Tighter connections were developed between the school’s strategic goals, teacher appraisals and PLD. More emphasis was put on ‘teaching as inquiry’ and implementing strategies Māori students, in particular, needed to succeed.

Central to the school’s efforts in 2013 was a target group of 15 Year 11 students (most of whom were Māori). These students were provided with additional support through academic mentoring, extra tuition, home visits, career advice and closer tracking. At the time of the ERO review these students were on track to achieve NCEA Level 1. Because of the initial success of this initiative, it was being widened in 2014 to include a target group of Year 10 students.

Schools with some effective inquiry and improvement processes

Approximately one-third of the schools in this evaluation (14 out of 40 schools) had inquiry and improvement processes that were somewhat effective. While they had typically developed some promising initiatives to raise student achievement, these schools did not have the same level of coordination, focus and effectiveness as those in the previous category.

These schools were a variety of sizes, deciles and types. Five were high decile schools, seven middle decile schools and two low decile schools. Their achievement profiles reflected achievement patterns that were, in many cases, broadly comparable with schools of a similar decile, with some schools being above and some below schools of a similar type. [18] From 2010 to 2012 some of these schools increased the number of students gaining some qualifications, for example in the percentage of students achieving NCEA Level 2, but their achievement also went down in other areas. This inconsistent pattern was also seen in the achievement of Māori and Pacific students at most of these schools. [19]The curriculum, pastoral care and careers systems of the somewhat effective schools showed a mix of effectiveness. While there were pockets of innovative and effective practice, many new initiatives were in the early stages and without evidence that they were consistently or significantly raising achievement.

Inquiry and improvement that was starting to inform practice

Although these schools carried out some analyses of student achievement, engagement and attendance, these analyses were not undertaken well enough to consistently inform practice. This inconsistency occurred across a range of inquiry and review contexts including the analysis of school-wide data, teaching as inquiry projects, the examination of individual student achievement, and reviewing the effectiveness of specific initiatives.

For these schools to have effective inquiry and improvement processes, they needed to extend from analysing ‘who’ might need additional support, to effectively identifying and implementing ‘what’ their targeted students needed in terms of a curriculum, pastoral and/or careers response. Many of these schools had identified possible ways to improve student achievement and some had implemented school-wide strategies. Generally these schools did not have robust evaluation processes in place to subsequently monitor these initiatives. The evidence, at the time of ERO’s review, tended to show that these initiatives had not resulted in any clear improvements in student achievement.

A significant factor affecting the quality of some school analyses was the use of participation data. [20] In approximately half the schools with somewhat effective inquiry and improvement processes, school-wide analyses were focused on participation data rather than roll-based data. The use of participation data (without reference to roll-based data) can obscure what has occurred for students who have not, for example, completed an academic year (or entered NCEA qualifications). This may include those who have left school or been placed in an Activity Centre, Alternative Education or Teen Parent Unit. For all schools there is potentially a rich source of information available in understanding why some students leave, and what could be done to improve the overall engagement of students.

Developing initiatives for students

Schools with some effective inquiry and improvement processes typically had some positive initiatives that aimed to improve student achievement. In many cases these schools had developed initiatives as a relatively broad response to achievement patterns or issues. For example, academic counselling approaches were introduced to improve engagement and vocational learning programmes were added to the curriculum to provide more options for students who were not succeeding. It was not always clear that the introduction of these initiatives was sufficiently well-targeted by these schools. In some cases these initiatives had yet to gain momentum. In terms of specific examples, one school had focused its teacher PLD on developing differentiated teaching approaches. Another school established a mentoring programme for Māori students. A third school put considerable emphasis on staff members developing teaching as inquiry projects.

These examples, and several others from across this group, show some potential to make a difference for students. Some of these initiatives only started in 2013 without firm evidence of increases in student achievement in NCEA available at the time of ERO’s review. Others, which started earlier, had been modified to better achieve success. In 2013, the above example of a mentoring programme for Māori students, was altered to become more focused on students setting academic goals and including whānau in the goal-setting process.

Overall, despite the promising start made by many of the initiatives, the inquiry and improvement approaches of these schools had not contributed to consistently or substantially improved outcomes for priority learners. In some cases the initiatives themselves were too broad and were not sufficiently focused on the specific issues affecting individual students. A key feature of the initiatives at schools judged to be somewhat effective was the limited way they responded to individual students and targeted groups of students. The staff at somewhat effective schools needed to know more about their students – the specific academic, pastoral, careers and whānau contexts of students – and translate this into a coordinated response.

Clear differences between effective and somewhat effective schools were evident in terms of the agency and urgency of leaders and teachers. Staff at the more effective schools were more focused on making a difference, they were more confident that they would find a way to improve student achievement, and they worked (one student at a time in many cases) to ensure that the initiatives they introduced specifically addressed identified issues and led to clear improvements in student achievement.

A school with somewhat effective processes for inquiry and improvement

At one small middle-decile school NCEA achievement has fluctuated from 2010 to 2012. While its NCEA Level 1 and 2 rates have been comparable to schools of a similar decile, their University Entrance rate has been well below that of similar schools. The school had identified student retention as an area for development.

In 2011, the school implemented some approaches to improve student engagement and retention. These included a Year 10 diploma, with a focus on work skills and preparation for NCEA qualifications, as well as an across the school focus on literacy. A change to the timetable structure, which saw the school shift from six subject lines to five was undertaken along with a tightening up of the students who could take part in Gateway. [21] These changes did not noticeably improve the NCEA results for 2011.

At the end of 2011 the school was accepted into the Starpath programme. [22] In 2012 the school introduced academic counselling to help students set goals and work with staff to improve their achievement. The school’s NCEA results rose slightly from 2011 to 2012 – although these results were still not as high as those achieved in 2010. The school’s review of its 2012 Starpath identified that some of the school’s targets were too broad. In 2013 the school has had a much tighter focus on the achievement of individual students – both in terms of its charter goals and in how it carries out academic counselling. A new student management system has also helped the school better analyse student achievement through the year. The school is anticipating improved results on the basis of these new initiatives.

Schools with limited inquiry and improvement processes

Just over one-third of the secondary schools in this evaluation (16 out of 40) had inquiry and improvement processes that were of limited effectiveness. Five of the 16 schools in this group were low decile, seven were middle decile and four were high decile. These schools were in a range of rural and urban settings, much like the schools in the more effective categories. They also ranged in size from small to very large.

Most of the NCEA data for these schools did not show improvement. The NCEA results for some of these schools fluctuated and/or were unchanged (from year to year). Some schools had NCEA results that were dropping and/or low compared to similar schools. A few of these schools had NCEA results that were rising and/or in line with similar schools. Where schools had relatively positive NCEA results there was little or no evidence that this was linked to school inquiry and improvement processes.

Overall, these schools did not have a strong culture of inquiry and improvement. Similarly, there was a lack of urgency to improve achievement from at least some staff at each of these schools. While many of these schools had introduced some initiatives to improve student engagement – including vocational courses and academic counselling, these initiatives were not a focus for ongoing self review. In some cases these initiatives were introduced while significant issues affecting achievement, such as student attendance or engagement, were not addressed.

The other significant features of these schools were their lack of:

  • monitoring of initiatives to improve achievement
  • school-wide frameworks for inquiring into achievement and developing effective responses to improve results
  • teacher skill or knowledge in identifying the issues affecting achievement and developing suitable responses
  • links that existed between teacher appraisals and efforts to improve student achievement
  • analysis and response to student achievement for students, especially for priority learners
  • challenging targets that could help provide a strong focus for improvement
  • focus on Year 9 and 10 student data as a way of improving student achievement.

Even the schools in this group with more positive NCEA achievement had a limited understanding of their inquiry and improvement context. Despite the achievement profiles of these schools, ERO found that staff lacked the capability to carry out inquiries into achievement. ERO also identified issues with school-wide inquiry and improvement processes, including issues with target-setting and the lack of specific actions to reach these targets.

All schools in this category had little capacity to be able to identify and extend the programmes or teaching approaches that were most beneficial for students in their school. Without this understanding it is unlikely that more students will achieve the qualification necessary for them to succeed in future education, training or employment.