Findings

This section includes ERO's findings from fieldwork in Terms 1 and 2, 2016 and then Term 4, 2016.

Findings from Terms 1 and 2, 2016

Student selection

During Terms 1 and 2, 2016, ERO asked schools about the basis on which students were nominated and selected to become participants in Year 9 Plus. This was to determine:

  • the criteria schools used to calculate the degree of risk to later learning
  • what schools thought about the nomination and selection process
  • whether the process could be improved.

What information did the Ministry request from nominating Year 8 schools?

In an email sent to 45 Wairoa/Gisborne/East Coast schools with Year 8 students in 2015, the Director of Education, Hawkes Bay/Tairawhiti, asked school principals to nominate 'which Year 8 students do you think will struggle next year and/or will benefit from additional support to be retained and succeed in the secondary school environment?' The Ministry asked that only names and dates of birth be forwarded.

The specifications schools used in nominating students for Year 9 Plus

Leaders at schools with Year 8 students interpreted these instructions in a variety of different ways. In April 2016, ERO asked leaders about their basis for nomination of students. Their responses as shown in Figure 5.

Figure 5: Basis for nominating students

1:School nominating
criteria for Year 9 Plus entry3 Specifics for nomination
School
A
School
B
School
C
School
D
School
E
School
F
School
G
School
H

Total
Schools

Issues with whānau    
  •  
 
  •  
  •  
  •  
  4
Year 8 students who need extra support
  •  
 
  •  
 
  •  
      3
Struggling already with literacy and numeracy    
  •  
 
  •  
 
  •  
  3
Likely to fall off rails at secondary school  
  •  
 
  •  
        2
High truancy or stand downs already          
  •  
 
  •  
2
Challenging or worrying behaviour already      
  •  
     
  •  
2
Students likely to leave without qualifications
  •  
              1
Siblings with poor secondary record            
  •  
  1
Those who might get lost or fall through gaps          
  •  
    1
Social isolates          
  •  
    1
Students who are academically bright but not coping at home or need extra push    
  •  
          1

What schools thought about the nomination process

Leaders at all the nominating schools we talked to viewed the Year 9 Plus initiative positively and were keen to contribute to what they thought was a timely and worthwhile initiative. They put careful and considerable thought into who they should nominate for Year 9 Plus entry.

Leaders were conscientious about nominating students on the basis of the Ministry communication, but were unsure whether they should be considering mainly academic or largely behavioural criteria, or some combination. There were some ethical considerations for these schools. For example, they knew that some of their nominated students might not be selected and were therefore unsure about consulting parents and whānau at this early stage, thus building expectations for support that might not be fulfilled. They were also worried about sending names that might be seen as 'labelling' students to someone they did not know personally.

ERO found primary and intermediate schools had considerable data on the needs and potential of students, and their schooling history. They knew their students and families well and applied that knowledge to good effect when forwarding nominations to the Ministry.

However most nominating schools expected that following their nominations they would be asked for further information to help with the selection of those students at highest risk of not completing secondary education. They were expecting to be further approached to share their thinking with the Ministry after nominations were made.

Selection Process: Risk factors likely to be important in Year 9 Plus outcomes

The Ministry's risk model was developed using demographic, socio-economic and behaviour data that the Ministry holds about students born between 1994 and 1996, from when they started school until when they left. This includes their attainment of NCEA Level 2. The following factors, in descending order of predictive influence, have been shown to highly predict the risk of not attaining NCEA Level 2:

  • ethnicity
  • school decile
  • school funding type (provides information on the school level and the kind of school)
  • stand-down and suspension information
  • gender
  • non-enrolment data (indicates students who are not enrolled when they should be)
  • the year level of the student at age 12.

The model has been evaluated for performance and has been found to differentiate well between those at high risk and those at low risk of non-attainment.

Applying the predictive risk model to those eligible for entry to Year 9 Plus, nearly all of the students participating in the programme from the start of 2016 were judged as being in the high risk category.

However, this predictive model does not consider how well any individual student has been engaged in earlier schooling or achieved in key learning areas at primary school. Such information is not available on the Ministry's database and was not requested from the schools.

In May 2016, ERO completed a literature review[4] of the risk factors likely to be important in Year 9 Plus outcomes in 2016, using a social investment approach. The highest risk factors found in the review to be significant to later educational success after Year 8 are in Figure 6.

Figure 6: Risk Factors to Year 9 student engagement and achievement

Family background
  • Household in a low SES location
  • Primary caregiver with no qualifications
  • Main income of household from a benefit
  • Family with history of CYF or Justice involvement
Early education
  • Poor oral language/vocabulary development before age 3
  • Preschool health issues, especially around speech and hearing
  • Poor school attendance, getting worse in middle and senior years of primary schooling
  • Transience during primary schooling - attending more than 5 pre-secondary schools
Transition to Year 9
  • Sole child from a small school or eldest sibling to attend the new secondary school
  • High anxiety/low self-esteem personality
  • Academic difficulty in primary school with maths or science
  • Males of a minority ethnicity group in the new school
Engagement in Year 9
  • Peer issues or problems with bullying
  • No strong relationship with any individual teacher
  • Teaching seen as ‘boring’ or ‘too academic’
  • Early patterns of truancy or stand-down

This literature review indicates the level of engagement in early schooling, in particular Years 7-8 and has a key influence on ease of transition to secondary school and success in Year 9.

What schools thought about the outcomes of the selection process

When we initially talked to school leaders, no full primary or intermediate schools had been formally informed about who from their nominations had been selected. Those schools with both Year 8 and Year 9 students found out who was on the programme early in the 2016 year, when champions came to assist with the enrolment and setting up of programme support for their selected students, towards the end of January 2016.

The secondary principals and senior leaders of two urban schools, where just under half the total number of Year 9 Plus (approximately 48) students attend, indicated that some of their most vulnerable students were in the programme. When asked if their most 'at risk' Year 8 students from 2015 were selected to enter Year 9 Plus in 2016, other school leaders said they were surprised at who got on the programme compared to those who were not selected.

In discussion with ERO, schools suggested clearer criteria for nomination and use of a wider range of information sources could have better informed the selection and transition process. Composite, primary and intermediate schools had substantial data about individual students exiting Year 8. They considered this vital information could have been better used when planning support for those Year 9 Plus students transferring school between Year 8 and Year 9.

Intermediate schools considered they had valuable information on risk factors that they could share with secondary schools face-to-face, if the latter thought this might be helpful for them in framing an immediate response the following year. They told us they believed such open sharing of information could be the basis for greater inter-school collaboration in future, as they believed all schools were committed to building improved learning pathways within local districts and the wider region.

Although use of such school-based information was not used for the purposes of Year 9 Plus, ERO recommends that school leaders consider how they can collaboratively develop processes to share and use information available in contributing schools in future. Sharing of such information as a regular part of transition processes could help ensure students potentially at risk can be identified earlier, and support provided to them and their whānau as they transition to secondary education.

Conclusions about student selection

ERO found that the Ministry did not provide clear criteria or use a wide enough range of information sources to reliably inform the 2015-16 nomination and selection process so the right selections were made. Identification of students requiring extra support to transition to, stay in, and succeed in secondary education could be improved if more attention was paid to the substantial local information available from nominating primary and intermediate schools on students' patterns of achievement and engagement in primary schooling, and any known risk factors.

In particular, ERO's literature review indicates early schooling, particularly the level of engagement in Years 7 and 8, has a key influence on ease of transition to secondary school and success in Year 9.

With regard to student selection in Year 9 Plus we found:

  • Contributing schools responded positively to the request for student nominations and were diligent in their analysis of school-based information during selection.
  • Although a range of criteria were used by schools, nominated students had been identified by their contributing schools as being at risk of not transitioning well to secondary schools.
  • The Ministry's use of a risk prediction model was a key component of the final selection of those students to be offered a place to enter the trial. The predictive modelling did not use school-based information about previous engagement and achievement, as this was not part of the available database.
  • Students selected were appropriate for the trial, although some schools were surprised at who was chosen and who was not.
  • Composite, primary and intermediate schools have substantial data about individual students exiting Year 8. The most valuable data relates to what has worked well for individual students in the past and what may have worked not so well. More use could be made of this vital information when planning supports for students transferring school between Year 8 and Year 9 in the future.

The Role of Champions

In terms of improving support for learning, the most significant factor within this initiative is the employment of champions. They are a key resource for change; by working with whānau, students and teachers within participating schools and contributing to school's responsiveness for learner success. See Appendix 1 for a list of participating schools, their numbers of Year 9 Plus students and allocated champions.

ERO's preliminary evaluation focus for Year 9 Plus was to find out how familiar schools are with the potential from this newly available resource to change patterns of earlier relationships and reconfigure future learning trajectories. In May 2016, we asked schools about what they expected of the champions, how they were introduced into participating schools, how these schools responded to them, and how well participating schools understood and were taking advantage of the transformational aims of the initiative.

Expectations of the Champions

The Ministry job description states the champion's purpose as follows: 'the champion will support and inspire young people to remain attached, engaged and achieving in secondary school - from entry (at Year 9) through to when they leave.'

The shared job description of each champion outlines nine key priorities for their work:

  • to work with schools, kura, attendance service providers, and Ministry colleagues to engage students in the target group who are at risk of disengaging from education
  • to support the Director of Education to establish and manage the delivery of Year 9 Plus within the education system
  • to establish positive relationships with each of the students they are responsible for, and harness the parents and whānau to support these young people achieve in education
  • to identify and remove barriers to young people gaining a full education - including in school and out of school issues
  • to coordinate work with schools, agency partners, relevant community organisations, and supporters/influencers of Year 9 Plus students to collectively support students to remain engaged in education and focused on achievement
  • to track, anticipate and respond to emerging issues, advise on priorities, and focus effort where it has most impact
  • to capture and report process and impact information, and emerging policy issues to inform improvements within the education system and beyond
  • to work consistently to established timeframes, managing their own input and ensuring others make contributions to enable the work to progress
  • to maintain open, responsive and constructive relationships with key stakeholders.

Champions told ERO in May 2016 their role was to act as proxies for parents to try to reduce risk factors for the students they were supporting, but they were not to replace parents and whānau.

Who are the Champions?

The personal specifications for appointment as a champion emphasised:

  • excellent local knowledge and networks
  • ability to work sympathetically with students, parents and whānau
  • knowledge of the education system and qualifications
  • a positive attitude and belief in young people and their potential, for Māori student success in particular.

Champions are not intended to be education professionals. They have previous experience in a variety of relevant contexts, including Iwi development, police force, army, teacher aides, and truancy service. ERO found all champions had considerable links with the community in fields such as youth work and community development.

Schools' views of champions' roles and activities

At the outset, there was a lack of clarity from school leaders about the role of the champions and protocols for their introduction and work in the schools. In April and May 2016, we found school leaders with Year 9 Plus students held a wide variety of views about the role of the champions, where they perceived the trial was headed, and how they saw the champion/s as a resource for supporting learning in schools.

Evidence from our initial discussions with principals and senior leadership teams in schools with Year 9 Plus students and champions indicated a range of views that are described in Figure 7.

Figure 7: School views on responsibilities of champions

Main ‘theme’ in a school’s expressed
view about champion
Underlying school view of the initiative’s aims and
transformational possibilities
  • High trust in champions as fellow professionals, engaged in the same educational enterprise; and/or
  • See their work as offering a huge opportunity to build student educational success across the curriculum, with a shared future focus.
  • School adds supporting academic success in class to the role, mainly in ‘basics’ of literacy and numeracy - especially with an early acceleration focus.
  • School adds supporting wider academic success to champion’s role - for example, supporting educational success as Māori; coaching in digital literacy and STEM; and fostering success as a learner with a future focus and/or by considering vocational pathways and/or becoming an active citizen and lifelong learner.

  • Moderate trust in their champion(s);and/or

  • Happy for them to play a role defined by their areas of previous expertise and agreed to by the school, particularly if it is related to student support activities.
  • School adds supporting student engagement through 'sorting out’ peer relationships and behavioural expectations – for example, acting on instances of physical or verbal bullying.
  • Sees champion as having an out-of-class focus.
  • School adds supporting the student’s and her/his whānau engagement in a range of school extra-curricular activities to champion’s role – for example, sees a role for them in supporting homework or engagement in sporting or holiday activities.
  • Don’t really trust them or regard them as fellow professionals; and/or

  • Unclear about their role and expectations beyond acting as a connecting point to family and whānau.
  • School sees programme as mainly about ‘presence’- role of champion is in assisting enrolment, attendance, and retention at school.
  • School may add improving engagement of family and whānau, and promoting parental involvement in child’s education to champion’s role.

At this early stage of implementation, ERO found a wide range of understandings across the Year 9 Plus participating schools about the potential role of champions in supporting student wellbeing and learning, and how they could work effectively with them within the existing arrangements of the school.

The schools using champions most effectively developed a working protocol with their champion and were developing high-trust relationships. Other schools were still trying to establish the exact parameters of the champion's role.

School leadership is critical to the success of this initiative. Some school leaders had a much better understanding of the transformational possibilities of the programme than others. Reservations about the Year 9 Plus initiative and lesser knowledge of the intent tended to come from less experienced principals.

School leaders who attended the trial's launch or who were otherwise involved in the setup of the initiative from an early point were well informed of the overall intentions, recent shifts and refinements in details. They were using the champion as intended - a key resource for change.

Other school leaders were less well informed about the aims of Year 9 Plus and the role of the champion as the key agent for change in the trial. These leaders missed out on critical information about the aims of the programme or the ways in which the champion could help facilitate these. ERO found the latter schools were least likely to be taking full advantage of the variety of opportunities that the champion offers to support the school's at risk students.

A promising finding

One secondary school told ERO it believed a significant number of the Year 9 Plus students at their school in 2016 would not have enrolled or still be attending by now if they had started at the school without the support of the champion. Champions played a significant role in many of these successful enrolment and retention cases. This school was tracking their Year 9 Plus cohort carefully and had given their champions full access to the relevant data trends to help with their work. They had developed a protocol outlining for both school and champion what the champion could and could not be expected to do or take action over. Year 9 Plus students ERO talked to at this school felt well supported.

Conclusions on the role of champions

At the start-up stage, secondary school leaders' confidence in, and underlying view of Year 9 Plus aims and transformational possibilities was dependent on the quality of information initially received, and their own perceptions of the capacity of the champion as a resource for supporting learning. ERO found:

  • there was an initial lack of common understanding in and across schools about the role of the champions as an agent of change, and protocols for their introduction and work with and in the schools
  • school leaders received differing degrees of information about the aims of the trial and potential role of the champion as a resource that could support schools to review their practice in working with vulnerable students and connecting with their families.

As a result some schools were using the champions as a key resource for change in how they responded to students at risk, and their parents and whānau; while others were still working to understand how champions might contribute to student engagement, participation and achievement.

Transitions into Year 9

The effectiveness of student transition is a key factor in success for all learners when students move school between Year 8 and Year 9. All students coming from Year 8 departing schools should expect to be successful at their receiving secondary school. A good start at secondary school is essential for helping students achieve the foundation skills necessary for future wellbeing, training and employment. The Year 9 Plus initiative is intended to support students make successful transitions to secondary school and support them as the move through.

How ERO framed its investigation

We framed this aspect of the Year 9 Plus evaluation on the key findings in ERO's 2012 national report on patterns in transitions from primary to secondary schools. Successful transition to secondary school is more complex than just developing a number of orientation processes for students to become familiar with the receiving school's environment, personnel and programmes. New Zealand's schools tended towards one or other of two differing orientations to transition from Year 8 to Year 9, and the journey towards more effective transition for any school featured less of some and more of other key features of these two orientations (Figure 8).

Figure 8: Two different orientations to Year 8 to Year 9 transition

A ‘transfer’ orientation: for effective
transitions of all students, (particularly
those at high risk), schools need to use less of these features and processes

A ‘transition’ orientation: for effective
transitions of all students, (particularly the
most vulnerable), schools need to use more of
these features and processes

  • departing schools not visited until well into Term 4
  • receiving school treats a variety of Year 8 students and contributing schools with a common approach and set of processes
  • process emphasises information giving (one-way communication) in the early stages
  • both departing and receiving schools expect that only a few people in the receiving school will manage transitions (e.g. deans and form teachers)
  • receiving school leaves the planning for supporting vulnerable students to the later stages of the process
  • receiving school plans common orientation processes early in Year 9 for all new students to become familiar with the school's environment, personnel, programmes, and philosophy/culture.
  • overall process assumes that all students from the same school have similar transition needs and once these common needs are met, transition has succeeded.
  • both departing and receiving schools know successful approaches start early, use two-way communication and include a wide variety of different people who are involved in closing one set of relationships and forging a new set in a different setting
  • includes in the transition planning process all the people who are connected to each individual student from both settings
  • both departing and receiving schools expect early identification in the process of individuals who are most vulnerable during transition, and who may need special support in the new setting
  • all schools understand relationship building among and between all teachers and students is crucial to success in any setting, and different individuals take differing amounts of time for these new bonds to develop in a new setting
  • the whole process is supported by ongoing relationships and communication with parents and whānau.








Transitions in Year 9 Plus

Using the framing in Figure 7, ERO investigated how well Year 9 Plus students transitioned from Year 8 to Year 9. We talked to all primary, intermediate and secondary principals in the Wairoa/Gisborne and East Coast region involved in the trial about the process they used in 2015/16 for student transition. ERO asked how this compared to the process they had used previously. We also talked to some of the Year 9 Plus students about their experiences during transition.

Figure 9 shows overall patterns across the schools (primary, intermediate and secondary) in 2015/2016 transitions.

Figure 9: Transition Patterns in 2015/2016

Contributing schools (primary schools and intermediates) with Year 8 students Secondary schools (both single sex and co-educational) with Year 9 Plus students
  • in late 2015, these schools used a well-established process for sharing information about prospective secondary schools with departing students and their parents/whānau
  • they ensured all city secondary schools had similar opportunities to inform students about their school
  • at the end of 2015, these schools provided standard information to secondary schools that departing students were pre-enrolling in, in a format secondary schools had designed
  • these schools had detailed profiles about their departing students (based on both achievement and engagement data) that were available to share with the next receiving school of choice if requested
  • these schools farewelled their departing Year 8 students reluctantly, and with due ceremony, and had a continuing interest in their progress in Year 9 and beyond.
  • in late 2015, these schools participated in a reciprocal planned process with other city schools for pre-enrolling Year 8 students into the secondary school of their choice
  • in early 2016, these schools used a planned
    process for the induction of arriving Year 9
    students including an orientation week, a buddy
    relationship with a senior, a class camp in the
    early weeks to get to know teachers and
    classmates, and testing to confirm academic
    placement
  • secondary schools did not always trust the
    information they got from primary and
    intermediate schools particularly around
    achievement; they preferred to arrange their
    own tests during the early weeks of the new
    school year
  • deans at these schools paid close attention to the needs of the vulnerable, valuing primary and intermediate school information about this, and using it to keep a ‘watching brief’ on those most at risk in Term 1, 2016.

In the schools where staff actively worked with champions during transition, some Year 9 students were well supported to continue to engage with education. However, the potential of transition support during the initial phase of implementation was not fully realised. This may in part be explained by the late start-up of the Year 9 Plus initiative and the differing understandings and expectations of the role of champions.

ERO found that secondary schools paid some attention to the depth of information from contributing schools about these new Year 9 students, in particular information about their specific learning and social-behavioural needs and what had worked (and not worked) in the past to address these. However, some secondary schools questioned the dependability of the information about academic achievement that was transferred to them and instead they spent a good part of the first month of Year 9 testing the students again with their own tools.

This testing regime is potentially daunting and demotivating for vulnerable students as they enter a new school environment. It risks early disengagement from school, and delays the process of accelerating their learning from day one of secondary schooling. Contributing Year 8 and receiving Year 9 schools should revisit and consider information that could be usefully shared so those learners at the highest risk of not transitioning successfully, and whose progress needs ongoing acceleration, can begin Year 9 with a 'flying start'.

An innovative transition process worked in one Years 7 to 13 school where the Year 8 programme formally finished on 30 November, and orientation for the next year started on 1 December, thus minimising 'down time' from academic work. This allowed the Year 9 school year to start promptly on 1 February. ERO found that the Year 9 Plus students interviewed in this school had all made a promising beginning to secondary education.

Choosing a School

We found the variety of reasons reported by students for their choice of secondary school often did not take account of individual student needs and aspirations, or how these might be best met from school options. The strongest reasons reported by students for choosing a particular school were based on historical experiences of the family with that school, followed by a preference for the ethos or culture of the chosen school. Parents and students need better information than this to make appropriate choices about schools of preference.

School leaders suggested that, if in place earlier, champions could have contacted parents and whānau of the most 'at risk' Year 8 students as early as August, to help with secondary school choice and the various requirements for enrolment. This suggestion would ensure a better match between individual student need and school options. It may also have helped avoid the 2016 experience, where a substantial number of the Year 9 Plus students remained unenrolled in any secondary school until late-January.

Champions also consider their support for students and families would have been useful in the second half of the preceding Year 8 school year to help with school choice, readiness, learning pathways and school-whānau relationships. In May 2016, ERO heard of three cases where champions were in the process of helping Year 9 Plus students move to a school that would better meet their needs.

Conclusions on student transitions

While in some schools, during transition, Year 9 Plus students were well supported to continue to engage with education, the potential of transition support during the initial phase of implementation was not fully realised. ERO found:

  • contributing schools provided secondary schools with standard information about transitioning students
  • contributing schools had detailed achievement and other data available to secondary schools, however the secondary schools generally preferred to test students at the beginning of the year rather than act on this information
  • this testing regime is potentially demotivating for vulnerable students, risking early disengagement, and delays the beginning of their secondary education
  • students and their families choose schools based on varying information.

Vulnerable students may require more and different support and guidance to make the right choice based on their needs and aspirations than is currently available.

Student engagement in learning

Responsive curriculum

ERO was interested in finding out how schools were responding to the identified learning needs of Year 9 Plus students. In the first six months of 2016, the preliminary evidence suggested that most composite and secondary schools participating in Year 9 Plus were continuing approaches that had worked for them in the past rather than adapting learning programmes for these students.

Appendix 4 provides a list of online resources and smart tools that participating schools could use to support the development of more responsive and personalised learning programmes.

Student engagement

ERO is particularly interested in the student perspective about the possibility of personal change for educational success, as this is a critical prerequisite to successful secondary school engagement (Gibbs and Poskitt, 2010). In May 2016, ERO had a first round of discussions with students, asking about their experiences with Year 9 Plus so far, and their perceptions of what had helped or hindered their transition into Year 9.[5] We interviewed two or three students from each participating school, talking to each student for about half an hour on a variety of topics.

We asked students about their previous school experiences and found many had struggled in the past. We also asked about students' interests and things they like doing. Most students told us about a variety of activities and aspirations. However, when asked if they had talked about these interests, or their future plans with either their champion or the school, most answered that they had not.

Champions had helped some students into extra-curricular activity, and other Year 9 Plus students had enjoyed either the homework club champions had recently arranged after school at a local wananga; or the April school holiday activities champions had offered to some. However other Year 9 Plus students were unaware of these opportunities or were unable to participate because of other commitments at the time, even if interested.

We asked Year 9 Plus students about how much they were able to identify future focused themes or personalised learning in the programmes they were studying in the first half year. We also asked if they could identify any specific careers guidance or vocational pathways advice. Only one or two were confident they had experienced any of these approaches.

We found a range in levels of initial student engagement in Year 9 learning (See Figure 10).

Figure 10: Year 9 Plus patterns of student engagement

Expressed perception of Year 9
Plus
ERO observation of student motivation to engage in learning
  • student presents as highly self aware and educationally resilient; and/or 
  • already making key steps towards becoming a successful lifelong learner.
  • step 1 to transforming: taking advantage of the full variety of opportunities offered by the champion for personal support for educational success
  • step 2 to transforming: taking advantage of the full variety of school curricular and extra-curricular activity to become an educational success (as Māori and as global citizen)
  • already shows both high agency and high responsibility.
  • student gaining knowledge and appreciation of the opportunity offered by Year 9 Plus, and/or
  • taking limited advantage of its opportunities so far.
  • student is meeting school expectations around attendance and behaviour - clearly wants to fit in and please
  • beginning to develop agency and sense of responsibility.

  • student unaware of Year 9 and Year 9 Plus as an opportunity to ‘turn things around’; and/or
  • yet to take advantage of the widening range of options to enjoy educational success in the future.
  • student is not yet aware of the range of possible supports available from the champion, or the new opportunities available in their school
  • has not experienced success in earlier schooling or in education so far, and does not see that things could improve for them in the future
  • needs encouragement to develop agency and personal responsibility

ERO found several students interviewed considered they were already on the way to successful engagement in Year 9. Most in this group appreciated the support so far from champions and their school. This was a preliminary finding, as we had not yet talked to the teachers of the students, to see if this positive trend was upheld from their perspective.

Conclusions about student engagement in learning

At this stage of the evaluation, ERO had begun looking at how schools responded to the identified learning needs of Year 9 Plus students through personalised and responsive curriculum. ERO was also interested in the student perspective about the possibility of personal change for educational success. In this early stage, ERO found several students interviewed considered they were already on the way to successful engagement in Year 9. However, there was still a need for schools to engage more fully with these students in order to develop personalised and authentic programmes of learning that reflected their aspirations for the future. 

Findings from Term 4, 2016

Student outcomes

In considering how well placed participating Year 9 Plus students were to succeed in their future schooling by the end of 2016, ERO considered:

  • development of student agency as the basis for positive engagement
  • their ability to articulate their successes over the year and the reasons for these successes
  • the extent to which opportunities offered by school and champion were used
  • the strength of family and whānau support
  • students' ability to articulate plans for a successful educational future.

Year 9 Plus students were grouped as follows:

Initial-steps[6]

ERO found many students had a fairly restricted view of success. In terms of engagement, they had begun making foundational change, but were not yet self-motivating. ERO called this group the initial-steps group. When asked what they saw as their main successes in 2016, this group talked about:

  • improving attendance from their time at primary school
  • staying at school until the end of the year
  • improving behaviour in class and school environs over the year
  • getting on better with their peers.

Most students in this initial-steps group saw their champion as the main factor that had fostered and supported their success. They told us about how champions would:

  • pick them up from home if necessary
  • check up on and monitor their timeliness and record of behaviour
  • give them strategies to deal better with peers or teachers they found they were unable to strike up a productive relationship with early on
  • help with homework or things they found difficult to understand
  • help in making better social choices or becoming more self managing.

Some students in this initial-steps group told us about how a teacher had also helped. This mainly occurred in a context where the teacher was teaching a subject the student already liked, or when they showed an interest in helping the student do better in the things they might have previously found difficult. (See Figure 11 for details).

Figure 11: Student voice - factors supporting success in 2016 

 Ways champions have helped students in 2016 
*helped me stay at school
*helped my attendance
*helped with my mental wellbeing
*listened and always there
*helped with relations at home
*helped my whānau connection (as Māori)
*readily contactable
*helped with my homework completion
*helped with my physical activity - swimming, playing more games
*helped by always being there for us
*helped improve my behaviour
*had set times to meet and can be relied on
*helped with homework
*checked up and followed up on us
*helped with relations with teachers
*someone extra to talk to
*treated us like whānau
*told whānau about our successes
*helped me do more work or better work at school and showed me how
*helped by feeding us 
 School or combined contribution to student success in 2016 
  • heightening expectations and aspirations (promoted by champions and/or teachers)
  • co-constructing goals and monitoring these (champions and/or teachers)
  • regular check-ups, rewards and feedback (champions and/or teachers)
  • champions knowing the students well and understanding family background and context
  • champions sharing students’ background and context with teachers (so they understood what else was happening in some students’ lives outside school), and at the same time being willing to tell the school’s side of the story to parents/whānau
  • teachers making learning more relevant and interesting
  • teachers using examples in ways that students understood
  • students’ interactions with Year 9 Plus students outside their own school (e.g. meeting them through attendance at a homework club or holiday activities)
  • positive engagement in outdoor activities and extra curricular activities.  

Aspirational

ERO found that some students had a more aspirational view of their success over the year. They knew what interested them and had achieved success in selected areas on their own terms. Students in this group talked about:

  • taking advantage of new educational opportunities
  • getting better grades
  • sporting or cultural successes over the year
  • doing well in a variety of planned school activities.

Students in the aspirational group saw the school as contributing strongly to their success. They told us of individual teachers who had made learning more relevant and interesting for them, taken an interest in their learning, checked what they did not understand and then explained this in terms they found helpful, or systematically built on their strengths and interests.

For some of these students there were situations where champions and teachers had combined to help. This included support in co-constructing goals and lifting aspirations, regular check-ups on progress, sharing helpful feedback, and providing recognition or rewards.

Some students in this aspirational group identified the champion as the main contributor to what they regarded as their successes of the year. This included champions:

  • developing strong initial relationships with their whānau and making them aware of the importance of keeping their child in school
  • getting to know the students' backgrounds well and understanding their home background
  • being prepared to share with the school the whānau perspective on the students' behalf, where student wellbeing may have been at risk.

Two students' stories

Two students shared remarkable stories about the extent of personal change over the year. For both, by the end of 2016, their life trajectory had definitely transformed. Though details of the stories differed, in both cases some similarities in the transformational process were evident:

  • both students had re-framed key understandings about themselves and their future over the year
  • both had developed self awareness of potential future strengths, which they wished to focus on from now on
  • both had planned and sought support from others to help make the personal changes needed to succeed in their selected area of aspiration
  • both were taking a leadership role in bringing about the changes deemed necessary.

Both students had started the year negatively and had transgressed on a regular basis early on. They decided they needed to turn things around and set out to do this in a systematic way. Both had gained parental support for their aspirations and were seen as a positive role model by other Year 9 Plus students at their school and by their teachers.

Constraints

While most students told us they had achieved success, ERO also heard about a small group of students who were clearly not yet succeeding at school. Factors contributing to this included:

  • ongoing concerning behavioural issues that needed highly specialist support that took time to access
  • finding their Year 9 programme, or aspects of it, of little relevance to what they wanted to do or were interested in
  • dislike of particular teachers or subjects, and misbehaving in these classes
  • continuing with anti-social activity such as drugs or alcohol usage, or fighting or verbal aggression
  • feeling suicidal and still struggling to respond positively to school.

ERO found circumstances where, despite champion's advocacy for students, barriers for students and whānau to gain better coordinated access to various social services remains an issue. We will investigate this further in later phases of the evaluation.

ERO also found that a small group of students were attending either Activity Centres (three) or Alternative Education (nine) centres at the time of our review. We will investigate the quality of the educational provision for these students and their progress in Term 2, 2017.

Conclusion about student outcomes

Discussions with students and teachers show that many students, champions and schools are working together for student success. We found that a majority of participating Year 9 Plus students were in an 'initial-steps' group by the end of 2016, in terms of moving towards future success. They had succeeded in staying at school to the end of the 2016 year and had passed the 'graduating standard' that a number of the larger schools used to gauge success for all Year 9 students in their current cohort.[7]

Anecdotal evidence suggests that a positive short-term outcome for the Year 9 Plus concept as a trial has been increased support for initial engagement in secondary schooling for these students at risk of leaving school early. Some schools told ERO that, in earlier years, many students with an entry profile of collective risk similar to that of the 2016 Year 9 Plus cohort, would probably no longer be in schooling by the end of the year, as most Year 9 Plus students were in 2016.

A group of aspirational Year 9 Plus students had already started to achieve academic success in their secondary schooling. A few from this group had begun the process of gaining the qualifications they will need for entering the future careers they had identified for themselves. By the end of 2016, most students in this group were fully engaged in working towards their academic or vocational ambition.

The initial success for this groups suggests what can be done when schools work together with champions (or other whānau support) with a clear goal in mind - in this case raising educational aspiration. Peer support is a critical factor in accelerating and broadening learning at this stage of schooling (Poskitt, 2015). Both champions and schools will need to work together in 2017, for continuing success for those students in this aspirational group.

Their academic future and/or personal wellbeing is at risk for those Year 9 Plus students yet to be fully engaged in secondary schooling. Their future learning success was uncertain when we visited in November 2016. One of these students told us to advise both schools and champions that 'we might be a pain, but do not give up on us'. The expertise of specialist support might be needed for some students, and this should be brought in or intensified as soon as possible. The aim should be to wrap around support and accelerate the future learning of these students.

School contribution

ERO gathered information on the responsiveness of schools to the Year 9 Plus students and concept trial. Of these, six were smaller composite schools or kura, mainly located in rural parts of the East Coast region. Four were larger secondary schools, located in Wairoa and Gisborne.

The following factors were considered when evaluating the degree of school responsiveness to Year 9 Plus students:

  • knowledge of students and their learning needs
  • knowledge of students and their wellbeing needs
  • strategic focus set by school's management and leadership for this group of students
  • responsiveness of the school's teaching and curriculum provision for these students
  • strength of the school's monitoring and internal evaluation processes for these students.

Smaller composite schools and kura

Leaders and teachers in the most responsive schools:

  • knew Year 9 Plus students' strengths and interests well
  • provided supportive opportunities for broadening student interests and fostering whānau wellbeing (for example, arranging a parent's enrolment in a te reo class where this would support their child's progress in a Māori medium setting)
  • identified and removed inhibiting factors relating to attendance and engagement
  • stressed individualised and strengths-based learning
  • co-constructed goal setting using school values (one school)
  • knew personal and/or whānau circumstances and took these into consideration
  • developed good collaborative relationships with the champion so both parties worked together in each child's interests.

In the most responsive composite schools and kura, staff were aware of students' strengths and what motivated them. Teachers incorporated the students' past successes into their challenges for the future, whether in academic, sporting or cultural activities. Goal setting and students' own evaluation of their progress were emphasised in one of these schools, along with differentiated learning supports. Career planning was in the early stages of development in most of these schools.

Some schools were less responsive to their Year 9 Plus students due to factors such as:

  • limited understanding of the rationale of the trial
  • poor relationships between the school and the champion
  • issues with curriculum relevance, and use of data or internal evaluation relating to Year 9 students
  • wider conditions in the school that took the focus off student achievement.

Large secondary schools

In the most responsive large secondary schools, there was evidence of:

  • distributed and collaborative leadership
  • shared vision and sense of overall direction provided from school leaders
  • a strong, values-driven school culture
  • key values embodied and implemented by a variety of teachers
  • improvements planned to build upon existing processes and systems known to work
  • a widening group of staff engaging with family and whānau and responding to what they wanted
  • a widening group of teachers and departments providing a culturally-responsive curriculum with a place-based emphasis
  • all Year 9 teachers using an effective tracking system, with rationale for credits and rewards known and understood by staff and students
  • in some instances, strong inquiry and internal evaluation to plan future educational improvements, based on analysis of achievement, attendance and behaviour patterns.

In the less responsive large secondary schools, there was a lack of any clear 'line of sight' between values and goals, plans and actions in areas such as:

  • all leaders in a school showing limited understanding of the purposes of, or their roles in, Year 9 Plus as a concept trial for educational improvement
  • a general impression of limited organisational responsiveness or urgency amongst leaders in relation to lifting student achievement
  • a lack of in-school understanding of assessment practices
  • uncertainty about the meaning of data patterns, indicating limited trust in, or understanding of, what available data was showing
  • limited understanding of assessment principles or of the critical need for common tools in the evaluation of the impact of any intervention
  • the data on student progress made available to ERO could not be aggregated or used to show comparative rates of progress or historical trends.

Examples of effective practices

Principals in the more responsive schools understood and took an interest in the trial and its outcomes over the year. They continued to monitor how the trial was progressing; support its implementation; and acted on reports from the person with delegated leadership of Year 9 Plus. The relationship between the delegated leader and other staff was also important in these schools.

Examples of effective practice included:

  • weekly monitoring meetings of the school's pastoral care team, with champions involved
  • the pastoral care team, under the guidance of a deputy principal, regularly taking actions based on key indicators of Year 9 Plus students' presence, progress and wellbeing
  • coordination of a school-wide system of interventions, rewards and sanctions used appropriately by staff
  • information from champions used by teachers to understand the needs and home circumstances of Year 9 Plus students, resulting in a better understanding of external factors influencing some students' behaviours, and helping teachers to understand what often lay behind the students' mental wellbeing, and the out-of-school context that could be impacting on their school attendance and attitude
  • using information about Year 9 Plus student progress, achievement and engagement to improve school systems and processes, and provide for increased future responsiveness to students. For example, one school introduced te reo me nga tikanga relevant to Te Tairawhiti in the curriculum for all Years 9 and 10 students in 2017
  • teachers collecting and using data on the Year 9 Plus students as a group, and planning responses based on data analysis
  • teachers recognising the need to build on present successes, and planning for new ways the champion might contribute to the overall welfare of students in future. For example, one school planned to provide access to the school gym during January 2017 holiday time, so the champions could run a summer programme of physical activity for the Year 9 Plus students at the school, including siblings and whānau members.

Conclusions about school contribution

The majority of kura and schools with Year 9 Plus students made an active contribution to these students' success in 2016. Common factors across these schools were:

  • knowing the Year 9 Plus students' needs and interests well
  • a shared understanding about the role of the champion and the way they might add value to what the school already had in place
  • a desire from school leaders and teachers to do the right thing for students whose learning was at risk, and an ability to develop actions to achieve this vision
  • systems for monitoring and evaluating the progress of students and other Year 9 students at the school towards success - both learning and wellbeing.

The main constraints to school contribution to success reported by students, schools and champions were:

  • continuing attendance issues for some students
  • length of travel time to school for some students
  • relationships of students with some teachers in some schools
  • the need for some schools to work purposefully with students to develop self-management skills
  • continuing stand-downs for anti-social activity (alcohol or drugs) for some students
  • addressing complex behaviours.

ERO's evaluation identified the following constraints in the practice of some schools that limited champions' contribution to student success in 2016:

  • continuing misunderstanding of some of the trial's key concepts, especially the role of the champion
  • school leadership not giving the trial the attention it deserves
  • a lack of useful data collected to gauge actual student learning progress and achievement
  • setting specific targets for Year 9 Plus students, and little internal evaluation or planning for their future success.

Champion contribution

The key considerations for the evaluation at this stage of the trial were:

  • how well champions forged an initial relationship with students, whānau and participating schools
  • the extent to which champions improved family and whānau engagement in education
  • the extent to which champions supported students' transition and engagement in school
  • champions' influence on the quality of education provided to individual students in 2016.

Building relationships with students and whānau

The champions started the year with an emphasis on building positive relationships with students and whānau. They did this by:

  • early face-to-face contact through home visits
  • taking and sharing key information with whānau about the concept trial and the nature of the support they offered
  • clearly conveying the programme was aimed at helping whānau as well as their children, and that it offered practical support
  • engendering trust by keeping positive, acting proactively and never giving up even if the initial response they received from whānau was negative.

Improving whānau engagement

Over time, champions built whānau trust because they:

  • listened sensitively and maintained contact (same person/same message)
  • worked on sharing positive stories and sharing student successes with whānau
  • acted as a broker between whānau and social services
  • aimed to empower whānau to advocate effectively for themselves and their child
  • used local networks and contacts to support whānau wherever possible
  • were persistent.

Relationships with schools

In developing a relationship with schools, champions initially tailored responses to student needs, prioritising time and attention to their most urgent cases and doing little more than monitoring their lower risk students in the meantime. They also initially 'piggy backed' off existing relationships within schools and set out to find key people in each school who had strategic influence who they could work alongside.

Building and supporting school engagement

As the relationship with schools developed, champions built and supported student engagement by:

  • focusing on student presence and wellbeing at school as their key priorities
  • recognising different schools might have different processes and expectations they would have to fit into and work around
  • going for small gains for students and incremental changes to school processes, where needed, to turn around any distrust of, or lack of confidence in them from teachers
  • making it clear to schools the champion role was to help connect with whānau
  • supporting and advocating for whānau in difficult cases and challenging schools where information was faulty, for example, around attendance or absence patterns
  • making it clear to schools that they were available to both them and to students if necessary
  • responding to student needs, for example, offering a homework club and holiday activities for Year 9 Plus students in out-of-school hours.

Year 9 Plus students universally reported valuing champion support. In accounting for their success in 2016, students and schools agreed on the importance of the champions:

  • focusing early on and strongly on attendance and engagement in school
  • promoting extra-curricular engagement and out-of-school activities
  • checking up on school progress on a regular basis
  • visiting homes when necessary to talk with whānau about both issues and successes
  • providing a continuing focus on student wellbeing when communicating with whānau and school.

Educational impact

Champions appeared to have most impact on the education of students when they helped them with getting to school, aspects of schoolwork such as using the library or access to digital technologies, and by setting both aspirational and day-to-day goals with students. They also had a positive impact on educational outcomes when they promoted engagement in the life of the school and the variety of activities offered, modelled positive adult relationships, and helped by explaining things in ways students understood.

Schools reported the main benefits of champions on the educational success of students occurred when they:

  • started by building on existing positive relationships they had with many students and whānau in the school
  • helped students with goal setting or monitoring progress
  • helped the school understand students' backgrounds and contexts
  • brokered the right social service supports for particular students or whānau needs
  • helped whānau understand what needed to happen next and why when dealing with official procedures
  • proactively maintained contact with whānau and followed up where needed
  • supported students' learning
  • helped students get counselling.

Challenges and constraints to impact

Key challenges champions reported they had to overcome in their work over 2016 were:

  • getting over the slow start in January and February - limiting their impact on students transition into Year 9
  • developing an understanding of Ministry and school processes - they reported receiving limited on-the-job induction about either of these
  • breaking assumptions and misunderstandings of some colleagues in schools and the Ministry that they were just truancy officers or a conduit to social services - they reported that there was no formal introduction made on their behalf to staff in schools by a Ministry official that the school already knew
  • the need to vary their approach for different schools (if working in multiple schools).

Given the intensity of the interactions on a daily basis, many in challenging circumstances, champion wellbeing may need to be considered.

Factors that limited champions' educational impact included:

  • the champion working outside what schools considered as reasonable boundaries
  • a lack of understanding about the role of the champion.

The following aspects of champions' work were still 'a work in progress' and need further attention:

  • elimination of barriers for students and whānau to gain better coordinated access to various social services
  • working relationships with some school leaders and staff in a few schools
  • Ministry communication with schools about progress of the trial.

Conclusions about champion contribution

Year 9 Plus students viewed the champion as a key contributor to their 2016 success. When asked how champions had helped them during the year, many students said their champion had helped them with staying at and attending school, and their wellbeing and behaviour. Most schools recognised the strong contribution of champions to students' success in 2016. Their contributions included: attendance and student wellbeing; relationship building with whānau; and reminding staff of the impact of family circumstances on engagement and learning.

Champions have contributed positively to retention and engagement of many students in this trial. The nature of champions' work to this point has mostly involved keeping students safe and promoting their health and wellbeing so they get to school. This has been time and energy consuming work. Support for champions' needs must be considered for sustainability of the initiative.

Champions reported two major concerns:

  • difficulty in balancing the focus of their work with students and whānau in homes, with that in schools, and the additional demands of liaising with a variety of social services
  • how they might best support students' educational success in their role as champions.

Some schools identified there was a need for:

  • communication about the champion's role, approaches, or specific actions with a particular student in some cases
  • more emphasis by some champions on students' academic progress, as well as social wellbeing
  • more work to clarify respective roles of school and champions in particular situations, for example, students in Activity Centres and Alternative Education and re-integration plans.
ERO has shared the findings from their 2016 evaluations with the parents and whānau of students participating in Year 9 Plus. See Appendix 6.


3  Note: Schools A, B and C are full primary (Y1-8) schools nominating from their senior class; schools D and E are intermediate schools (Y7&8); school F is a Years 7 to 13 school; and schools G and H are composite schools (Y1-13).

4  The references used in this literature review are listed in Appendix 2.

5 Schools selected the students ERO talked to, around a general guideline that we were seeking a cross section of views.

6 Most Year 9 Plus students were in the initial-steps group.

7  In most schools, this 'graduating standard' was made up of a combination of satisfactory Year 9 attendance, behaviour and work completion.