Section Two: Evidence of improved assessment literacy since 2007

Through a decade of national evaluations, ERO has identified a trajectory of improvement in the collection and effective use of assessment in primary schools from 2007 to 2016.

2009 Reading and Writing

ERO’s report Reading and Writing in Years 1 and 2 (ERO, 2009) showed more teachers of Years 1 and 2 children collected and used assessment information for their reading and writing programmes than in the 2007. Thirty‑two percent made very good use of their reading assessments to inform teaching, while seven percent made little use. The use of assessment in writing was a little lower, with 27 percent of the teachers making very good use, and 19 percent making little use, of any assessments.

However, the report reinforced and expanded on many of the assessment issues found in schools during the previous 2007 evaluation. In the best schools, leaders used their data to inquire into what teaching practices were working, whether these should be modified, and where resources were needed to help children who were not succeeding. Teachers were adept at using a variety of assessment sources to make judgements about children’s literacy progress and achievement. They also applied a ‘teaching as inquiry’ process to find out what children had already learnt, and what changes to their teaching were required based on what children needed to learn next.

Teachers who did not understand or use reading and writing assessment processes well, were more likely to focus on whole‑class teaching and activities without a strong instructional literacy emphasis. They used assessment sporadically and did not use the information gained to reflect on or improve their practice.

One of the key issues identified was unclear expectations from leaders and teachers about good achievement for Years 1 and 2 children.  Many had minimal expectations for children’s achievement in their first two years at school. Only 26 percent of schools had an expectation that Year 1 children would achieve to the reading level later set as part of National Standards (Green, Levels 12‑14).  Furthermore, about 29 percent of schools had an expectation that matched the end of Year 2 National Standards (Turquoise, Level 17‑18). In many of the remaining schools, expectations were unclear, or very low.

For writing, 33 percent of schools used exemplar levels from The New Zealand Curriculum Exemplars for Year 1 children, and 28 percent of schools used them for Year 2 expectations. The remainder of schools had either no writing expectations, had unexplained expectations or focused only on a narrow part of writing development.  Our report highlighted the need for teachers, school leaders and board members to be clear about their important roles in setting achievement expectations and monitoring how their teaching practices and processes help Years 1 and 2 children to be successful young readers and writers.

2010‑2012 working with National Standards

Over the next few years, ERO saw a considerably increased focus on assessment across primary schools. ERO’s report Working with the National Standards to Promote Students’ Progress and Achievement (ERO, 2012a) was one of the final reports of a series published over three years about National Standards. In 2011, most schools were making progress with understanding and working with National Standards. Twenty‑two percent (97) of the 439 schools included in this evaluation, were working well with the National Standards. Fifty‑nine percent (258 schools) were developing systems and processes to work with them; 19 percent (84 schools) were opposed to the standards and not working with all the requirements associated with implementing them.

Another 2012 ERO publication, Reporting to Parents: National Standards Years 4 to 8 (2012b) highlighted that the new National Standards reporting requirements had led many schools to review their reporting formats. Often this consultation had included parents and students.

Seventy‑two percent of the schools ERO investigated had met the reporting requirements as set out in the National Administrative Guidelines. The previous year, 60 percent of schools in the sample had met the requirements.

2011 and 2012 Teaching as Inquiry

In The New Zealand Curriculum, teaching as inquiry is described as a cyclical process in which teachers identify the learning needs of groups of target students, and respond to them through planned programmes. These programmes are subsequently evaluated for their impact on student outcomes, leading to programme changes if the teaching has not had the desired impact. It may also identify new target groups of students. 

Inquiry practices are usually used in the classroom by individual teachers, or amongst groups of teachers working towards a common goal; the focus is the progress and achievement of all learners.  Inquiry is particularly beneficial for accelerating the progress of priority learners who are not achieving well.  Māori and Pacific students, students with learning needs and students from low socio‑economic backgrounds make up a large proportion of these learners.  Teaching as inquiry, put into practice well by teachers, and supported effectively by school leaders, has the potential to make a significant difference for these students. 

Levels of Support and guidance for Teaching as Inquiry in primary schools (by percentage)

Highly informative and able to support decision making

28%

Somewhat informative and able to support decision making

35%

Minimally informative and able to support decision making

27%

No Teaching as Inquiry

 10%

 

In 2012, ERO identified many schools were using Teaching as Inquiry to investigate the impact of the decisions and practice on students in the report Teaching as Inquiry: Responding to Learners (ERO, 2012c).  In the most successful schools, leaders had created routines and protocols that facilitated discussion about student achievement and teaching practice. 

In the schools where Teaching as Inquiry was highly informative, research projects in self‑selected areas were carried out by individual and groups of teachers who analysed student data, set targets for groups of students whose progress needed to be accelerated, and reviewed outcomes for those students.

In other schools, while teachers had anecdotal information about the effectiveness of their teaching, they seldom based their claims on evidence of improved outcomes for students. Evaluation documents instead contained descriptions of teaching and learning activities, and students’ reactions to them. In some schools, end‑of‑unit evaluations or compliance with the school’s appraisal systems did not contribute to any improvements for students.

2013 Mathematics Years 4 to 8

ERO reported in Mathematics in Years 4 to 8: Developing a Responsive Curriculum (ERO, 2013a) that the use of schools’ assessment data by trustees, leaders, teachers and students was still highly variable. Often the focus of the schools’ self review neglected to look at aspects of teaching practice that might have impacted on achievement outcomes.  Leaders often addressed the ‘what’ (content) of the curriculum that should be taught, without considering the ‘how’ (teaching approaches and strategies) or the ‘so what’ (outcomes for students).  Many schools failed to adapt their curriculum to respond to successes and challenges identified in their assessment data. Their predetermined or prescriptive curriculum did not always match the identified strengths, interests and learning needs of the current group of students.

In about half the schools, trustees, leaders and teachers had assessment information they were able to use well, or make some use of, to inform decisions. Students’ use of achievement information was weaker, as shown below. Over 25 percent of trustees, teachers and leaders were using assessment well compared to less than 20 percent of schools where assessment was not used.

Use of achievement information by trustees, leaders, teachers and students

The table below illustrates the difference in practice when comparing schools where information was well used with those where it was not, for mathematics programmes in 2013.

 

Well Used

Not Used

Trustees

  • Boards received good quality information regularly from school leaders and were active and engaged – independently questioning the data and seeking to further their own understanding. 
  • They used the data to inform resourcing decisions, which were targeted and responsive to areas of need. 
  • Boards also used the information to set appropriate targets to raise achievement and align them with strategic goals.
  • Robust self‑review processes were evident.
  • Boards received some information from school leaders, but this was not analysed and, in some cases, ERO had concerns about the validity of the data. 
  • Boards in this category showed no evidence of considering the information in depth or using it to inform resourcing decisions, strategic planning or target setting.  This was sometimes due to paucity of information, and sometimes due to a lack of board capability. 
  • No evidence of self review.

Leaders

  • Leaders regularly collected and presented comprehensive student achievement information across all strands of mathematics. 
  • Information was analysed to show progress over time and to assess the efficacy of interventions. 
  • The information was used to inform decisions around PLD and curriculum, allocate additional staffing and set targets. 
  • The information was used as part of school self review.
  • Most leaders had not collected and analysed the information. 
  • In many cases ERO had concerns about the validity of the data, or the robustness of overall teacher judgements made after considering achievement information (OTJs). 
  • Data was not used to inform target setting or identify professional learning and development priorities.

Teachers

  • Teachers collected high quality data from a range of sources to inform their OTJs.
  • This information was used to plan programmes and identify teaching strategies. 
  • They focused on learners requiring additional support. 
  • Teachers showed a commitment to and understanding of teaching as inquiry. 
  • They provided regular opportunities to involve students, parents and whānau in learning conferences and goal setting for mathematics standards.
  • Teachers were either: making minimal use of assessment information to teach students, with no clear link to the mathematics standards; or making no use of assessment information to inform their planning and practice.

Students

  • Teachers had explained the mathematics standards for students.
  • Students were therefore able to use assessment information to reflect on their own learning. 
  • Students could talk about where they were in relation to the standards and their next steps. 
  • Students took an active role in
    goal setting and participated fully in learning conferences along with teachers, parents and, whānau.
  • Students were well supported by teachers to understand their achievement.

 

  • Students were not aware of how well they were achieving in relation to the mathematics standards or informed about their next steps for learning. 
  • They had limited or no knowledge of the standards. 
  • In some cases teachers did not share information with students. 

 

ERO also identified that although many schools were using their data to identify students who needed additional support; they tended to use the same teaching strategies and adopted a ‘business as usual’ approach to how they responded to the identified students. They used practices such as ability grouping (streaming across classes) and/or used teacher aides to support these students, without any evaluation of how these practices improved outcomes for the children involved. 

2013 Priority Learners

ERO’s report Accelerating the Progress of Priority Learners in Primary Schools (ERO, 2013b) evaluated how well teachers, leaders and trustees contributed to improved outcomes for priority learners. This evaluation focus extended beyond the use of assessment to investigate teachers’ confidence with strategies to respond to students needing additional support. However, teachers’, leaders and trustees’ use of assessment, was key to the support provided for students. Examples of good practices found are shared below.

Teachers with many highly effective practices used assessment data well to identify those students for whom they needed to accelerate progress. They had good knowledge of their students’ strengths and needs. Teachers developed flexible, responsive learning plans for individuals and groups of students. They were reflective practitioners and followed an inquiry cycle of teaching and learning by using assessment data to review the impact of their teaching, and changing strategies as necessary.

Principals used achievement data effectively to identify priority groups, to monitor their progress and to evaluate the impact of programmes and systems over time. They drew on the knowledge that their teachers had of individual students in these analysis processes. Leaders benefitted from having information about the specific teaching points that needed to be reinforced, rather than just knowing the numbers and names of students below the National Standards.

In the boards with effectively used processes, trustees were kept well‑informed by the principal about student achievement in general, and received well‑considered recommendations for priority learners in particular. Trustees were committed to raising student achievement. They were active participants in the charter target‑setting process and interrogated achievement data provided by the principal. They allocated appropriate resourcing for programmes to accelerate learning, based on this information.

We found that well over half of teachers, principals and trustees contributed to improvements for priority learners in the sample of schools reviewed. ‘Business as usual’ was no longer seen as good enough. Teachers were reflective practitioners who were constantly looking for better ways to improve their students’ achievements.

Teachers, principals and trustees’ contribution to improved outcomes for priority learners

 

To a great extent

To some extent

To a Limited extent

Not at all

The extent to which teachers contributed to improved outcomes for priority learners

28% of schools

51% of schools

20% of schools

1% of schools

The extent to which principals contributed to improved outcomes for priority learners

29% of schools

37% of schools

33% of schools

1% of schools

The extent to which trustees contributed to improved outcomes for priority learners

17% of schools

48% of schools

32% of schools

2% of schools

The schools that had effectively accelerated students’ progress fully used school‑wide data to determine the specific extra teaching individual students needed. Leaders collated teachers’ analysed data identifying individual students’ specific strengths and next learning steps. Leaders also looked for achievement trends over time to establish how well their systems and programmes were working.

In contrast, in schools where leaders mostly aggregated the numbers of students who were achieving expectations, they lacked the information to decide on their school‑wide professional development or resourcing needs. Issues with the validity, reliability and sufficiency of assessment data meant leaders had difficulties identifying which students needed additional support, and the specific concepts they needed to master to make progress.  The lack of aggregation of data about each individual’s next learning steps meant that in many schools, students may have participated in an intervention that did not teach the concepts and skills they needed to accelerate their progress.

2014 Raising Achievement

ERO’s 2014 report Raising Achievement in Primary Schools (ERO, 2014) provided evidence of considerable improvement in teachers and leaders’ capability to use assessment data to respond to Years 1 to 8 students achieving below expectations.  Half the schools investigated had used deliberate actions to support priority students to accelerate their progress, which had resulted in improved achievement. Many of these schools had not restricted their focus to one particular year group or curriculum area. They focused on students whose achievement needed to accelerate across all year levels, and for mathematics, reading and writing. 

About half the schools had moved beyond merely identifying the students needing additional support, to extending their teaching practices by researching and trialling new approaches. They then closely monitored students’ progress to identify practices that were successful and should be continued. In the best instances, they were sharing the new strategies and approaches to allow more students to benefit. School leaders in many of these schools promoted teamwork and high quality relationships with students, their parents and whānau, and other professionals, to support acceleration of progress. 

Leaders and teachers in these schools were able to explain what had worked for the students and were continuing to use the successful strategies in classrooms. Teachers, leaders and students were energised by their success. Parents and teachers were fully involved and contributing to the improvements.

Schools where the students were active partners in designing the plan to accelerate progress were more likely to improve student outcomes to a greater extent.  Their plans included:

  • learning contexts based on student interests
  • collaborative group tasks
  • a lot of oral work
  • self and peer assessment
  • student feedback to teachers about what worked and what did not.

Many schools developed literacy and mathematics progressions children could understand, that helped them monitor their own progress while describing what they had learnt, what they needed to learn and how they learnt. Parents and whānau were well informed about their child’s need to accelerate progress in reading, writing or mathematics.  This need was explained in ways that made it clear teachers and leaders knew they were responsible for raising student achievement, but needed help from parents and whānau to do so. 

The other half of schools responded to underachievement with ‘more of the same’. However, for some students it was not working.  The schools were effectively identifying the students needing additional support and were using time, effort and resources to provide extra support. However, they did not have specific implementation plans or evaluation processes to determine the effectiveness of their strategies.  Most of these schools were aware of the need to support students to catch up, though some had little sense of urgency. Leaders had not developed a coherent plan to improve achievement that included both long‑term preventative and short‑term remedial responses. Instead, they often focused on short‑term actions that were not well resourced or evaluated for impact. Any gains by students from supplementary instruction programmes were often not maintained, as they did not complement classroom experiences.

Leaders and teachers at the less successful schools tended to work on improvements in isolation, focusing on the student alone. They had analysed achievement data, but used it only to monitor student progress rather than also evaluate the impact of teaching actions. Leaders at these schools knew it was important to develop good learning relationships with students and parents, and often had it as a school goal, but were not willing to be specific in their request for parent and whānau support.  This meant many of the actions to develop such relationships appeared superficial.

2010 ‑2015 Students with Special Needs

The ERO report Inclusive Practices for Students with Special Needs (ERO, 2015a) was the fourth national report on inclusive practices in New Zealand schools. The report identified more schools were more inclusive but there was still room for improvement. The most effective schools used high quality teaching practices, developed high quality individual education plans (IEPs) based on evidence, and responded flexibly to individual needs. Features of good IEPs included:

  • goals based on data and focused on what the student could do, and their strengths and interests
  • well‑developed objectives for student learning and development for social, learning,  communication, physical, sensory, behaviour and life skills
  • SMART (Specific, Measurable, Achievable, Relevant, Time‑bound)goals to help the school show learning, progress and next steps
  • teaching strategies and clear responsibilities for staff
  • regular review of the goals in consultation with parents, staff, specialist teachers, specialists, and sometimes students.

In the best instances, schools collated information about the progress of all students with special education needs in a way that allowed them to analyse where progress had been accelerated, and to identify and share the most successful practices. However, ERO also identified some schools where teachers needed to improve their knowledge of how to modify the curriculum, develop specific IEP goals and use achievement data to inform their teaching for children with special education needs.

An ongoing finding from all four of the reports on inclusive education was that many boards were not well informed about the impact of their resourcing on progress and achievement of students with special education needs. Reports to boards mostly shared what was provided for the students, rather than outcomes for students or the effectiveness of the school’s practices. An analysis of school reports to boards showed that only 15 percent of schools provided any achievement information for students with special needs. Without this information, it was difficult for boards and leaders to determine priorities, decide on specific targets, identify PLD needs, and develop a detailed plan to improve provisions for students with special education needs.

In this report ERO also notes the paucity and weaknesses in the available assessment tools for students with low cognitive ability/functioning within Level 1 of the curriculum.

2015 Learner‑centred Relationships with Parents

The ERO report Educationally Powerful Connections with Parents and Whānau (ERO, 2015b) evaluated how well schools worked with the parents, families and whānau of students at risk of underachievement. Educationally powerful connections and relationships were learning‑focused and supported the two‑way sharing of expertise, in ways that acknowledged, understood and celebrated similarities and differences. Schools that had learning‑centred relationships involved parents, along with their child, to set goals and next steps. Teachers and parents each shared what they knew about the child’s strengths, interests and needs and decided how they would contribute to the child’s goals.

In schools with low quality learning‑centred relationships with parents of students at risk of underachievement, teachers and leaders believed they could only reach a certain proportion of parents, and the lack of involvement of hard‑to‑reach parents was justified. These schools generally did not seek ways to improve parental involvement. In a few schools, there was a pervasive view from teachers and leaders that ‘teachers know best’.

2015 Transition to School

ERO’s 2015 report Continuity of Learning: Transitions from Early Childhood Services to School (ERO, 2015c) found considerable variability in how well services and schools supported children to transition to school, particularly children at risk of poor educational outcomes.  Leaders and teachers in the very responsive schools could demonstrate they had real knowledge about their newly‑enrolled children. They took care to translate that knowledge into providing the best possible environment and education for each child. This enabled smooth transitions to the school.

New entrant teachers in the very responsive schools also quickly found out about each child’s interests, strengths, culture and capabilities before they started school, through:

  • observations in the early learning service and on school visits
  • talking with the early learning service’s teachers and child’s parents and whānau
  • referring to the children’s portfolio or learning story journal.

After starting school, the new entrant teachers learnt about the child through:

  • ongoing observations and discussions with parents and whānau
  • formal and informal assessments.

The relationship with parents was very important and an essential, informal way for teachers to build a complete picture of each child. This picture helped teachers to manage transitions.

The less responsive schools tended towards a ‘one size fits most’ approach. These schools had few strategies in place to recognise or respond to children as individuals with their own interests, strengths and capabilities. They rarely took into account the children’s prior knowledge or learning. In the worst cases, the new entrant child had to fit into a rigid system where no part of that system catered for them as an individual.

Most schools worked well with children with special education needs during transition. School leaders and teachers took time to find out about the children before they reached the school. They set up meetings with appropriate people and external agencies to develop IEPs and made sure applications for appropriate funding were made well in advance of children starting school. However, such good assessment practices were not used as often for other children that may have been at risk of not achieving well.

2015 Charter Targets

Primary school leaders and boards have made considerable improvements in setting and responding to charter achievement targets. The report Raising Student Achievement through Targeted Actions (ERO, 2015d) investigated the extent that targeted actions supported the rate of progress of students at risk of underachieving.

Findings showed 80 percent of primary schools in the sample responded to specific targets, resulting in progress for some or many of the targeted students.

Although many schools had a focus on underachievement when setting targets, they were less effective in taking actions to raise achievement. Two key conditions were required for effective target setting in successful schools. These were having:

  • optimum challenge in the targets, to ‘stretch’ expectations for success
  • maximum visibility of targets, so that those needing to take actions (trustees, leaders, teachers, students, parents and whānau) shared responsibility.

The number of primary and secondary schools effectively setting and responding to targets (during 2014)

Actions too general and not focused on acceleration

Up to 40% of targeted learners accelerated their progress

40‑69% of targeted learners accelerated their progress

Over 70% of targeted learners accelerated their progress

Primary:       57

Secondary:   21

Primary:     110

Secondary: 4

Primary:     79

Secondary: 9

Primary:          64

Secondary:      7

The sample included 310 primary schools and 41 secondary schools.

Some of the most successful primary schools set targets for fewer students, rather than the whole cohort. They had a clear understanding of who they needed to target actions to, and were also able to monitor their actions to determine if they resulted in positive gains. Board members, leaders, teachers, parents and whānau and students all knew what they had to do to make the desired improvement. In the best instances, schools provided targeted support for the students not achieving well and, at the same time, built teacher capability to avoid such underachievement in the future. Both students and teachers in these schools were energised by their visible success.

In the less successful schools, targets were often more generalised, without clearly identifying the students teachers needed to focus on. Targets outlined the percentage of students they wanted to reach the target, without identifying specific needs and actions for individual students. As a result, there was less coherence in teachers’ response to at‑risk students’ needs and interests. In other instances, teachers identified actions but these were not clear or followed through. Individual teachers may have been taking actions to raise the achievement for selected students, but these actions were not coordinated across the school.

Newly Graduated Teachers (2017)

In the report Newly Graduated Teachers: Preparation and Confidence (ERO 2017b), ERO found newly graduated teachers tended to be more confident about their content and pedagogical knowledge than their ability to use assessment data to show progress, plan strategies and refine their practice. Nearly one‑third of teachers that completed the survey were only somewhat confident or not confident at all to use data to inform their planning and teaching. Secondary school teachers rated themselves as more confident than primary school teachers.

Assessment, and its analysis and use to inform teaching and learning, was a common area that needed strengthening. Leaders told ERO that newly graduated teachers often had little understanding of assessment tools, moderation, data analysis or data use. They said NGTs’ knowledge and understanding was dependent on what they learnt on practicum, and many were learning about assessment ‘on the job’.

Although much progress is evident in leaders and teachers’ capacity to collect and use assessment, ERO reports have continued to identify considerable variability in assessment capability between schools. This variability continues to impact negatively on individual children and their families, and New Zealand’s national and international achievement results.  One of ERO’s most recent reports also highlights issues with assessment confidence for some newly graduated teachers that could contribute to ongoing variability in the future.

Shifts in Practice 2007‑2017

The table below summarises some of the shifts in assessment practice in effective schools from 2007 to 2017.  These improvements, while encouraging, are not yet universal.

Teachers working together in a professional community

FROM 2007

TO 2017

In the effective schools, teachers had good systems for sharing assessment information about student achievement with other staff.

Teachers analysed the data together, asked challenging questions and suggested ways to respond to the needs they identify together.

Some teachers gained little information until the end of the unit of work and summarised achievements without adapting their programmes in response to their students’ abilities.

Teachers in over half the schools collected mathematics data, and used it to identify students’ achievement and plan responsive programmes.

Many teachers had effective systems for identifying students at risk of not achieving and provided interventions to support them. However, few teachers checked the impact of the intervention on outcomes for students.

Teachers in successful schools had a case management approach for students at risk of not achieving that meant each student’s progress, strengths and needs were regularly discussed and the effectiveness of teachers’ responses were regularly explored.

 

Leaders

FROM 2007

TO 2017

The measures used for determining and reporting overall student achievement were too general.

Leaders worked collaboratively to analyse school‑wide data, to determine the diverse and specific needs of students at risk of not achieving.

In some schools, assessment generated limited information about students’ knowledge and abilities and, in many cases, was not closely linked to learning priorities.

In about half the schools, leaders promoted teamwork and high‑quality relationships with students, their peers and whānau. Teachers and leaders were able to explain how others could help them raise achievment, while also being clear that they were responsible for student achievement.

Some school managers reported overall student achievement to the board, to meet a compliance requirement, but did not then use the information to review and improve learning programmes.

 

In effective schools, leaders used the required planning and reporting tools towards key goals, set targets, focus internal evaluation, plan interventions and reduce disparity.

 

 

Boards of Trustees

FROM 2007

TO 2017

In many schools, trustees, leaders and teachers did not have the statistical knowledge to analyse and interpret school‑wide achievement information accurately.

In effective schools, trustees demanded achievement‑based reports about the impact of their resourcing.

 

 

Students

FROM 2007

TO 2017

 

Students often received superficial feedback comments in their books or writing portfolios that mainly praised effort and neatness.

Students used rubrics and information from assessments to reflect on their learning and took an active role in goal setting. They were able to explain how they were progressing and achieving.

Many students were not well informed about how well they were achieving, or what they needed to do to improve their learning. Students were not involved in discussions. Learning expectations were not clear and sometimes only described teaching activities.

In effective schools, students involved in an intervention knew why they needed to catch up, that teachers believed they could succeed, what their goals were, what worked for them and how they were going.

 

Working with parents

FROM 2007

TO 2017

Schools used a variety of ways to inform parents about their child’s progress, most commonly a combination of interviews and written reports, usually two per year but sometimes more frequently.

Teachers and leaders at schools with successful working relationships with parents and whānau of students at risk of underachievement expected parents to be involved, and knew that the school's role was to help parents be involved. There was a sense of manaakitanga ‑teachers and leaders recognised their responsibility to care for the wellbeing of parents and whānau when working together.