Evaluative findings

School-wide approach to assessment processes and information

How effectively does the school develop and implement an integrated school-wide approach to assessment processes and information?

Effective practice requires that schools make many decisions about how and why they will assess student learning and achievement. They need to develop an agreed understanding among staff members about the purposes of assessment and about appropriate learning and achievement expectations for their students. They need to decide what will be assessed, and how and when assessment activities will be undertaken.

ERO evaluated how effectively schools had developed and implemented an integrated school-wide approach to assessment. This was evaluated in relation to the following indicators: [6]

  • the school had developed clear, school-wide expectations for student learning and achievement that were well-founded and used to inform teaching;
  • assessment processes were closely linked to stated learning priorities;
  • clear rationale and appropriate systems were implemented across the school; and
  • processes were in place to strengthen assessment consistency and judgements.

Review officers also considered any additional or supporting information that was relevant to how effectively the school was developing and implementing an integrated school-wide approach to assessment processes and information.

Effectiveness of school-wide approaches

Sixty percent of schools had developed and implemented an effective integrated school‑wide approach to assessment processes and information. As Figure 1 illustrates, 17 percent of schools were highly effective and a further 43 percent were effective with minor weaknesses in their school-wide approach to assessment processes and information. ERO found that 36 percent of schools were partially effective with substantial weaknesses and four percent were not effective.

Figure 1: Effectiveness of school-wide approach to assessment processes and information

This is a bar graph. The Y axis is called Percent of schools and is ranged from 0-100 at intervals of 20. The x axis has four labels they are Highly effective 17%, Effective 43%, Partially effective 36% and Not effective 4%.

ERO compared the overall effectiveness of the school-wide approach to assessment between urban and rural schools. There was no statistically significant difference. [7]

A comparison of schools by decile groupings revealed statistically significant differences. [8] High decile schools were more effective than medium and low decile schools at developing and implementing their school-wide approach to assessment processes and information. There was no statistically significant difference between medium and low decile schools.

Primary schools were slightly more effective than secondary schools in their school‑wide approach to assessment processes and information. The difference between primary and secondary schools was not statistically significant [9] .

As can be seen from Figure 2, the school-wide approaches to assessment were highly effective in 17 percent of primary schools and five percent of secondary schools. Forty‑one percent of primary schools and 31 percent of secondary schools were effective with minor weaknesses. Thirty-eight percent of primary schools and 49 percent of secondary schools were partially effective with substantial weaknesses and four percent of primary and 15 percent of secondary schools were not effective in this area.

Figure 2: Effectiveness of primary and secondary schools’ school-wide approach to assessment processes and information

This is a multi bar graph. The Y axis is called percent of school and is ranged from 0-100 at intervals of 20. The x axis has two graphs one called Primary and one Secondary. The figures for Primary are Highly effective 17%, 41% and Partially effective 38% and 4%. The figures for Secondary are Highly effective 5%,31% Partially effective 49% and 15%.

Effective practice

Teachers are provided with clear school-wide expectations for student learning. The guidelines for departments are comprehensive and detailed. They detail procedures for departments for identifying barriers to learning and suggestions for developing strategies to address these barriers.

Medium-sized urban secondary school

Teachers are well supported with comprehensive assessment guidelines and expectations for achievement. They meet regularly to discuss assessment procedures, to compare student successes and to moderate their judgements about levels of achievement.

Small urban contributing primary school

In the schools where effective practice was observed, teachers had worked together to develop the school-wide expectations and goals for students, based on aggregated and analysed student achievement data. The expectations and goals set by the teachers were meaningful and specific. They focused on educationally significant learning and were challenging enough to raise all students’ achievement. Where appropriate, there were close links between nationally referenced standards of achievement and the schools’ expectations for students.

Effective schools had developed meaningful targets for their school charter [10] that clarified the learning priorities determined by the school and its communities. In many cases these schools had set targets for specific groups of students, for example Māori students or students in specific year groups. The school leaders and teachers were developing and refining ways to measure students’ progress in these areas reliably and validly.

Teachers’ assessment practices were supported with useful guidelines and policies. The teachers saw the guidelines as ‘living documents’ – the suggested goals and expectations were reviewed, rationalised and improved as new information came to light. They followed an assessment schedule (sometimes called the school assessment plan or cycle) and understood the purpose or purposes for each of the activities in the schedule. Students also understood the purpose of their school’s assessment processes and activities.

Effective schools had developed and implemented systems for ensuring high levels of consistency of judgements between teachers. The teachers reported that establishing effective moderation processes for assessment tasks was a time-consuming activity but had positive benefits. The professional discussions required to arrive at consensus established or clarified the rationale for assessment in a school, increased teachers’ knowledge of their students’ abilities and led to a deeper understanding of the curriculum area and how their students were learning.

Secondary schools were generally more effective at establishing and implementing moderation processes than primary schools. ERO found that about two thirds of secondary schools were doing this effectively compared to about half of the primary schools.

The following process was observed in one secondary school:

In the junior school most departments discuss content and processes involved in assessment tasks. In general, teachers in charge of a particular assessment activity mark a few scripts and then meet with others to confirm judgements. Random scripts are shared among other markers for consistency in marking and a panel will meet in order to give a judgement on borderline decisions. Any further variations in marking are referred to the Head of Department. Reports from the moderation of internally assessed NQF standards are discussed within the department and any differences reviewed.

A further example from a primary school demonstrates how the teachers were using both externally referenced assessment tools and school-designed tasks to measure student achievement.

The teachers use national exemplars and school-developed exemplars when making decisions about students’ levels of achievement. They meet regularly to moderate their work. They plan very carefully what will be assessed and how the assessment will be carried out to achieve consistency.

Secondary schools that were using information from the New Zealand Qualifications Authority (NZQA) well had implemented good systems for collating this data across the school. They used the NZQA information as an external reference point to monitor progress as they compared the achievement of their students with national benchmarks and the achievement patterns of similar schools.

Issues and challenges

Setting appropriate school-wide expectations for student achievement was very challenging in schools that did not have robust systems for collecting and analysing student achievement data. The managers and teachers in these schools were unable to make evidence-based decisions on the learning priorities for their students.

Many large schools, especially large secondary schools, needed to work towards having agreed school-wide expectations for student success. ERO found some schools where this was well established. For example in one school with a roll of nearly 2000 students ERO reported that:

School-wide department programmes and NCEA course guidelines include clear and sound expectations for achievement for Years 9 to 13. Teachers are also provided with sound guidance and expectations for learning and assessment requirements.

In some schools there was a need to rationalise the content and amount of information that teachers collected. For example, some school senior managers required teachers to record large quantities of data about their students. Although some of this data was being analysed and used to support further learning, some information was not being used at all.

A small number of schools had invested in computer software packages to analyse and report on achievement patterns and trends. These packages could assist schools to analyse student achievement information. However, in some schools the principal and teachers were either not familiar with the programme or were unable to interpret the information provided. In other schools the assessment data was not accessible to those who needed it to inform learning.

ERO found, on a few occasions, that assessment practices were used inappropriately. For example, in one school students said they were being trained to achieve in asTTle tests but did not see the relevance of the training for their learning. This situation appeared to arise from competition among the teachers for students to attain high marks in these tests.

The demonstration of students’ achievements and progress

How effectively does assessment information demonstrate students’ achievements and progress?

Teachers, school managers, parents and students need rich and comprehensive information about what students know and can do. They also need information on students’ progress and development.

ERO evaluated how effectively assessment information demonstrated students’ achievements and progress in relation to the following indicators:

  • assessment information demonstrates individual students’ achievement;
  • assessment information demonstrates individual students’ progress;
  • student achievements are referenced to national and local sources of achievement information; and
  • decisions on students’ achievement are based on multiple information sources.

Review officers also considered any additional or supporting information that was relevant to how well the assessment information demonstrated students’ achievements and progress.

Overall effectiveness of schools’ demonstrating students’ achievement and progress

Overall, 58 percent of schools were highly effective or effective at demonstrating students’ achievement and progress. As can be seen in Figure 3, 12 percent of schools were highly effective and a further 46 percent were effective but with minor weaknesses in demonstrating students’ achievements and progress. ERO found that 40 percent of schools were partially effective but with substantial weaknesses and two percent were not effective.

Figure 3: Overall effectiveness of schools’ demonstration of students’ achievement and progress

This is a bar graph. The Y axis is called percent of schools ranged 0-100 at intervals of 20. The x axis has four labels which are Highly effective 12%, Effective 46%, Partially effective 40% and Not effective 2%.

There was no statistically significant difference between urban and rural schools in their effectiveness in demonstrating students’ achievements and progress.

A comparison among schools of different decile groupings revealed that high decile schools were more likely than low decile schools to be effective at demonstrating student achievement and progress. There was no statistical significance between medium and low or medium and high decile schools.

Secondary schools were slightly more effective than primary schools in demonstrating students’ achievement and progress, although this difference was not statistically significant. As can be seen in Figure 4, 13 percent of primary and five percent of secondary schools were highly effective at demonstrating students’ achievement and progress through assessment information. A further 44 percent of primary and 57 percent of secondary schools were effective with minor weaknesses. Forty‑two percent of the primary and 35 percent of the secondary schools were partially effective with substantial weaknesses and one percent of primary and three percent of secondary schools were not effective in this area.

Figure 4: Overall effectiveness of schools’ demonstration of students’ achievement and progress in primary and secondary schools

This is a mutli bar graph. The Y axis is called Percent of schools ans is ranged from 0-100 at intervals of 20. The x axis has two graphs one for Primary and Secondary.  Each has four bars called highly effective, Effective, Partially effective and Not effective. The numbers for Primary are 13%, 44%, 42% and 1% respectively. The numbers for secondary are 5%, 57%, 35% and 3%.

The demonstration of students’ achievement and progress in curriculum areas

The assessment information held by teachers and schools in the eight essential learning areas was evaluated. For each learning area, ERO reviewed both how effectively the information demonstrated students’ achievements and how effectively the information demonstrated students’ progress. Figures 5 and 6 show that:

  • schools were considerably more effective at demonstrating students’ overall achievement than they were in showing their progress; and
  • there was a distinctly different pattern across curriculum areas between primary and secondary schools.

Student achievement

Figure 5 shows that over 90 percent of primary schools were able to demonstrate effectively their students’ achievements in the curriculum areas of English and mathematics. However, only a third of primary schools were able to demonstrate their students’ achievements in other curriculum areas.

In secondary schools the difference among the curriculum areas was not as great. In secondary schools, teachers were slightly more effective at demonstrating students’ achievements in English, mathematics and the arts.

Figure 5: The effectiveness of primary and secondary schools at demonstrating students’ achievement (by curriculum area)

This is a bar graph. The Y axis is called percent of effective schools and ranged 0-100 at intervals of 20. The x axis has a series of labels each with two bars. The two bars represent Primary and Secondary. The labels and percentages are as follows; English 93% and 71%, Mathematics 91% and 69%, Science 37% and 51%, Technology 35% and 44%, Health and Physical education 45% and 48%, Social studies 39% and 48%, Languages 28% and 39%, Arts 36% and 54% and Other 5% and 18%.

Student progress

Schools were much less effective at demonstrating students’ progress in each curriculum area than they were at demonstrating achievement. Much of the information held by teachers showed students’ achievements in work that had been completed recently. Few teachers had analysed student achievement information in a way that showed improvement over time in students’ learning in curriculum areas.

The exception to this was that most primary schools were able to demonstrate students’ progress in English and in mathematics.

Most secondary schools were not able to show students’ progress across the curriculum. About a third of all secondary schools were able to demonstrate students’ progress over time in English and mathematics, but very few could do this in other curriculum areas.

Figure 6: The effectiveness of primary and secondary schools at demonstrating students’ progress (by curriculum area)

This is a bar graph. The Y axis is called percent of effective schools and ranged 0-100 at intervals of 20. The x axis has a series of labels each with two bars. The two bars represent Primary and Secondary. The labels and percentages are as follows; English 88% and 34%, Mathematics 82% and 30%, Science 17% and 13%, Technology 17% and 12%, Health and Physical education 22% and 16%, Social studies 18% and 10%, Languages 17% and 10%, Arts 18% and 12% and Other 1% and 1%.

Issues and challenges

These findings demonstrate that:

  • most primary schools did not collect and analyse their students’ achievements in curriculum areas other than mathematics and English;
  • secondary schools were generally more effective than primary schools at collecting and using assessment information to demonstrate students’ achievements in all curriculum areas, (as discussed later, this was mainly in the senior secondary school and not as prevalent for students in Years 9 and 10); and
  • in both primary and secondary schools, teachers needed to develop processes and systems for collecting and analysing information on students’ achievements over time in all curriculum areas to demonstrate their students’ progress.

Primary schools

Almost all primary schools had made literacy and numeracy key learning priorities. Teachers had, therefore, spent considerable periods of time developing learning programmes, assessment processes and tools in these areas. In most schools the teachers had built a shared understanding of how, when and why to measure student achievement in these areas. There are more assessment tools available to measure student achievement and progress in literacy and numeracy than in other curriculum areas. Teachers reported that they felt more confident in measuring student achievement when they were able to use nationally normed assessment tools to moderate or inform their professional judgements.

In the curriculum areas of English and mathematics, teachers gathered information on student achievement from several information sources. These included formal testing, systematic observations of students in their work and the teachers’ knowledge about their students gained through their daily interactions.

Teachers also used a wide range of assessment tools to compare their students’ achievements with national standards. Commonly used tools included: asTTle; Progressive Achievement Tests (PATs); School Entry Assessments (SEA); six year nets; and assessment tasks from the numeracy projects. Schools also used a range of reading assessment tools. The most common were STAR (Supplementary Test of Achievement in Reading); PROBE (Prose reading observation behaviour and evaluation of comprehension); PM (Price Milburn) Benchmarks; and the Burt Word Reading Test. Schools also used tests of spelling proficiency.

There were some common challenges for primary school teachers in gathering information on students’ progress. These were present to some extent in the areas of English and mathematics, and to a greater extent across all other learning areas.

These common challenges were:

  • identifying the progression of skills and understandings between topics in a curriculum area. For example, in a science programme a teacher may have planned lessons around a theme of birds in Term 1 and volcanoes in Term 2. The assessment information gathered by the teacher might show students’ achievement in the individual unit of work but not their ongoing progress in developing scientific skills or understanding through a sequence of units;
  • gathering good quality information about students’ learning across the breadth of a curriculum area. For example, some teachers had gathered information on students’ achievements in physical activities (such as swimming or athletics) and generalised the information to report progress in the health and physical education curriculum;
  • gathering comprehensive information on students’ achievements. If teachers make their judgements about students’ overall achievements on limited evidence, the information can be misleading. A single assessment activity is unlikely to provide information that is accurate or dependable enough to describe a student’s achievements in a curriculum area; and
  • collecting assessment information that made it easy to have reliable comparisons of student achievement and progress over time and between teachers.

Secondary schools

Secondary schools were more effective at demonstrating the achievement and progress of their senior students than of their students in the junior secondary school (Years 7 to 10 or Years 9 and 10). Teachers of junior students faced the same common challenges listed above for primary schools.

The key source of student achievement information in secondary schools came from the NZQA about students’ achievement in the National Qualifications Framework (NQF).

The school, the students and their families received detailed information from NZQA about students’ achievement in unit and achievement standards. When schools used this information most effectively, the teachers, with the students, interpreted the information to identify the students’ areas of strength and areas for development. The students were aware of the coherence and progression of their learning and did not view their achievements in the NQF as a series of one-off attainments.

In most secondary schools, senior staff supported teachers in analysing the NZQA information. Some schools had specialist analysts who worked with heads of department to analyse and interpret the information. In other schools teachers worked together to interpret this information.

Other than the NZQA information, secondary schools did not use as many nationally standardised assessment tools as primary schools. They most frequently reported using asTTle, PATs and MidYIS (Middle Years Information System).

The interaction of assessment with teaching and learning

How effective is the interaction of assessment with teaching and learning?

The relationship between assessment, teaching and learning is dynamic, interactive and interdependent. ERO evaluated how effectively teachers used assessment information to guide the teaching and learning programme. For this study, this concept has been summarised as the interaction of assessment with teaching and learning.

ERO evaluated the effectiveness of the interaction of assessment with teaching and learning in relation to the following indicators:

  • teachers analysed student achievement information to identify learning needs;
  • teachers used information about students’ achievements to identify the learning needs of students; and
  • teachers advised and guided students to make effective choices about their learning based on assessment information.

Review officers also considered any additional or supporting information that was relevant to the effectiveness of the interaction of assessment with teaching and learning.

Overall effectiveness of the interaction of assessment with teaching and learning

ERO found that teachers were effective in using assessment information to inform their teaching in about half the schools. As can be seen in Figure 7, nine percent of schools were highly effective and a further 43 percent were effective with minor weaknesses. ERO found that 43 percent of schools were partially effective with substantial weaknesses and five percent were not effective.

This is a bar graph. The Y axis is called percent of schools and ranged from 0-100 at intervals of 20. The x axis has four labels they are Highly effective 9%, Effective 43%, Partially effective 43% and Not effective 5%.

ERO compared the overall effectiveness of teachers’ use of assessment information in urban and rural schools and in schools of different deciles. There was no statistically significant difference.

Primary schools were more effective than secondary schools in the interaction of assessment with teaching and learning. This difference was statistically significant.

Figure 8 shows that 10 percent of primary schools and three percent of secondary schools were highly effective in their interaction of assessment with teaching and learning. A further 44 percent of primary schools and 39 percent of secondary schools were effective with minor weaknesses. Forty-one percent of primary schools and 51 percent of secondary schools were partially effective with substantial weaknesses. Five percent of primary and seven percent of secondary schools were not effectively using assessment to inform teaching and learning.

Figure 8: The effectiveness of the interaction of assessment with teaching and learning in primary and secondary schools

This is a multi bar graph. One graph is for Primary and one for Secondary. Each graph has four bars they are Highly effective, Effective, Partially efffective and Not effective. The numbers for Primary are 10%, 44%, 41% and 5% respectively and for Secondary they are 3%, 39%, 51% and 7%.

Effective practice

Information about student achievement is used by most teachers as the basis for planning further learning experiences. Information on students’ abilities in literacy is shared with all teachers so teachers in all curriculum areas know when students may need extra support.

Large urban secondary school

There is comprehensive and frequent monitoring of student progress through the analysis of achievement information. There are clear expectations that this will inform teaching at both classroom and syndicate level.

Medium rural full primary school

In effective schools the teachers had a considerable amount of rich information about their students. They gathered both formative and summative information in many different ways and from a variety of assessment tools, tests, observations and conversations. The teachers were proficient at analysing the information to develop a useful picture of the progress of individual and groups of students.

Achievement and progress information was integral to the teaching and learning programmes of these teachers. The teaching activities were not separate from the assessment activities or from the teachers’ knowledge of their students’ interests, needs and abilities. The relationship of assessment with teaching programmes was not linear (‘plan‑teach‑assess’) but a dynamic interaction of the teachers’ knowledge of their students with deliberate acts of teaching.

The teachers had friendly and professional relationships with their students. They made time during class and formal teaching times to have individual discussions with their students about their interests, strengths and aspirations. In particular the teachers used their knowledge about each student to give them specific and constructive feedback on their learning and progress against standards of expected performance. They gave this feedback in a variety of forms, such as:

  • discussing learning and achievement with students – individually, with groups of students or with the whole class;
  • involving students in discussions and decisions about the assessment activities and purposes;
  • helping students identify strategies that would help them close any gap between their performance and the expected standard;
  • providing feedback information (oral and written as appropriate) for some work samples that showed students what they had done well and how they could improve; and
  • ensuring students were clear about what test scores, grades or marks meant and how they could use the information for future learning.

In the effective schools, teachers had good systems for sharing information with other staff on students’ achievements. They recorded the information and analysed it in ways that were useful for other people, including other teachers, department or syndicate leaders, or school managers.

In one school the teachers analysed aggregated data on students’ achievements and made decisions together about how best to assist groups of students with their learning. In another school where ERO observed highly effective practice, the teaching teams had been organised so that teachers who were proficient in analysing and interpreting achievement information mentored other teachers in these areas.

In the effective secondary schools, there was evidence that the specific learning needs of students in the junior school (Years 7 to 10 or Years 9 and 10) had been identified and addressed.

Issues and challenges

Some teachers did not collect sufficient information on their students’ understanding and achievement for informing their teaching programme. In other cases, teachers gathered little assessment information until the end of a unit of work. Although they were then able to summarise how well students had achieved, there was limited evidence that teachers had adapted their teaching style or programme content during the teaching of unit in response to their students’ abilities.

The assessment processes used by some teachers did not measure the skills they were intended to measure. For example, students’ ability to read for meaning was based on an assessment task that measured the student’s fluency in reading aloud, not their understanding of the text.

The feedback processes used by some teachers could have had a negative impact on their students because of the inappropriate style and content of their comments or marking systems. The information given to students was unlikely either to guide them in further learning or to enhance their concept of themselves as a learner.

Another challenge for schools was the lack of effective processes for sharing information on students’ achievements with other teachers. In some schools, limited information about students was shared between teachers as students moved through the school. Few schools had effective systems for understanding and using information given by other education providers such as early childhood education teachers or other schools. There were also few examples of primary and secondary schools sharing information well.

In addition, only a small number of schools had effective processes for sharing information on students’ learning with other educational professionals. For example, while some Resource Teachers: Learning and Behaviour (RTLBs) had detailed information on the students they worked with, ERO found little evidence that this information was contributing to the classroom teacher’s programme. This was also true to some extent for information gathered by Resource Teachers: Literacy (RT:Lits), reading recovery teachers, and other professionals that supported students learning.

Differences between curriculum areas

Primary and secondary classes differed in the extent to which teachers analysed assessment information to identify learning needs and to make decisions about the learning programme.

In primary schools, over 80 percent of teachers were using assessment information effectively to identify learning needs in literacy. Seventy-five percent did this effectively in mathematics, especially in numeracy. About 15 percent of primary school teachers used information about student achievement to make decisions about their teaching in health and physical education. In all other curriculum areas, that is, in science, social studies, technology, the arts and in the teaching of languages including te reo Māori, fewer than 10 percent of primary schools used assessment information effectively to enhance the teaching programme.

In about half of the secondary schools, ERO found little evidence that teachers were effective in using assessment information to inform their teaching and learning programmes. This was done best in English and mathematics, and least well in science subjects and health and physical education.

In Years 9 and 10, about half of the teachers were effective in their use of assessment information to inform the teaching of English, and about 40 percent in mathematics. In all other curriculum areas, teachers in about 20 percent of schools used assessment information effectively to inform their teaching programme.

For Years 11 to 13 classes, the pattern was slightly different. Across all curriculum areas, between 20 percent and 35 percent of teachers used assessment information effectively to inform their teaching programme.

Students’ use of achievement information for further learning

How effectively do students use information about their achievement for further learning?

When students are well informed about their own progress they are better equipped to make good decisions for future learning. ERO evaluated how effectively students were using information about their achievement for further learning in relation to the following indicators:

  • students knew about assessment processes;
  • students interpreted and used information for further learning;
  • students knew how well they were learning in relation to personal and curriculum goals; and
  • students received appropriate information on what they could do and their next learning steps.

Review officers also considered any additional or supporting information relevant to how effectively students were using information about their achievement for further learning.

Overall effectiveness of students’ use of achievement information for further learning

ERO found that students in about 40 percent of schools were effectively using information about their achievement for further learning. Seven percent of schools were highly effective and a further 33 percent were effective with minor weaknesses in assisting students to use information about their achievement for further learning. ERO found that 46 percent of schools were partially effective with substantial weaknesses and 14 percent were not effective in this area. (See Figure 9.)

Figure 9: Overall effectiveness of students’ use of achievement information for further learning

This is a bar graph. The Y axis is called percent of schools and is ranged from 0-100 at intervals 20. The x axis has four labels which are Highly effective 7%, Effective 33%, Partially effective 46% and Not effective 14%.

ERO compared the overall effectiveness of students’ use of achievement information for urban and rural schools. There was no statistically significant difference for this grouping.

A comparison among schools of different decile groupings revealed that high decile schools were more likely than low decile schools to be effectively assisting students to use information about their achievement for further learning. There was no statistical significance in the difference between high and medium and medium and low decile schools.

Primary schools were slightly more effective than secondary schools in assisting students to use information about their achievement for further learning. This difference was not statistically significant.

Figure 10 shows that nine percent of primary schools and no secondary schools were highly effective in assisting students to use achievement information for further learning. ERO found that 33 percent of primary schools and 31 percent of secondary schools students were effective with minor weaknesses in assisting students to use achievement information for further learning. Forty-three percent of primary and 57 percent of secondary schools were partially effective with substantial weaknesses and 15 percent of primary and 12 percent of secondary schools were not effective in this area.

Figure 10: The effectiveness of students’ use of achievement information for further learning in primary and secondary schools

This is a multi bar graph. The Y axis is called percent of schools and is ranged from 0-100 at intervals of 20.  The x axis has two graphs one for Primary and one for Secondary. Each graph has four bars they are Highly effective, Effective, Partially effective and Not effective. The numbers for Primary are 9%, 33%, 43% and 15%. The numbers for Secondary are Highly effective 0%, Effective 31%, Partially effective 57% and Not effective 12%.

Effective practice

Students readily use the language of assessment and demonstrate their understanding of what it means. They understand the purpose of different types of assessment such as diagnostic testing or summative reporting.

Large urban intermediate school

Students at all levels understand how they are assessed. They receive good quality information about what they will be assessed on and how. Students are increasingly using the provided learning intentions and success criteria to help them monitor their own learning.

Large urban secondary school

Students as young as five years old are taught to self and peer assess in a constructive, meaningful way.

Small rural full primary school

In effective schools students appraised their own learning and achievements. That is, they assessed their own progress. They were also aware of their own academic growth and development and were able to compare their current performance to their past achievements.

The teachers used a range of formative assessment practices in their teaching programme as they gave their students timely information on what they had achieved and what they needed to do next.

There were rich conversations between teachers and students about students’ learning and achievements. The students articulated their progress knowledgeably, knew about the school’s assessment processes and how assessment information was used to improve learning. They discussed their learning and their next learning steps with their teachers in meaningful and appropriate ways.

Students were aware of the learning intentions or what they should know or be able to do as a result of the lesson. Where appropriate, the students worked with the teachers to develop or articulate the learning intentions and the assessment or success criteria for learning tasks. Knowing the success criteria helped students’ awareness of the quality or standard of the work required in order to achieve success in the learning activity.

Students also received feedback during their lessons that helped them to advance their learning. The feedback was effective as it:

  • focused on the learning intention of the task;
  • occurred as the students were doing the learning;
  • provided information on how and why the student had understood and misunderstood aspects of the learning;
  • provided strategies to help the student to improve; and
  • assisted the student to understand the goals of the learning and the expected standards of achievement.

Issues and challenges

In many schools students were not well informed about how well they were achieving or what they needed to do to improve their learning. The teachers did not involve students in decisions and discussions about their learning. Students did not know about the purpose and the expected outcomes of their learning activities.

In some schools ERO found that students who had been identified as high achieving were being taught differently from their peers. Teachers helped high achieving students identify their own learning processes and taught them strategies for self-assessment and meta-cognition. Although this was beneficial for these students, other students were disadvantaged by not experiencing the same high quality teaching strategies.

Some teachers referred to the use of formative assessment strategies in their teaching, but they were not using these strategies effectively. Sometimes what was called a learning intention did not refer to the learning that was expected but instead described the teaching activities. In other classes learning intentions were not presented in appropriate language and were not understood by all students.

In other classrooms students were routinely recording learning intentions without thinking about them. The students could not say how this would improve their learning or affect what they were doing.

In a few classrooms it appeared that the strategy of giving students information on their next learning steps was overused. The students were given so much information on areas that they needed to achieve that they were finding the process overwhelming or demotivating.

In some schools, teachers had given students information about their achievement and progress in ways that were unhelpful and, in a few cases, detrimental to their well-being and their further learning. This occurred when teachers made unhelpful comparisons between students or provided the achievement information to students in ways that were discouraging for the student.

Sometimes students seemed to know about their progress towards a series of very specific goals but were less aware of how well they were progressing generally. For example, students could discuss their achievement in the last unit of work in science but were unaware of their overall progress in science.

The use of school-wide information to improve student achievement

How effectively is school-wide information established and used to improve student achievement?

Assessment information is a key source of information for schools in reviewing the effectiveness of their programmes and resourcing decisions. It gives school managers and trustees evidence of how well the school is meeting the learning needs of all students.

ERO evaluated how effectively school-wide information was used to improve student achievement in relation to the following indicators of good practice:

  • collated assessment information gives useful information about how well students and groups of students are achieving and progressing;
  • information on students’ achievements is used to gauge and monitor the effectiveness of teaching and programmes;
  • information on students’ achievements is used to identify and monitor groups of students who may be of interest or concern; and
  • trustees use information about students’ achievements to inform policy, strategic planning and resourcing.

Review officers also considered any additional or supporting information that was relevant to the effectiveness of the use of school-wide information to improve student achievement.

Overall effectiveness of the establishment and use of school-wide information

Less than half the schools, about 40 percent, were effectively using school-wide information to improve student achievement. As Figure 11 illustrates, 13 percent of all schools in the study were highly effective and a further 31 percent were effective with minor weaknesses in using school-wide information to improve student achievement. ERO found that 44 percent of schools were partially effective with substantial weaknesses and 12 percent were not effective.

Figure 11: Overall effectiveness of the use of school-wide information

This is a bar graph. The Y axis is called percent of schools and is ranged from 0-100 at intervals of 20.  The x axis has four labels which are Highly effective 13%, Effective 31%, Partially effective 44% and Not effective 12%.

ERO compared the use of school-wide information in urban and rural schools and in schools of differing deciles. There was no statistically significant difference.

As can be seen from Figure 12, 13 percent of primary schools and 10 percent of secondary schools were highly effective at using school-wide information. A further 30 percent of primary schools and 36 percent of secondary schools were effective with minor weaknesses. Forty-five percent of primary schools and 43 percent of secondary schools were partially effective with substantial weaknesses and 12 percent of primary and 11 percent of secondary schools were not effective in this area.

Figure 12: The effectiveness of the use of school-wide information in primary and secondary schools

This is a multi bar graph. The Y axis is called percent of schools and is ranged from 0-100 at intervals of 20. The x axis has two graphs on for Primary and one for Secondary. Each has four bars which are labelled Highly effective, Effective, Partially effective and Not effective. The numbers for Primary are 13%, 30%, 45% and 12% respectively. The numbers for secondary are 10%, 36%, 43% and 11%.

There were no statistically significant differences for how effectively primary and secondary schools used school-wide information.

Effective practice

A continual cycle of self review means the school is well placed to ensure improvement in achievement outcomes for students as individuals and collectively in year levels. The school has valid information on student achievement from the collated data that is used by teachers and reported to trustees and parents. Teachers have a sound awareness of student achievement across the school and recognise their own role in improving achievement.

Large urban contributing primary school

Senior managers have established a well-organised, accessible assessment information management system. Longitudinal information is available showing achievement over time school-wide, by subject, by course and by class. Other information is referenced against schools of a similar type, nearby schools and schools with similar approaches.

Large urban secondary school

In effective schools, managers and teachers had established a robust review cycle based on identified learning priorities for their students. Achievement expectations for learning priorities were clear and teachers understood and used the agreed procedures for gathering information on how well their students were achieving. Collated information provided an accurate picture of students’ learning and progress.

Some teachers and school leaders used this rich information to identify groups of students who were not achieving as well as expected. They monitored the achievement and progress of these and other selected groups of students.

Effective secondary schools compared school achievement information with data provided by NZQA to review the effectiveness of their programmes.

In addition to programme review, the information was used to:

  • inform policy decisions, charter targets, strategic planning and resourcing decisions;
  • appraise teachers’ performance;
  • identify departments or syndicates that required additional support; and
  • review the effectiveness of special initiatives, for example, provide extra tuition for students with learning difficulties, improve literacy, or provide extension opportunities for students who were achieving highly.

Issues and challenges

The lack of high quality information that could be satisfactorily collated to give an accurate picture of school-wide achievement was a key barrier to the effectiveness of schools’ use of assessment information effectively to review their teaching and learning programmes.

In some schools, school-wide achievement information was gathered on narrow aspects of course content. The assessment items generated limited information about students’ knowledge and abilities and, in many cases, were not closely linked to the learning priorities of the schools. These schools appeared to be making choices about school‑wide assessments based on what was measured easily rather than on what was important for the school or their students.

In other schools the measures used for determining and reporting overall student achievement were too general. These did not provide an accurate picture of the achievements of all students. The reports concealed information about groups of students who were later found to be underachieving.

In many schools, trustees, leaders and teachers did not have the statistical knowledge required to analyse and interpret school-wide achievement information accurately. As a result, teachers spent a lot of time testing students and preparing reports that were of little use or developed incomplete or misleading conclusions. Principals reported that it was particularly difficult to prepare useful and meaningful information on school-wide trends when a considerable proportion of the roll was made up of children who moved schools frequently.

Information on student achievement enables boards of trustees to make decisions about programmes and resourcing. In many schools, trustees identified areas of concern about the achievement of individual or groups of students and approved the implementation of initiatives designed to meet the needs of those students. Few schools, however, systematically reviewed the effectiveness of those initiatives. There was also a small proportion of schools where ERO found some resistance by staff to using available student achievement data for school review.

Although some school leaders gave boards information that identified issues, trustees did not respond to recommendations. For example, in one school the principal reported that Māori students’ progress was of “grave concern” but the board took no action in response to this message.

In some schools, school managers reported overall student achievement to the board to meet a compliance requirement but did not then use the information to review and improve learning programmes. In one school, the principal and trustees said that the setting of planning and reporting targets was “a paper exercise for the Ministry of Education” and they did not use the information gathered for this in their decision-making.

In very small schools, ERO found variation in the extent to which school-wide achievement information was collated and analysed. For example, in one school of 14 students the board received high quality reports that demonstrated trends and patterns over time. In another school of 20 students the principal cited the size of the school as the reason for not presenting school-wide achievement reports to the board or trustees. Classroom data in small schools gives good information to the principal and can be discussed with the board of trustees and the analysis of student achievement information over time can provide important and useful information for future planning.

Identification of groups of students

Despite widely known evidence that some groups of students are not achieving as well in our schooling system as others[11] only a very small proportion of schools had effectively analysed information on the achievement of specific groups of their students. Primary schools were more likely to have done this in aspects of English or mathematics, although few primary schools had this information in other learning areas. Some secondary schools had analysed the information provided by NZQA to identify the progress of specific groups of students. They were less likely to do this for groups of junior students.

Few schools analysed information to identify the progress of groups of students who made up significant proportions of the school’s roll. Seventeen percent of schools used assessment information to make decisions about the learning needs of Māori students. Only five percent of schools did this for students for whom English was a second or additional language.

There was very little evidence that this occurred in schools for other groups of students, including those schools with relatively high proportions of students belonging to specific ethnic groups. For example, although in one primary school 17 percent of the students were Korean (99 students) and in a secondary school 30 percent of students were Indian (546 students), these schools did not analyse information to monitor the progress of these groups of students.

About 11 percent of the schools used assessment data to monitor the achievement of boys (or certain subgroups such as Māori boys), often at specific year levels. In contrast, only two percent monitored the achievement of girls.

Some schools identified groups of gifted and talented students that would benefit from extension work of some kind. It was far more common, however, for schools to identify low achieving or ‘at risk’ groups, in literacy and/or numeracy. Approximately two‑thirds of the schools, both primary and secondary, identified groups of students that were underachieving in literacy (typically reading, writing and/or spelling). Also, nearly one third of the schools identified groups that were underachieving in numeracy.

Reporting to the community

How effectively is information about students’ achievement reported to the community?

The National Administration Guidelines [12] state that each board of trustees, with the principal and teaching staff, is required to:

report to students and their parents on the achievement of individual students, and to the school's community on the achievement of students…

ERO evaluated how effectively information about students’ achievements was reported to the community in relation to the following indicators of good practice:

  • teachers share good quality, relevant information about achievements and progress with parents;
  • parents receive good quality, relevant information that is helpful for supporting their child’s next learning steps;
  • parents are informed of the school’s assessment processes;
  • the school seeks and values parents’ opinions and ideas when developing and reviewing assessment and reporting processes; and
  • the school’s Māori community/whānau are consulted to develop and make known targets and plans to improve the achievement of Māori students.

Review officers also considered any additional or supporting information that was relevant to the effectiveness of reporting to the community.

Overall effectiveness of the reporting achievement information to the community

As can be seen in Figure 13, seven percent of all schools in the study were highly effective and a further 43 percent were effective with minor weaknesses at reporting students’ achievements to the community. ERO found that 39 percent of schools were partially effective with substantial weaknesses and 11 percent were not effective.

Figure 13: Overall effectiveness of the reporting of achievement information to the community

This is a bar graph. The Y axis is called percent of schools and is ranged from 0-100 at intervals of 20. The x axis has four labels and is Highly effective 7%, Effective 43%, Partially effective 39% and Not effective 11%.

There was no statistically significant difference in the overall effectiveness of the reporting of achievement information to the community for urban and rural schools.

A comparison between schools of different decile groupings revealed that high decile schools were more likely than low decile schools to be effective at reporting achievement information to the community. There was no statistical significance in the difference between high and medium or medium and low decile schools.

Primary and secondary schools were similar in relation to how effectively they reported achievement information to their communities. Eight percent of primary schools and three percent of secondary schools were highly effective in reporting students’ achievements to the community. A further 43 percent of primary and 44 percent of secondary were effective with minor weaknesses. Thirty-nine percent of primary schools and 41 percent of secondary schools were partially effective with substantial weaknesses and 10 percent of primary and 12 percent of secondary were not effective in this area. These findings are presented in Figure 14.

Figure 14: The effectiveness of the reporting of achievement information to the community in primary and secondary schools

This is a multi bar graph. The Y axis is called percent of school and is ranged 0-100 at intervals 20. The x axis has two graphs one for Primary and one for Secondary. Each graph has four labels which are Highly effective, Effective, Partially effective and Not effective. The numbers for Primary are 8%, 43%, 39% and 10% respectively. The numbers for Secondary are 3%, 44%, 41% and 12%.

Schools used a variety of ways to inform parents about their child’s progress. Most commonly these involved a combination of interviews and written reports, usually two per year but sometimes more frequently. One school provided five opportunities for parents to meet with their child’s teachers – at the beginning of the year and then once per term. In almost all secondary schools and many primary schools interviews were a three-way conference with teachers, parents and the student. Many schools used portfolios of students’ work samples as a basis for discussions in interviews.

Effective practice

A wide range of strategies is used to provide parents with good quality information about their child’s progress. The written reports and parent‑student‑teacher conference evenings give parents information about expected achievement levels, age and stage and their child’s actual achievement. Parents also receive information about assessment practices and school expectations for achievement through ‘parent sharing days’.

Small rural full primary school

Teachers share good quality information with parents on their child’s achievement and progress. Parents receive written reports and invitations to meet to discuss progress in three-way conferences twice a year. When teachers have concerns they contact parents to discuss those concerns. Parents know, through contact with teachers and through regular reminders in school newsletters, that they can ask to see teachers at any time. Parents are well informed on progress and how they can help their child’s learning.

Medium-sized urban contributing primary school

Many schools, particularly primary schools, reported high levels of parental attendance at the times provided by schools for parents to discuss their children’s progress. One school reported that grandparents, as well as parents and caregivers, attended interviews and reporting evenings.

A key characteristic of the highly effective schools was a purposeful and meaningful consultation with parents about how they received information about their child. The parents were aware of why assessment activities were conducted and what the findings meant for their child. They were also well informed about how the school was working to meet the child’s interests, aspirations and learning needs and how the partnership between home and school could be enhanced. The schools monitored parents’ continuing satisfaction with the reporting process through activities like surveys, focus groups, random sampling telephone interviews or feedback during parent interviews. They also made specific efforts to meet groups of parents that had not initiated contact with teachers and may have been reluctant to attend reporting evenings.

Effective schools provided parents with comprehensive information on their child’s actual and expected achievement in the New Zealand Curriculum. Parents also had opportunities to discuss next learning steps with the teachers and, where appropriate, with the child. At all age levels, but particularly for secondary school students, the schools promoted students’ responsibility for their learning and were establishing effective three-way partnerships between the student, their family and the school.

Where portfolios were used, parents could see their child’s progress from the work samples included. The parents and the student were aware of the purpose of the work sample and the actual achievement of the child against their expected achievement, the skills that had been mastered and, where appropriate, suggestions for future learning.

Portfolios were used more often in primary schools. In secondary schools they were more likely to be used in subject areas such as visual arts, aspects of technology and, less frequently, English.

As well as making arrangements to meet the parents of individual students, there were opportunities in some schools to meet with parents of groups of students. In particular ERO found that some teachers made special arrangements for meeting the parents of groups of students such as those who identified as Māori, or as Pacific, or those who were achieving very highly. In some cases these meetings were held off the school grounds, for example, at a local hall or marae.

A few schools were also very effective at sharing information with the wider community. The schools presented information on assessment processes, the school’s learning priorities, expectations for student achievement and achievement trends and patterns. School websites were a useful tool for informing parents and families about school life and, in some cases, schools used the website to communicate information on students’ achievements and to celebrate particular successes. Other schools used school community gatherings such as meetings of the parent teacher association, board of trustees’ meetings and school prizegivings to give information to their community.

Issues and challenges

While almost every school provided parents with some information about their child’s learning, the information given by many schools was of limited value. ERO identified the following issues for schools’ reporting to parents:

  • providing a description of activities the student had undertaken (for example, projects completed or a list of units of work), but limited information on achievement;
  • giving information on the student’s recent achievements but no information on actual or expected progress in that learning area;
  • reporting achievement on a grading system used by the school, such as a 1 to 5 scale, a below/meets/exceeds expectations scale, or a NAME scale (not achieved, achieved, merit or excellence), with little guidance on what each grade meant in each subject area or how the scores had been reached;
  • having a lack of inter-subject reliability: parents could not be assured that a grade given in one curriculum area or by one teacher could be considered equivalent to a grade given by another; and
  • using a report format that prevented teachers from providing useful information about the student’s progress, for example when sections of the school’s report form contained predetermined comments and the teachers ticked beside the ‘best fit’ option.

In some schools the teachers were unable to prepare meaningful reports on students’ achievement and progress as they did not gather good quality assessment information that indicated validly what the students knew or could do. Some teachers reported that, while they specifically measured students’ progress in some areas such as reading or mathematics, their reporting in other curriculum areas relied on ‘best guess’ observations rather than analysed achievement information.

Managing the assessment and reporting process was particularly challenging for secondary schools. The range of timetabling options and subject choices available to students could make the preparation of individual students’ reports administratively demanding. The task of coordinating the reporting process, writing report sections, collating other teachers’ sections, proof reading and monitoring the quality of the report was very time-consuming.

In some secondary schools the reporting cycle was out of sequence with the assessment cycle. Reports were sent home to parents with very limited information because teachers’ assessments of student achievement came at the end of units of work that had not been completed before the reporting date. The schools had not aligned the assessment and reporting processes in a way that ensured that parents were well informed.

ERO found instances where schools’ reports to parents were not written in language that parents could readily understand. Many schools had families who were not fluent speakers or readers of English. In some cases the use of educational jargon was a barrier to parents in understanding the reports. In other cases, the information was presented using codes referring to other documents such as the curriculum statements. Although parents could come to the school to look up the information in the relevant curriculum statement, they were not easily able to interpret the information. These schools needed to make greater efforts to provide information in ways that parents could understand it.

The information schools gave to their community also varied in quality. In some schools, limited information on assessment processes and overall achievement was provided. In other schools, the information was selective and, while useful for marketing the school, did not represent an accurate or complete picture – that is, the community was only told of successes.

Reporting to the Māori community

The National Administration Guidelines [13] say that schools must, in consultation with the school’s Māori community, develop and make known to the school’s community policies, plans and targets for improving the achievement of Māori students.

Some schools worked effectively with the Māori community to develop plans and targets to improve the achievement of their Māori students. They consulted with parents and advised them about school initiatives to improve learning and achievement, and the ways that they were endeavouring to meet the needs of Māori students specifically. For example, in one school ERO reported that:

The school has undertaken comprehensive consultation with its Māori community. Parents are invited to hui to communicate plans and targets to improve achievement for Māori students. They receive information on the curriculum, how it will be taught, and are consulted over achievement expectations and how the school can work with families.

Medium-sized urban secondary school

Other schools reported that their consultation with the Māori community had not been successful. Where ERO found less effective practice, the schools had interpreted a low level of response to invitations as a lack of interest and had not sought different ways to reach this community.

Although schools with high proportions of Māori students were more likely to be consulting effectively with the Māori community, this was not always the case. In one school ERO found that, although 8 percent of the roll was Māori, there was limited communication between the school and the Māori community. In another case, 16 out of about 700 students were Māori (less than 3 percent) but the school had established a very effective relationship with the Māori community.

It appeared that having a key link with the community increased the effectiveness of the communication process. In some schools the consultation and liaison with the Māori community was the responsibility of one trustee or one staff member, so when that person was no longer available the school had been unable to sustain the relationship.

In many schools ERO found that, although the school reported a consultation process, there was little evidence that this was more than an information exercise. There were no means of gathering opinions and suggestions, or evidence of actions taken as a result of the consultation. The methods used during meetings did not allow for two-way information sharing or effective consultation over policies, plans and targets to improve achievement. This was particularly true when, for example, the school gave information or announcements at kapa haka or cultural presentations.

Other schools said that they relied on informal conversations at events such as sports matches for consulting with Māori parents and community.