Appraisal as a catalyst for improved learner outcomes one year on

Overview

The Education Council of Aotearoa New Zealand (the Council) was established on 1 July 2015 to replace the New Zealand Teachers Council. The new entity was set up as an independent statutory body with an expanded role.  As well as its teacher registration functions, this included providing professional leadership to the profession, establishing a code of professional responsibility, maintaining a disciplinary regime for teachers, and undertaking quality assurance functions particularly around the setting of standards for entry into the profession and for ongoing practice.

One of the sixteen functions outlined in the enabling legislation for the Council was the requirement to audit and moderate the appraisals of at least 10% of the practising certificates issued or renewed each year.  This focus was to ensure that appraisals that were the basis for teacher certification achieved a “reasonable and consistent standard[1]”. On the first day of its operation the Council signed an agreement with the Education Review Office (ERO) to undertake this work on behalf of the new Council.

This report is based on the first year results of this contractual work. ERO completed just over 4000 individual audits across 841 institutions (schools and early childhood services), representing approximately 10% of the endorsements made by professional leaders. The audit was undertaken in all regular education reviews across New Zealand where there had been teachers endorsed for a practising certificate in the twelve months prior to the review.  

The data gathered in the first year of the Audit Agreement will provide a baseline from which future progress can be tracked.  Overall, 73% of the endorsements were found to be based on meaningful appraisal processes. The level of consistency within a school or service was sometimes more variable. In 68% of schools and services, professional leaders achieved a reasonable and consistent standard across the institution. There was a steady improvement in compliance, quality and consistency throughout the first year.

Next steps

ERO recommends that:

  1. The Council continues to rationalise and simplify the expectations for teachers’ professional practice
  2. Information about appraisal of teachers and leaders is aligned and consistent across the education sector, education agencies and organisations
  3. Consideration is given to extending provision of professional learning opportunities in appraisal workshops as these have been deemed highly practical and valuable by those who have already attended them.
  4. School leaders continue to develop effective appraisal systems and processes that focus on the learner and promote continuous improvement in the quality of teaching.
  5. Teachers take professional responsibility for curating evidence from their everyday practice to show that they continue to meet the Practising Teacher Criteria. 

Introduction

One of the important changes included in the Education Amendment Act 2015 was the separation of teacher registration (that recognises that a person is qualified and fit to be a member of the profession) from the issuing of practising certificates (that recognises the competencies and experience required to work as a teacher). 

This separation reinforces renewal of the practising certificate as a way to assess the continued competence of teachers. It also provides the opportunity to align appraisal processes consistently with the Standards required for endorsing practising certificates.

The Council is required under s328 (1) (i) Part 32 of the Education Act 1989 to ensure that appraisals made by professional leaders for the issue and renewal of practising certificates achieve a ‘reasonable and consistent’ standard. To achieve this, the legislation requires that the appraisals for at least 10% of the practising certificates issued or renewed each year are audited and moderated. ERO was contracted by the Council on 1 July 2015 to carry out this new audit and moderation function for an initial three-year term.

The audit and moderation work has an accountability function, ensuring that professional leaders are carrying out their appraisal responsibilities in accordance with requirements, and a focus on improving the use of performance management by professional leaders to ensure high quality teaching and leadership, lifting the status of the profession.

Background

Since the early 2000s, ERO has included appraisal requirements as part of education reviews.  These requirements are laid out in an Assurance Statement format that covers the regulatory requirements for the day-to-day operation and management of a school or early learning service.

Boards of Trustees are asked to attest that they are meeting the requirements in relation to appraisal based on the Practising Teacher Criteria, or to indicate where there are aspects that they are unsure about. Similarly, ECE managers attest they are meeting these requirements or indicate aspects of which they are unsure.

ERO was well placed to undertake the audit and moderation function on behalf of the Education Council in addition to its existing interest in how well professional leaders use appraisal to improve the quality of teaching and outcomes for children and young people.  ERO began this work in July 2015.

Based on the endorsement by the school or service leaders and the evidence available during the on-site evaluation, review officers determine the extent and depth they need to investigate, particularly where there are concerns about performance or the system as a whole.

What ERO knows about effective appraisal

Previous ERO national evaluation reporting

ERO has published national evaluation reports on the quality of aspects of performance management and appraisal since 2000. The most recent reports were published in 2014.[2]  The suite of five reports included evaluations about effective principal and teacher appraisal, board employment responsibilities and employment responsibilities in kindergartens and education and care services.

The reports found that the schools reviewed had compliant systems for appraising teachers that included all the accountability aspects required. However, there was limited evidence of appraisal systems operating as an integral component of overall school improvement. The appraisal systems did not contribute sufficiently to improving teacher capability and student outcomes. Effective appraisal should be a component part of an internal evaluation framework that focuses on improving achievement for all students in the school.

There were similar findings in relation to the appraisal of principals. Most principals in the sample had been appraised as required and this process generally contributed well to the principal’s own development. However, the overall effectiveness of appraisal needed to have greater impact on leading teaching and learning.

In the evaluation of employment practices in early childhood services, ERO found that the most effective appraisal practices focused on systematically developing staff and improving teaching practice. In these services, well-documented processes were aligned to the Registered Teacher Criteria (now the Practising Teacher Criteria) and included teacher reflection, observations and regular monitoring of progress towards identified goals. However, in many services, appraisals were carried out as a one-off activity rather than being part of a coherent, ongoing process. In these services, appraisal was not necessarily viewed as a professional activity to improve and develop practice.

ERO’s findings highlighted the need for appraisal processes to be embedded in a performance management system that had a clear focus on building staff capability and improving teaching practice. Appraisal goals needed to be linked to the service’s vision, strategic plan and goals, and professional development to have the greatest impact on teaching and learning outcomes for children.

OECD reporting

The Organisation for Economic Co-operation and Development (OECD) has also commented on New Zealand’s appraisal practices. The OECD report on New Zealand’s evaluation and assessment framework for improving school outcomes[3] noted that although annual appraisal for principals and teachers had been a requirement for many years, its implementation was variable across schools.  The report recommended several areas for improvement including a stronger connection between principal appraisal and school development, strengthening the links between appraisal and professional development, and providing training and support for boards and principals to carry out effective appraisal.  The OECD report noted that ERO reviews had a role in guaranteeing that systematic and coherent developmental appraisal is conducted in all schools. 

Pont et al[4] noted that school leaders play a key role in improving school outcomes by influencing teachers’ motivation and capacity. They suggested school leaders play a more active role in instructional leadership, including monitoring and evaluating teacher performance.

Requirements for appraisal

The annual appraisal of principals and teachers became mandatory in 1997.  Appraisal is intended to balance the requirements for accountability with a focus on improvement.[5] Appraisal informs recommendations about applications for full certification (previously registration) and for renewing practising certificates (as well as guiding salary progression).

The mandatory appraisal requirements currently consist of two sets of teaching standards. The Practising Teacher Criteria (Practising Teacher Criteria | Education Council) set out mandatory criteria for teachers seeking to gain full registration or renew their practising certificates.  The criteria describe elements of professional knowledge in practice and the relationships and values successful teaching requires.  These elements are common to all teaching, regardless of the context or teachers’ experience.

The Professional Standards set out the Government’s expectations of professional performance.  The standards are included in the various industrial agreements as a teacher’s progression on the salary scale is linked to their achievement of the standards.  In 1998 the Ministry published the professional standards for primary school teachers, and in 1999 a similar set of professional standards and criteria was published for secondary school and area school teachers.

Appraisal involves setting goals, planning how to meet them, identifying indicators of progress or achievement, and discussing the impact of teaching practices on learning. Clear linking of appraisal with the strategic direction, goals, priorities and targets, and appraisal goals, means leaders and managers can prioritise their planning for improvement and allocation of resources, particularly professional learning and development activities. Leaders have a pivotal role in ensuring appraisal procedures are working well, appraising teaching staff, and using the results of appraisal to improve teaching.

The audit of appraisals undertaken by ERO on behalf of the Council is the first systematic investigation into how well professional leaders are following the specific requirements of the Council in endorsing applications for the issue or renewal of practising certificates. 

The audit of appraisals in schools and early learning services

ERO’s external evaluations have a strong improvement focus in both schools and early learning services. We have had a long established interest in how well professional leaders assess and manage the performance of their teachers. A key focus of ERO’s audit and moderation is on determining the extent to which the leaders responsible have been able to achieve a ‘reasonable and consistent standard’ in the decisions they have made in endorsing the issue or renewal of practising certificates.

In conjunction with the Council, ERO developed a set of indicators of good practice that frame the discussions about the quality of the appraisal process and its use in endorsing the issue and renewal of practising certificates (See Appendix 1).

All teachers whose practising certificate was issued or renewed in the 12 months prior to the ERO review were included in the audit sample for each school or early learning service. Review officers made a judgment about whether the evidence of regular appraisal for each teacher provided a satisfactory basis for the endorsement by the professional leader. Reviewers then considered all the evidence of the school’s appraisal and accountability processes in making judgements about the quality of the school’s/service’s systems for managing and improving the quality of teaching.

ERO sought answers to these overarching questions:

  • Were the endorsements for practising certificates based on meaningful appraisal? (refer to indicators of good practice for appraisal]
  • Do the appraisals by professional leaders in this school/service for the issue and renewal of practising certificates achieve a reasonable and consistent standard? [refer to indicators of good practice for system level]

ERO carried out 4002 audits in 841 institutions between July 2015 and June 2016. These included 416 early childhood services, 296 primary schools, 47 composite schools and 82 secondary schools. A breakdown of the audits is included as an appendix.

What did ERO find?

ERO found that nearly two thirds of the appraisals supporting renewal of practising certificates the first year of the contract were satisfactory. A significant improvement was seen from the beginning of the audit process in July 2015 to the last quarter.

Trends for the 2015-2016 year

 

     July/August    

September

1st quarter    

December

2nd Quarter 

March

3rd Quarter 

June EOY

 

Issues

Satisfactory

60% 71% 85% 65% 77%

Renewals

Satisfactory

58% 58% 65% 71% 65%

Appraisal

Meaningful

69% 70% 70% 68% 73%
Reasonable and Consistent Standard  65% 67% 64% 67%

68%

 

 

It was evident as the year progressed that professional leaders were becoming more knowledgeable about the expectations and requirements for meaningful appraisal. Schools and early learning services received pre-review information about the audits. This explained ERO’s process and directed leaders to the Education Council website for up-to-date information about appraisal and how this process supports the issue and renewal of practising certificates. Professional discussions between leaders and ERO in the course of the audit process and wider education review led to greater awareness in the education sector of the expectations for professional leaders in endorsing teachers for practising certificates.

There is increasing understanding of the importance of the role of the professional leader. As part of the endorsement process the professional leader is attesting on behalf of the education profession that teachers in their school or early learning service are either competent to be teaching, or competent to continue as teachers. This endorsement is based on evidence that the teacher has been regularly assessed using, and has met, the Practising Teacher Criteria.

We found the process for those being issued with a full practising certificate was generally more thorough and consistent than for those who were applying for renewal. Over three-quarters of the audited sample was satisfactory. Provision of programmes for provisionally certificated teachers has been well resourced in recent years through research, guidelines and workshops provided by the New Zealand Teachers Council (Education Council). Details of this work can be found on the Education Council website https://educationcouncil.org.nz/content/induction-and-mentoring-1. The guidelines[6] are well used by provisionally certificated teachers and their mentors and managers.

Professional development opportunities

Schools and services that had participated in professional learning and development (PLD) offered by the previous New Zealand Teachers Council and now through the Education Council have found these meetings and workshops very useful in improving their system and processes for appraisal. Almost half of the institutions in the sample had some access to PLD delivered by the Council. We found that these institutions had either already developed or were developing appraisal systems that met compliance requirements as well as promoting high quality teaching and learning.

The Education Council website provides very useful resources to help professional leaders in developing effective appraisal. The Appraisal systems analysistool: Components diagram[7] below outlines all the components of effective appraisal and the Council’s What is Appraisal workshop materials include templates, annual summary examples, webinars and FAQs.

 

Appraisal systems analysis tool: Components diagram

The Council emphasises that “the wellbeing, learning and achievement of students lies at the heart of appraisal”. In the best examples, ERO found the whole process focused on valued outcomes for students, from goal-setting, evidence gathering and analysis, observations and professional discussions to the annual report. Teachers paid particular attention to priority learners who needed to make accelerated progress in their learning. In some schools, evidence from teachers’ inquiries into the progress of targeted students formed the basis of appraisal discussions. In early learning services, evidence was sometimes drawn from service internal reviews. These approaches are useful as they connect individual performance to professional learning and development for the school or early learning setting as a whole.

Which groups are not benefitting from effective appraisal?

Part time, fixed term or casual relief teachers in the sample were more likely not to have been appraised or to have inadequate evidence of meaningful appraisal. Where teachers in the sample had not been appraised, they were more likely to be classified as part of the ‘Other’ category.  ERO found that teachers in specialist positions, such as: guidance counsellors; careers advisers; Resource Teachers of Learning and Behaviour, Literacy and Māori (RTLB, RTLit, RTM); senior managers; principals and service managers were less likely to have been part of a consistently implemented appraisal process. Teachers and leaders in immersion settings reported that it was challenging to provide evidence of appraisal and of meeting the PTC when working in that context, both because of close whānau relationships and the oral nature of their interactions and communications.

The Education Council has responded to ERO’s findings by making changes to the Teacher Education Refresh (TER) programme for 2017 so that it provides more flexible options for teachers to remain certificated. Some groups, such as itinerant music teachers, have been discussing with the Council how to support their teachers to meet the requirements of the profession while continuing in their teaching roles.

Keeping effective records

There is some confusion over what records a school/service should retain of the evidence supporting the endorsement of practising certificates. Some schools consider that the discussions between the appraiser and appraisee should remain confidential to those parties. Other schools and services support a more transparent sharing of evidence, reflections and feedback. It was often the case that a school or service held no records of appraisal of a teacher they had endorsed in the previous 12 months but had subsequently left the school or was on leave.

Best practice is where the professional leader is able to access an annual appraisal summary in hard or electronic form that supports both the endorsement and the external audit process. The Council website has examples of such summaries. These personnel records should be retained in the school as official records for seven years.

When a teacher moves to another school or service, he/she should take with them sufficient records of appraisal for the previous three years so that the next professional leader is able to be confident that the teacher has been part of regular appraisal. If the professional leader is asked to endorse a teacher’s application early in their new position, this evidence can support the new leader’s assessment of the teacher’s competence.

Principal appraisal

An early finding from the audit was that the appraisal process for principals seeking to renew their practising certificate seldom included reference to the Practising Teacher Criteria (PTC). ERO reported that there was some degree of confusion among principals about what the requirements were for their appraisal.

In addition, principal appraisals are carried out by a range of people, including consultants, retired and current principals and trustees, some of whom were not well informed about current legislation and regulations. A New Zealand Principals’ Federation (NZPF) survey confirmed this lack of understanding by principals of the requirements for their appraisal.

As a result, the NZPF facilitated a series of meetings involving the Council, ERO, New Zealand School Trustees Association (NZSTA), Secondary Principals Association of New Zealand, New Zealand Educational Institute that resulted in clearer and more consistent communications across different websites.

An Appraisal for Principals professional development programme was also offered in 2016, involving principals and their board chairs. Principals reported finding this very beneficial. Other leaders reported attending useful PLD on appraisal delivered by other contracted agencies. Another outcome of these discussions has been collaboration between the NZSTA and the Council in the training of the NZSTA Endorsed Consultants. These consultants can assist and advise boards about principal performance review (appraisal) and principal appointment.

The Practising Teacher Criteria and appraisal

The most common reason for an appraisal not being considered satisfactory during the first year of the audits was that the process did not incorporate the Practising Teacher Criteria (previously known as the Registered Teacher Criteria). There was some confusion among teachers about what standards and criteria they were to use - PTC, professional standards and/or Tātaiako.

Both the PTC and Tātaiako prompt teachers and professional leaders to be thinking about the relationship between their teaching and outcomes for their students. The professional standards are part of the various Collective Agreements between the teacher unions and the Ministry of Education. The Council has produced templates that integrate these sets of standards, indicators and criteria. However, the main basis of appraisal should be the PTC since regular appraisal using these criteria are required for the certification process.

The Council has convened a sector working group during 2016 to undertake a review of Standards for the Teaching Profession and develop a new set of standards. The Council has also received feedback from across the profession indicating there is confusion between the current Practising Teacher Criteria and the various other standards that exist for different purposes. The group is currently working on new professional standards that will simplify the situation and help raise the status and quality of teaching in Aotearoa New Zealand. The Council is required to have the new standards in place by 1 July 2017.

A link between effective appraisal and effective school and service practice

ERO found a strong correlation between those institutions on an ERO longitudinal 1-2 year process or a category 1 or 2 return in the early learning services and the lack of an effective appraisal system. Sometimes this was as a result of leadership and ownership changes where current leaders did not know about the previous systems or were not aware of the requirements and expectations for effective performance management.

Notwithstanding this ERO found that poor leadership and teaching often resulted from ineffective appraisal practices and poor internal evaluation systems.  There was a poor understanding of the PTC and little awareness of the personal responsibility of leaders and teachers to gather evidence from their day-to-day practice and reflect on this information particularly with regard to the outcomes for the students in their school or class.  Professional leaders should seek advice from the Council and also inform the Council Relationship Manager where particular support is required.

On the other hand most appraisal systems were satisfactory and had strong links to the PTC. Most professional leaders, particularly in the school sector, are providing teachers with regular opportunities for professional learning and discussion. The focus of appraisal is on continual improvement in the quality of teaching and learning.

ERO found commonalities in the very best examples of practice.  These included:

  • appraisal practices that formed an integrated part of an ongoing cycle of reflection and feedback
  • the collection and analysis of appraisal evidence from a range of perspectives (students, peers, assessment information, surveys)
  • leaders and teachers taking personal responsibility to curate evidence from their everyday practice in order to demonstrate how they were continuing to meet the PTC and other standards
  • placing an unrelenting focus on the impact of teaching on improving student outcomes, particularly those at risk of not achieving. 

Conclusion

The Audit Services Agreement between the Council and ERO states that:

As a result of the information provided by ERO, the Education Council will be better able to fulfil its function to improve performance management by professional leaders to ensure high quality teaching and leadership and lift the status of the profession.

The first year of ERO’s audit and moderation work is helping professional leaders in schools and early childhood services to understand and meet the expectations for the appraisal and certification of teachers.

Monthly reports and regular communication with the Council have enabled them to respond to issues as they arose in the audits and provide responsive PLD. Collaboration among the sector agencies and organisations has led to positive outcomes for the sector in terms of clarity of information and alignment of messages across agencies.

Next Steps

ERO recommends that:

  1. The Council continues to rationalise and simplify the expectations for teachers’ professional practice.
  2. Information about appraisal of teachers and leaders is aligned and consistent across the education sector, education agencies and organisations.
  3. Consideration is given to extending provision of professional learning opportunities in appraisal workshops as these have been deemed highly practical and valuable by those who have already attended them.
  4. School leaders continue to develop effective appraisal systems and processes that focus on the learner and promote continuous improvement in the quality of teaching.
  5. Teachers take professional responsibility for curating evidence from their everyday practice to show that they continue to meet the Practising Teacher Criteria. 

 

"Appraisal and feedback have a strong positive influence on teachers and their work. Teachers report that it increases their job satisfaction and, to some degree, their job security, and it significantly increases their development as teachers."   OECD TALIS Report.

 



[1] s382(1)(i),Part 32 of the Education Act 1989

[2] Education Review Office. (2014). Supporting school improvement through effective principal appraisal. Wellington, NZ: Author.

Education Review Office. (2014).Supporting school improvement through effective teacher appraisal. Wellington, NZ: Author.

Education Review Office. (2014). Student safety in schools: Recruiting and managing staff. Wellington, NZ: Author.

Education Review Office. (2014). Improving Quality: Employment responsibilities in kindergartens and education and care services. Wellington, NZ: Author.

Education Review Office. (2014). Board employment responsibilities: Linking charter targets to appraisal in primary schools. Wellington, NZ: Author.

[3] Nusche, D., et al (2012). OECD Reviews of Evaluation and Assessment in Education: New Zealand 2011, OECD Publishing. http://dx.doi.org/10.1787/9789264116917-en

[4] Pont B., D Nusche and H Moorman (2008). Improving school leadership, V1: Policy and Practice, OECD, Paris.

[5] Appraisal is also used to guide salary progression, and inform recommendations for full registration and for renewing teachers’ practising certificates.  See Education Review Office. (2014). Student safety in schools: Recruiting and managing staff. Wellington, NZ: Author.

[6] Guidelines for Induction and Mentoring and Mentor Teachers 2011
Te Hāpai Ō - Induction and Mentoring in Māori-medium Settings 2012

[7] Education Council Appraisal systems analysis tool: components diagram retrieved from https://educationcouncil.org.nz/sites/default/files/Appraisal Systems Components_0.pdf