Findings

ERO investigated how well self review was understood and implemented in 397 early childhood services in Terms 1, 2 and 3, 2008.

In this section ERO’s findings are discussed in relation to:

  • how well self review was understood and implemented in early childhood services;
  • the features of self review, where it was well understood and implemented;
  • factors that contributed to self review being well understood and implemented;
  • the outcomes of self review;
  • the challenges for services in understanding and implementing self review;
  • the progress made by some services between 2005 and 2008; and
  • improving self review by building on understanding and practice.

How well was self review understood and implemented?

As shown in Figure 1, in just over half of the services self review was understood and implemented, but there was variation in how well this happened.

Figure 1: Understanding and implementing self review in early childhood services

This is a bar graph called Understanding an implementing self review in early childhood services. The y axis is called Percent of services and is ranged from 0-100 at intervals of 20. The x axis has four labels which are Well understood and implemented 14%, Some understanding and implemenation 42%, Limited understanding and implementation 36% and No understanding and implementation 8%.

In services where self review was well implemented, improvement and accountability were understood to be the main purposes of self review. If management and educators had a similar understanding about self review, they were more likely to implement review that fulfilled both purposes. Leadership support and direction for self review had a positive influence on understanding and implementation. Many of these services had documented the outcomes of review, particularly those related to children. Self review was embedded in practice and integral to their operation.

Purposeful and inclusive self review were two other important factors. The complementary relationship between self review and external review was well understood in only a few services. In these services, management and educators considered the areas for improvement and recommendations identified in their previous ERO report, and used the findings as a basis for their self review.

In services where self review was not well understood or implemented, ERO highlighted the need for managers and educators to gain a mutual understanding about the importance of self review and why it was undertaken. In some of these services managers and educators were negative about self review and did not recognise its value. Policy review was largely a paper exercise, and educators did not have time to meet to plan and discuss reviews.

Many of these services needed to develop a framework that linked self review to their long and short term planning and set out expectations for how to undertake it. ERO also identified the need for further professional development to help increase managers’ and educators’ understanding and assist the service to set up systems to support ongoing self review.

In some services, a lack of self review indicated more serious concerns, which were identified and reported by ERO. In half the services where self review was lacking, self review was just one area needing improvement, with ERO also reporting concerns about the quality of education and the services’ management. In these services, ERO indicated that it would undertake a further review in 12 months. ERO identified concerns in these services relating to one or more of the following:

  • the learning programme;
  • health and safety practices;
  • personnel management;
  • meeting qualification requirements; and
  • governance and management issues.

The overall findings of this evaluation show that there is considerable variation in how well services understand and implement self review. An analysis of the findings showed no significant statistical difference between services based on their type or location [6]in terms of how well self review was understood and implemented.

What did self review look like where it was well understood and implemented?

In the 14 percent of early childhood services where self review was well understood and implemented, some common features of practice and contributing factors set these services apart from others. These are discussed and supported by examples of good practice from the education review reports of individual services.

What did these services do?

Self review was integral to the operation of the service and focused strongly on improvement. Well-established procedures guided self review, and reviews were both planned and spontaneous. Planned review included scheduled policy review and more in-depth reviews of targeted areas of practice. Review included teaching practice, and led to ongoing improvement to the quality of the programme for children. Spontaneous reviews were more informal and responded to what was happening on a day-to-day basis.

Examples of good self-review practice

The licensee and staff developed a formal and systematic framework to guide self review. They had a regular cycle for review so that all aspects of operation were considered over time. This included planning and recording the discussion, processes and action taken. They also documented the impact of any changes.

The licensee and staff had a good understanding of the purpose of self review as a tool for ongoing improvement, aligned with centre philosophy. They also used self review to monitor accountability and compliance. Reviews of the centre policies and practices and the implementation of the Revised Statement of Desirable Objectives and Practices 1996 (DOPs) were thorough and systematic. Reviews of routines and the programme were often spontaneous and responsive to emerging issues. Changes, as a result of informed decision-making, contributed to enhanced learning experiences for children and an improved programme.

Self review involved the gathering, analysis and use of information relevant to the review focus. Data was gathered from a range of sources and included the perspectives of managers, teachers, parents and whanau, and children. Staff had time to discuss and analyse this information. A few services used existing indicators of good practice, or developed their own to use as a framework for analysing data

Examples of good practice

Teachers, children and families all played a part in self review and all contributions were valued and considered. Reviews could be prompted by adults or children and were clear in focus. Teachers were able to articulate positive changes to teaching and learning that had been the result of review. Some reviews were formally planned while others occurred spontaneously. Value was placed on both of these processes.

Teachers used evidence-based processes for reviewing centre operations in order to improve outcomes for children. They gathered data informally through discussions, surveys, and observations. Analysis of information led to effective decision-making and the identification of well-defined indicators of high quality practice. Opportunities to revisit changes to practice were documented.

Review findings informed decisions about changes to practice and were also used to develop long and short term plans. Such plans served as a useful basis for self review, enabling services to monitor progress towards goals and track improvement over time.A few services used their previous ERO report for ongoing self review. Managers and educators in these services were developing an understanding of the complementary relationship between external review and self review.

Example of good practice

The new teaching team used the findings of the last ERO review to guide change to their practice. They engaged in this review in a way that demonstrated how external and internal review could complement each other. The review team was able to use the kindergarten's own self-review findings to set the direction of the evaluation. Both teachers were extremely reflective and willing to consider alternative perspectives while still articulating and explaining their current practices. Both had a deep commitment to professional learning.

In services that were doing well with self review, management and educators demonstrated a commitment to ongoing improvement and to increasing their capacity to engage in manageable and meaningful self review, as demonstrated in the following example.

Documentation for formal self review had been strengthened since the 2005 ERO review. The approach promoted by Ngā Arohaehae Whai Hua: Self-review Guidelines for Early Childhood Education was used to guide and record the process. This included planning for the review, deciding how information was to be gathered and analysed, and evaluating the impact of the resulting changes. Teachers planned to use the ERO Self-reporting document as a guide to their own future recording of informal review so that overall progress could be readily identified. The stability in both the management body and teaching team assisted in developing a shared understanding. It has contributed to building their capacity for self review.

What were the contributing factors?

ERO identified factors common to early childhood services where self review was well understood and implemented. These included:

  • strong leadership in promoting self review;
  • professional development to support self review;
  • stable and collaborative staff; and
  • sound systems for self review, and the use of relevant resources.

Each of these factors is discussed and supported by examples of good practice.

Leadership

In many services where self review was well understood and implemented, leaders made sure self review happened and played a key role in encouraging and supporting staff. They facilitated regular meetings to plan reviews, analyse data and discuss review findings. Staff were expected to work as a team and leaders often took on a mentoring role, particularly with new or less experienced staff. Leaders played an important role in establishing a collegial and reflective culture in the service. ERO’s findings highlighted the importance of early childhood services having well-qualified and experienced staff who understood the importance of self review.

Examples of leadership promoting self review

The centre manager demonstrated a good understanding of the purposes and benefits of self review. She established a sound framework for review and the teaching team was focused on improving the service. The centre manager and teaching team all understood that self review should improve the quality of the programme and teaching practices. The manager/licensee recently attended a course on self review and gained a good understanding which she has discussed with teachers. She recorded three self-review initiatives, one of which is still ongoing. As a result of her leadership, the centre has made impressive progress since the last review.

Professional development

Many services were proactive in seeking professional development to improve self-review practice. Staff in these services had a positive attitude and managers supported professional learning and development for all staff. This helped to build educators’ understanding about self review and gave them opportunities to put new learning into practice. ERO’s findings highlighted the value of managers and educators in early childhood services having access to expertise, in-house or external, to support self review.

Examples of professional development supporting self review

Professional development and guidance from the professional services manager assisted teachers in working towards and further strengthening their understanding about the purpose and process of self review. There were very clear expectations and guidelines for self review. It was planned and spontaneous, and linked to the annual plan.

As a result of professional development with an external facilitator in 2007, documentation of self review was strengthened. Self review covered both formal and spontaneous review and was improvement focused. The process was aligned to that promoted in Ngâ Arohaehae Whai Hua: Self-review Guidelines for Early Childhood Education. The senior teacher’s understanding of the importance of self review and its documentation increased as a result of ongoing support from an external facilitator.

Staffing

In many services where self review was well understood and implemented, staffing was stable, enabling self-review practices to be trialled and embedded as part of everyday operations. Many services had induction processes that introduced new staff members to self review. Staff were ‘like-minded’ in their approach to self review and had a sense of ownership of both the process and the outcomes. They worked in an environment that encouraged critical reflection on practice. ERO’s findings highlighted the importance of managers and educators having a common understanding of self review’s purpose and value, and of how to implement it.

Example of stable and collaborative staffing supporting self review

Management and many of the current staff had worked together for some time and had in place a well-developed and documented process for self review. Self review was a natural part of their practice as professionals and over time has covered a broad range of teaching and learning matters. Open and ongoing discussion in formal and informal settings provided opportunities for staff to continue to reflect on their teaching practice.

Systems and resources

Many services had documented the purpose of self review and had set out the processes to be followed. Some services had developed a framework to guide planned reviews that were aligned to the documented strategic direction in their long and short term plans. Where services operated under an umbrella organisation, the organisation often determined the expectations for self review and developed frameworks and templates or tools to guide self review in individual services. ERO’s findings highlighted the usefulness of services having established procedures to guide self review.

In services where self review was well understood and implemented, it was systematic in ensuring no aspects of practice were overlooked. Reviews were well planned and services documented them in ways that were manageable and showed both the process and the outcomes. Some services were trialling templates for planning reviews and documenting the findings. Self review was flexible enough to enable managers and/or educators to respond to emerging issues or events.

ERO’s findings also highlighted the value of staff having access to a wide range of resources to support them in self review. Resources generally took the form of published guidelines such as the Ministry of Education’s Nga Arohaehae Whai Hua: Self-review Guidelines for Early Childhood Education (2006) and internally developed frameworks or guidance documents and templates.

Examples of sound systems and useful resources supporting self review

Teachers were focused on improvement, particularly the head teacher who was eager to ensure that they provided high quality education. She understood that review was the way to achieve some of their aspirations for children’s learning. A schedule, developed annually, allowed for planned reviews, some of which were in-depth, and a curriculum area was reviewed each term. Provision was also made to undertake reviews that were more responsive to issues and questions as they arose.

Planned self review was soundly based on the kindergarten management plan. The teaching team were reflective practitioners and had a good understanding of the importance of internal and external self review as tools to support ongoing improvement. The team participated in focused professional development using Ngâ Arohaehae Whai Hua: Self-review Guidelines for Early Childhood Education with an external facilitator. This raised their awareness of the need to document all stages of the review cycle.

Teachers gave consideration to the tools provided by the association, together with ideas from The Quality Journey and the new Ministry of Education guidelines, Ngā Arohaehae Whai Hua: Self-review Guidelines for Early Childhood Education. As a result, they have developed a good framework based on gathering, analysing and using information to bring about improvement.

Teachers found the Ministry of Education support guidelines on self review a very useful guide to help them develop systems and questions to use when evaluating practices and procedures. Reflective and evaluative thinking has become an embedded practice in the centre.

Strong leadership promoting self review, professional development that supported self review, stable and collaborative staff, sound systems for self review, and the use of relevant resources all influenced the extent to which self review was understood and implemented in the early childhood services in this study. Together they contributed to a culture of improving the awareness, understanding (capacity) and skills (capability) of managers and educators for implementing self review.

What were the outcomes of self review?

In services where self review was well understood and implemented, outcomes for children were documented, evident in practice and visible in assessment information about children’s learning. Services monitored the impact of changes on practice through self review, with a particular focus on what was happening for children.

Examples of self review having a positive impact on children’s learning

A successful review of the centre philosophy became a positive process for all staff, who shared in this new development. The review resulted in positive outcomes for children through changes to the daily timetable and mealtimes. Children now have more opportunities to participate in sustained play and investigation before choosing to eat. Meal times are social times during which children and adults interact calmly.

Teachers have carried out in-depth reviews that have resulted in improvements to benefit children’s learning. Changes to the environment as a result of review have added interest and challenge for children and the review process has encouraged staff to reflect on opportunities for children’s learning. A well-established process of self review that leads to positive outcomes for children is evident.

Self review was undertaken to evaluate the success of reciprocal visits with the neighbouring school. Teachers used a consultative approach, involving the new-entrant teacher and students in reflective discussion to better inform practice. Changes were made based on analysis of this information. Teachers identified positive outcomes for children’s learning and identified that good relationships were affirmed and extended as a result of this initiative.

What did services tell ERO about what was working well?

At the beginning of each education review, ERO invited services to provide information about their most recent self review, the process they used, whom they involved, and any resultant outcomes. Services were asked to comment on what was working well, what could be improved and what needed to be developed. In services where self review was well understood and implemented, common themes about what was working well included having:

  • the whole team working together on self review;
  • a sense of ownership of the process and outcomes of self review; and
  • greater awareness of improvements to the programme and to the operation of the service.

The following examples are indicative of these themes, and have been taken from the self-reporting information provided by managers and/or educators.

The whole team has input, rather than one or two people. Parents’ views are welcomed. We found new strengths within staff members as we have an approach where no ideas are wrong until we try them. We trial the changes first to see where we could change if need be. The centre is running a lot better with the staff and children all benefiting.

We have processes for gathering data and engage in reflective discussions that result from this data. Teachers are more empowered as we are now working in line with our beliefs and with what we know to be quality practice. Review that is happening has resulted from an occurrence and is really meaningful.

New formats from the association give our teaching team focus, and highlight areas for us to review. We incorporate all ideas from the senior teacher, teachers, children, families and whânau. It’s great because we see the positive results of our review in action. The self-review process is working extremely well in our team and the results of each individual review are now evident in the team, as they work with children, families, committee and the wider community. Although we have completed four reviews in the last year, they all intertwine. This is wise practice. As a team we find the process exciting but challenging. We enjoy the reflective discussions that come about because of the self-review process.

We have already completed our first self review as a centre and have started our second. Through this we have discovered we have a very close team that work together well through all areas of the process. The staff have taken new ideas on board well and use these in their day-to-day teaching with the children. The team members enjoy reflecting on their practice and join together well to plan and implement changes. Our plans for self review are based on staff discussions, meetings, questionnaires and parent contributions and feedback.

Our team is committed to improvement and centre development to provide high quality outcomes for children and their families. A self-review plan (wall planner) and document (book) are good guides to the process. An external facilitator provides 'hands on' support. This is very valuable.

Comments from services` self-reporting noted that as a result of self review children were:

  • contributing to and directing their learning;
  • setting their own goals;
  • assessing their own learning;
  • taking an interest in the learning of others; and
  • feeling that their ideas are valued.

Other outcomes reported by services included:

  • a safer environment for children;
  • improved relationships between teachers and whanau;
  • parents more involved in the service;
  • better access to equipment;
  • teachers getting more professional development;
  • a reduction in noise levels in the centre;
  • programme planning based on session evaluations;
  • more streamlined systems for managing children’s portfolios;
  • a more confident and competent supervision team; and
  • increased consistency of teaching practice.

When self review was well understood and implemented, it was integral to the operation of the service and undertaken in inclusive and purposeful ways. The outcomes of self review resulted in changes to practice and had visible benefits for children’s learning and development.

What were the challenges for servies?

All services faced challenges as managers and educators strived to gain an understanding of self review and to implement it in ways that improved the quality of education provided for children.

In many services, leadership influenced the extent to which self review was seen as a worthwhile activity, and on whether or not it was given priority in terms of professional development and time allocated to implement it well. This was particularly so for some new services. About 10 percent of services in this study were newly licensed and this was their first ERO review. Some of these services had more pressing priorities associated with getting established.

Other challenges related to:

  • making explicit the links between self review and children’s learning;
  • linking self review with systems for planning and reporting;
  • valuing self review as a process to critique as well as affirm practice;
  • including a wider range of perspectives in reviews;
  • using the findings of external review to guide the focus of some self review; and
  • documenting the process and outcomes of self review in manageable ways.

For many of the services with either limited or no self review, the starting point needed to be improving understanding of self review, and why and how to do it. Self review was not always well implemented because managers and educators in leadership roles did not have:

  • a good understanding of self review;
  • a professional commitment to supporting self review;
  • an awareness of the value and purpose of self review; or
  • the knowledge and skills to develop and implement manageable systems to guide self review.

In some cases limited self review was attributed to leaders’ inexperience and to their lacking an early childhood education background and/or qualification. ERO found that in a few services, managers and supervisors or head teachers were isolated and working without professional support, which made it difficult for them to develop and implement manageable self-review systems.

In services where self review was in the early stages of development, challenges were associated with widening the focus of review beyond compliance audits such as health and safety checks. Some services were attempting self review without having developed a systematic approach that included procedures to guide practice. In others, there had been changes in management or ownership that meant that systems lapsed or other priorities dominated.

Many services struggled to manage the impact that staff turnover had on embedding self review as an integral part of the operation. Services also found it difficult to bring new staff on board with the expectations and practices for self review. Staffing issues often resulted in poor team work, which led to teams not developing a common understanding of self review.

What progress were services making with self review?

A national evaluation report by ERO, Self-review in Early Childhood Education Services, December 2005, found that most services were developing an understanding of the purpose of self review and of how to integrate it into their practice. Where it was carried out effectively, self review processes focused on the education programme, the environment and compliance with legal requirements. This 2005 report highlighted the need to increase staff awareness of self review, to involve staff more, to share good practice and to monitor the effectiveness of self review. ERO concluded that services needed ongoing support through high quality resources and continued provision of professional development.

ERO analysed the 2005 reports of 107 services that included a focus on self review, and compared the findings with their 2008 education review reports. ERO also considered what these services reported on themselves prior to their education review. Factors contributing to or hindering progress were identified.

Fifty-two percent of services showed progress in some aspects of their self review. Aspects of self-review practice where these services made the most progress included:

  • understanding the purpose of self review;
  • reviewing charters and philosophy statements;
  • aligning long and short term planning and self review; and
  • developing a framework or cycle for self review.

Fewer of these services made progress in terms of involving parents and whanau in self review, documenting reviews, reviewing teaching and learning practices, and monitoring the impact of changes following a review.

Just under a quarter of the services were still at an early stage of developing their self review. Four percent had not moved forward, with evidence of self-review practices and systems lapsing. Aspects of self-review practice where these services had made the least progress included:

  • developing a systematic approach to self review;
  • using indicators to evaluate teaching and learning programmes and practices;
  • documenting self-review processes, actions, and their impact;
  • focusing on outcomes for children; and
  • aligning self review with strategic planning.

What contributed to progress?

The services (52 percent) that had made good progress with self review were on the continuum of development. One third of these services demonstrated good understanding of self review and its implementation. The current practice of the remainder was not high quality, but they had made progress in various aspects of self review. Key factors for the services that had improved their self review included:

  • a team approach, with staff and managers jointly involved in professional discussions and decision-making;
  • a framework to support formal, cyclical and spontaneous review;
  • informed leadership; and
  • parent and whanau involvement in self review.

What hindered progress?

The services that had made little or no progress between 2005 and 2008 were also at different stages on a continuum of development. A lack of understanding about self review was a common theme in both the 2005 and 2008 reports for these services. Two other factors were the lack of a framework for systematic review, and a lack of staff involvement and participation in self review. As a result, self review was typically spasmodic and ad hoc, with no links to strategic planning. ERO frequently recommended external advisory support or professional development to help managers and/or educators build a mutual understanding of self review.

Some services had changes of leaders, managers and/or educators, and this lack of continuity was perceived by the services, or reported by ERO, as a barrier to progress. However, two services that had made good progress saw such changes as an opportunity rather than a barrier. One service referred to new strengths in the team, and another said that the philosophy underpinning practice had given continuity and collaboration in improving outcomes for children. In services where staff turnover was a barrier to progress, there were other contributing issues. Several services had been preoccupied with matters such as building projects or moving to different session schedules, and they attributed their lack of progress with self review to such factors.

What did services need to do to improve self review?

Half the services’ ERO reports included recommendations about improving self review. Almost a quarter of these affirmed the progress services were making and indicated the need to continue what they were doing. ERO found that managers and educators in these services had sound understanding of self review, had established manageable systems and practices, and were moving in a positive direction. In a few services where self review was quite well understood and implemented, ERO recommended that they consolidate, embed, enhance or extend existing practices.

A third of the recommendations related to improving existing self-review practices. These included: better use of Ministry resources; more systematic documentation of self-review processes and findings; developing an understanding of self review among managers and educators; monitoring outcomes; and ensuring that self review is manageable.

About a fifth of recommendations focused on services seeking external advice and professional development to help them develop their self review. A further quarter of the recommendations were about services developing a framework to guide self review. Services needed to establish a basic understanding and formalise practices such as using the DOPs, aligning self review with their strategic plans, allocating times and resources for self review, and developing a cycle or schedule for regular review.

What did services tell ERO?

Key areas for further development identified by services through self-reporting included: documenting self review; extending understanding; widening who gets involved in self review; improving analysis of information collected and the quality of evidence; and aspects associated with leadership.