Framework for Kōhanga Reo reviews

Summary

This framework explains our process for reviewing kōhanga reo. It explains how we work with kōhanga reo throughout the review. The framework includes resources for kōhanga reo, ERO and Te Kōhanga Reo National Trust.

 We worked with Te Kōhanga Reo National Trust to create this specialised review process. This process supports the philosophy of kōhanga reo.

 

The framework supports kōhanga reo

We designed this framework to make our reviews more useful to kōhanga reo. In a review, we identify:

• what’s going well 
• what needs improving
• what should happen next.

We also encourage kōhanga reo to improve self-review processes.

 

Our review framework has four strands

1. Kōhanga reo priorities — We review how whānau create good outcomes for tamariki.
2. Planning and evaluation — We focus on helping kōhanga reo develop good processes for self review.
3. Areas of national interest — We observe how well specific policies are working in kōhanga reo. We share this information with the Government.
4. Compliance — We check whānau have taken all reasonable steps to meet legal requirements. We include checklists for kōhanga reo to fill out.

 

More detail about kōhanga reo reviews

This framework includes the evaluation indicators we review against. These indicators are explained in a separate resource.

Read the Evaluation Indicators for Education Reviews in Kōhanga Reo

Read our review process for Kōhanga Reo

 

1 Introduction

The purpose of this framework

This framework contains information about the process that the Education Review Office (ERO) uses for managing kōhanga reo reviews. This information forms part of ERO’s standard procedures for reviews of kōhanga reo.

The framework contains process guidelines and resources intended for use by ERO, Te Kōhanga Reo National Trust and kōhanga reo. The following table shows the purpose of process guidelines and resources.

Content	Description
Process Guidelines	These contain information about ERO’s approach
Resources	These can be used by: ERO in planning and scoping reviews; and kōhanga reo in their processes of self review

The process guidelines indicate the stages at which the different resources could be used by ERO, the National Trust and kōhanga reo.

ERO has adopted this approach to the provision of guidelines and resources so that its procedures are transparent to kōhanga reo, and the same information is available for self review as it uses for external review.

 

ERO’s approach to kōhanga reo reviews

ERO is an independent external evaluator. Its review process is based on a Manual of Standard Procedures and Code of Ethical Conduct for Review Officers.

ERO takes an evidence-based approach to reviews. Reviewers make independent judgements based on evidence and the use of evaluation indicators.

ERO’s approach to kōhanga reo reviews focuses on how whānau management:

• contributes to the learning, development, safety and wellbeing of tamariki;
• contributes to the learning and development of whānau; and
• incorporates the philosophies of kōhanga reo.

The review approach incorporates principles used in ERO’s evaluation of schools, early childhood services and kura kaupapa Māori that operate in accordance with Te Aho Matua.

 

Background

The kōhanga reo review process was developed in consultation with Te Kōhanga Reo National Trust. It incorporates:

• aspects of the kura kaupapa Māori review approach – to reflect the Māori context; and
• ERO’s ‘assess and assist’ approach – focusing on accountability and improvement.

The contributing approaches are outlined below.

 

Te Aho Matua review approach

The Education Review Office has developed a review approach for kura kaupapa Māori that operates in accordance with Te Aho Matua that considers Māori protocols and the guiding principles of Te Aho Matua.

This approach incorporates self review and external review, with dialogue between the kura and the external review agency underpinning the process.

Marae protocols of encounter between manuhiri (the review team) and tangata whenua (kura) are integrated into the external review process, as a cultural means of protecting the integrity of the whānau of the kura and the reviewers.

In addition, the following key principles underpin this review process:

• dialogue is an important part of the process;
• evaluation indicators are based on the principles of Te Aho Matua;
• there is an understanding of the nature of Māori language; and
• there is an understanding that the curriculum development process takes place within the context of kura kaupapa Māori efforts to re-establish mātauranga Māori and tikanga Māori.

 

The ‘assess and assist’ approach

ERO’s revised approach to reviews is sometimes characterised as an ‘assess and assist’ approach. The term ‘assess and assist’ reflects the balance between ERO’s twin purposes of accountability and improvement, but in some ways it is misleading because it implies that ‘assess’ and ‘assist’ are two separate activities that take place sequentially.

ERO’s view is that the best way it can assist kōhanga reo is through its reviews. ERO intends to carry out reviews that are focused on whānau priorities and concerns, give the best possible information and identify useful strategies for improvement in its recommendations.

In ERO’s view, ‘assess and assist’ is a single process that is a legitimate part of the role of an external evaluator. Activities that fall outside the scope of evaluation, such as providing ongoing advice and support, are not part of ERO’s role.

In practical terms this means that ERO may be involved in developing recommendations and formulating action plans, but it is not ERO’s role to be involved in the implementation of those plans.

 

Key features of ERO’s approach

This framework has been developed to support reviews of kōhanga reo. It reflects key principles that underpin all ERO reviews.

The key features of ERO’s approach to reviews relate to:

• ERO’s role and purpose;
• what ERO evaluates; and
• how ERO evaluates.

The role and purpose include ERO’s:

• status as an independent, external evaluator;
• dual purpose in helping to bring about improvements in the quality of education provided for tamariki, while ensuring accountability; and
• role in providing information to whānau, communities and the Government to inform their decision making.

What ERO evaluates reflects:

• learning and development outcomes for tamariki; and
• the way in which ERO investigates how programmes and processes within kōhanga reo contribute to outcomes for tamariki.

How ERO evaluates includes:

• the basis on which ERO reviews are conducted – using a Manual of Standard Procedures and a Code of Ethical Conduct for Review Officers;
• the evidence-based approach to evaluations;
• the use of evaluation indicators to inform judgements;
• the participatory approach to education reviews, in which ERO discusses the priorities for review and develops recommendations with key stakeholders; and

ERO’s emphasis on developing links between external evaluation and self review.

Some of these features, for example ERO’s independence and use of evidence in evaluations, have always been central to the way in which ERO operates.  Other features include components of the review process for kura kaupapa Māori that operate in accordance with Te Aho Matua and links to self review that have been developed as a part of the new approach to reviews.

 

2 The kōhanga reo context

The kōhanga reo movement is a unique initiative based on total immersion in Māori language and values with the aim of passing on Māori culture to future generations.

Kōhanga reo emphasise the revitalisation of the culture through te reo Māori and are dependent on the active participation of the whole whānau. The kaupapa of kōhanga reo aims to strengthen whānau capabilities in cultural, social, economic, spiritual and political matters. Education, health and wellbeing are inherent within all aspects of the kaupapa. The prime focus is the whānau – its collective development, growth, accountability and wellbeing set within a Māori cultural context.

Kōhanga reo do not regard tamariki in isolation, but as important members of the whānau. The kōhanga reo philosophy is that tamariki will benefit when whānau successfully operate according to the kaupapa. Tamariki are an integral part of whānau development.

There are two main layers of administration and management of kōhanga reo: one at the national level and the other at the level of the individual kōhanga reo. At the national level, Te Kohanga Reo National Trust is an incorporated society to which individual kōhanga reo are affiliated. At the level of the individual kōhanga reo, each whānau is responsible for the management and administration of its kōhanga reo. The whānau may consist of the parents of tamariki attending, elders and the community involved in the kōhanga reo.

Individual kōhanga reo also reflect the kaupapa and the aspirations of their local hapū and iwi, which means that kōhanga reo may differ significantly in some aspects of their operation. Whānau members are likely to be influenced by these considerations when choosing to enrol their tamariki in a kōhanga reo. For this reason, this framework is enabling not prescriptive. The key features include participation, rigour, transparency, and an improvement focus that are common to all ERO reviews.

 

A flexible approach

ERO’s aim is for its reviews of kōhanga reo to reflect:

• consideration for individual whānau aims and aspirations of their local hapū and iwi; and
• variations in the performance of kōhanga reo.

Kōhanga reo whānau are given the opportunity to have Te Kōhanga Reo National Trust involvement during the course of a review through a kaupapa kaimahi, a Trust-appointed whānau support person (see Resource D). However, the whānau may choose an alternative representative or support person.

 

Building on self review

One assumption in this framework is that ERO’s procedures and indicators are transparent and are made available to kōhanga reo to assist in their own processes of self review, in self audit and in planning for external review.

Self review is a process through which kōhanga reo evaluate the effectiveness of what they do, with the aim of improving the quality of their practice.

Self audit is a process of checking compliance with legal requirements.

ERO’s aim is to encourage and support the development of self review in kōhanga reo through raising awareness of review processes and incorporating whānau self-review information in its own reviews.

At the same time, ERO’s processes need to be sufficiently flexible and robust not to depend on self-review information, because in some instances no information relevant to the review priorities is available.

The Government’s strategic plan for the early childhood education sector contains the following objectives:

• ERO evaluations will stimulate services to take an improvement approach, with emphasis on the services' own management goals and review processes;
• early childhood services will carry out self review; and
• external reviews by ERO will check self-review processes and the use made of them.

ERO’s strategic plan also notes that effective implementation of services’ self review and good external review will require development of indicators of quality practice.

In ERO’s experience, the quality of self review in kōhanga reo is variable. Where kōhanga reo are carrying out high quality self review, the results will be used to inform ERO evaluations. In many kōhanga reo, however, self review is not highly developed. In such cases ERO reviews will aim, through the external evaluation process, to support the development of self review within kōhanga reo.

ERO has developed its own evaluation indicators for kōhanga reo reviews, and makes these available to kōhanga reo along with guidelines on how to use them. The aim is to enhance kōhanga reo understanding of review processes and assist services to prepare for ERO reviews.

 

3 The dual role of ERO

The approach outlined above involves ERO and kōhanga reo working together to identify strengths and weaknesses and to develop proposals for improvement.

Kōhanga reo receive government funding and operate in a policy and regulatory environment established by the Government. ERO is a government department with a responsibility for contributing to educational improvement through evaluating the quality of education and the effective use of public funds. The audience for ERO reports includes the Government and the public as well as those in kōhanga reo.

External evaluation in kōhanga reo has two main purposes – accountability and educational improvement. Evaluation for accountability purposes involves reporting on goals and standards (including checking on compliance matters) while an improvement focus involves assisting kōhanga reo to develop themselves through feedback.

There are tensions between ERO’s improvement and accountability functions, and also between the information needs of different audiences for ERO reports. This framework is intended to provide ERO reviewers and kōhanga reo with resources that focus on improvement, while at the same time clarifying ERO’s accountability purpose.

 

4 The review strands

ERO’s framework for reviewing and reporting is designed to make the process more useful to kōhanga reo by identifying:

• what they are doing well;
• where they need to improve; and
• what they should do next.

The framework is based on four review strands, A, B, C, and D as shown below.

A. Kōhanga reo priorities:

• Areas of good performance
• Areas where the kōhanga reo needs to improve
• Recommendations

B. Planning and evaluation

• Areas of good performance
• Areas where the kōhanga reo needs to improve
• Recommendations

C. Areas of national interest

• Areas of good performance
• Areas where the kōhanga reo needs to improve
• Recommendations

D. Compliance issues

• Areas of good performance
• Areas where the kōhanga reo needs to improve
• Recommendations

 

A Kōhanga reo priorities

This strand focuses on how whānau management and programme implementation influence positive outcomes for tamariki through the use of components of Te Whāriki (the New Zealand early childhood curriculum).

The whānau may select strands from Te Whāriki for review. This supports the review and reporting of kōhanga reo philosophy. The review processes and indicators in each strand will cover both the quality of whānau management and the quality of education.

Te Whāriki strands, and the mana, are interpreted within the context of each kōhanga reo.

Te Kōhanga reo review overview

 

B Planning and evaluation

All ERO kōhanga reo reviews will give priority to the quality of planning and evaluation. This area is most likely to support long-term improvement.

In many kōhanga reo, self review is not highly developed. ERO reviews will aim, through the external evaluation process, to support the development of self review.

This strand, like Strand A, deals with both the quality of whānau management and the quality of education (see diagram below).

Consideration is also given to programme planning, evaluation and assessment. This approach is to improve the quality of programme delivery focused on specific learning outcomes.

Te Kōhanga Reo Review Approach

 

C Areas of specific Government interest

This strand provides information to the Government about how well specific policies are working in kōhanga reo. The topics covered may change from year to year to reflect the Government’s priorities. This section will only be covered in kōhanga reo reviews where the topics are applicable to kōhanga reo.

 

D Compliance issues

Legal compliance is an important responsibility of kōhanga reo. ERO’s approach to compliance is designed to support whānau management of compliance functions and places some reliance on whānau reporting on compliance.

ERO’s evidence shows a wide variation in kōhanga reo compliance with legislation. Kōhanga reo operate under a comprehensive set of regulations (reflecting the importance of safety and wellbeing of tamariki in kōhanga reo) and there is a public expectation that ERO will continue to check on compliance with these regulations.

ERO needs to have the flexibility to vary its approach to reflect the circumstances of an individual kōhanga reo and the extent to which evidence indicates it is complying with legislative requirements.

ERO has developed the Kōhanga Reo Whānau Management Assurance Statement and Self Audit Checklists (KWAS) for kōhanga reo to use to provide assurance that the whānau has taken all reasonable steps to meet its legal requirements. When scoping and planning a review, ERO will use the service’s identification of non-compliance and actions to be taken, as shown in the KWAS.

During the course of all reviews, ERO checks performance with respect to compliance as attested in the KWAS. In the event that the checking process indicates any significant problem areas, ERO may decide to investigate further.

Compliance will not be a major focus of reviews unless it appears that there are significant levels of risk.

For many kōhanga reo, ERO’s main interest in this strand is the extent to which the service is auditing its own compliance. Any non-compliance not identified by the whānau and discovered by ERO may be reported in the ERO report and, if significant, may result in ERO changing the focus of the review. This might then lead to a supplementary review. ERO will decide whether or not to report non-compliance issues identified by the kōhanga reo.

 

Balance between the strands

The length of time devoted to each of the four strands (outlined above), and the detailed processes for gathering information and reporting findings, varies among kōhanga reo.

Decisions about the balance between the four strands and where to focus reviews depend on ERO’s assessment of risk and is informed by an analysis of the KWAS and information provided by the whānau at the initial hui.

In some kōhanga reo, reviews need to focus on compliance because of risks to the safety and wellbeing of tamariki. This is likely to be the case where, although the KWAS indicates high levels of compliance, there do not appear to be adequate systems for self audit of compliance.

Strands A and B will have the greatest emphasis for kōhanga reo where it is not necessary for reviews to focus on compliance.

 

Use of resources in the review strands

ERO has prepared resources for use in reviewing within the review strands, and these are included at the back of this framework.

Resource A: Identifying the priorities for review; and

Resource B: The Chain of Quality
are intended to be used in determining the priorities for reviews in Strands A and B.

Resource C: Information for whānau, iwi and hapū;

Resource D: Guidelines for involvement of the kaupapa kaimahi or alternative representative; and

Resource E: Developing recommendations
are intended to be used throughout the review process, but are again particularly applicable to Strands A and B.

Resource F: National interest
outlines the approach used to gather information on areas of specific national interest and is to be used in Strand C.

Resource G: Evaluation indicators
This is a separate document, and is to be used in Strands A and B

The Guidelines for Kōhanga Reo Whānau Management Assurance Statement and Self Audit Checklists, which are contained in a separate document, are intended to be used in Strand D.

 

5 Process guidelines for review

ERO begins the review process with a notification letter that gives the kōhanga reo time to prepare for the external review.

 

Review set up (exchange information)

In the initial stages of the review, information is exchanged between ERO and the kōhanga reo before the review team comes to the kōhanga reo for the on-site part of the review.

During the exchange of information ERO:

• begins the process with a notification letter that gives kōhanga reo adequate time to prepare for the external review;
• provides kōhanga reo with information about the approach to reviews, a copy of this framework, the Guidelines for Kōhanga Whanau Management Assurance Statement and Self Audit Checklists, and ERO’s evaluation indicators;
• informs kōhanga reo that ERO always focuses on the quality of education (Strands A and B) as this has a direct impact on outcomes for tamariki. As well, ERO always checks performance in respect to compliance in the KWAS to ensure there are systems established that provide an environment adequate for the safety and wellbeing of tamariki;
• makes telephone contact with kōhanga reo whānau. The review coordinator discusses requirements of the review with the whānau. This gives kōhanga reo an opportunity to clarify information about the review process;
• invites kōhanga reo to consider which aspects of the programme should be given priority in the review (Strand A); and
• clarifies the details for the whānau hui and discussion about review priorities along with details relating to the kaupapa kaimahi or alternative representative.

 

Whānau hui (meet to discuss priorities)

The Education Review Office team arrives as manuhiri.

While the previous exchange of information has generally been by mail, telephone or email, ERO expects that the whānau will further discuss priority areas of the review. This discussion takes place during the whānau hui. The whānau use this time to identify their areas of focus. They may nominate whānau members to speak about the areas of focus.

Documentation to support the review process, including the KWAS, is made available to the review team.

 

Scoping (refine the review process)

When initially scoping the review, ERO uses information that may be provided by the kōhanga reo during the whānau hui, including the completed KWAS and previous reporting history as applicable.

The review team:

• confirms the scope of the review after this discussion and reading the documents;
• plans the process taking into account the resources available for the review; and
• plans for working with the kaupapa kaimahi or alternative representative.

 

Investigation and synthesis at the kōhanga reo

The review team completes investigations in light of scoping information.

During its time at the kōhanga reo the review team:

• carries out the investigation using evaluative questions and evaluation indicators as a basis to gather and document evidence;
• reads whānau documentation, talks with kaiako, whānau (as a group or individually) and tamariki, and observes programmes in action (ongoing interaction between whānau and the review team will be a feature of the process); and
• synthesises the review findings.

 

Whānau discussion (discuss findings and develop recommendations)

The review team:

• meets to discuss the review findings with the kōhanga reo whānau. The discussion highlights areas of good performance and areas for improvement. ERO expects that whānau attending this meeting will include those who will have the responsibility for implementing any recommendations developed and for responding to the subsequent ERO report; and
• discusses and develops, with the whānau, recommendations to be included in the report. If appropriate they will indicate the likelihood of a follow-up supplementary review.

 

Reporting

The written report captures the essence of the dialogue that has taken place in the final whānau discussion. The format identifies:

• areas of good performance;
• areas for improvement; and
• recommendations.

The process for confirming the report follows standard processes outlined in ERO’s Manual of Standard Procedures. Where ERO identifies the need for a supplementary review this will be included in the report.

 

6      The Conduct of Reviews

Overview

ERO’s ‘assess and assist’ approach to reviews reflects a balance between accountability and improvement. Given ERO’s dual purpose, it is important for ERO and kōhanga reo to have a common understanding of the contribution each party makes to the review.

The following tables summarise how ERO will manage relationships and processes in reviews, and sets out ERO’s expectations of kōhanga reo whānau.

Information

ERO will:

• make information about the review process available to whānau; and
• take into consideration the results of self review and self audit when scoping the review.

ERO expects the kōhanga reo whānau to:

• make all relevant information available to ERO, including self review and self audit results;complete the KWAS;
• work constructively with review teams to provide access to information held at the kōhanga reo; and
• facilitate discussions with whānau members.

Priorities for the review

ERO will:

• discuss the priorities for the review with the whānau and take into account their suggestions; and
• inform the whānau if ERO needs to change the focus of the review as a result of its investigations.          

ERO expects the whānau to suggest priority areas for review that:

• reflect the views of whānau; and
• include a balance between issues where the whānau is already performing well and those where there is a need for improvement.

The review process

ERO will:

• reach judgements based on evidence;
• outline its findings to the whānau so that there are ‘no surprises’ in its report;
• outline the evidential basis for key findings;
• develop recommendations for improvement in consultation with the whānau; and
• not release its reports publicly until two weeks after they have been confirmed and sent to the kōhanga reo.          

ERO expects the whānau to:

• notify whānau that a review is scheduled to take place;
• inform ERO about protocols and kaupapa kaimahi details (or alternative representative);
• brief the kaupapa kaimahi or alternative representative about the review; and
• work constructively with review teams to develop recommendations based on ERO’s findings.

 

Resource A: Identifying the priorities for review

Overview

This resource is intended to assist in:

• defining areas for review in Strand A (Kōhanga Reo Priorities); and
• determining which aspects of planning and evaluation should be investigated in Strand B (Planning and Evaluation).

The questions in this resource can be used in a structured way to help define the priorities for review from a range of possible options.

The resource can be used by:

• ERO when scoping and planning the review (drawing on documentation and dialogue with the kōhanga reo whānau);
• kaupapa kaimahi or an alternative representative when considering priorities for review
• individual kōhanga reo whānau when considering priorities and preparing for the review; and
• individual kōhanga reo whānau and ERO to inform discussion at the start of the review.

 

Identifying review priorities

ERO will use the questions below as a basis for structuring and guiding discussions with the kōhanga reo whānau about priorities for review in Strands A and B.

 

1 How does the potential review priority contribute to outcomes for tamariki?

Consider how the issue contributes to the learning, development and wellbeing of tamariki using the chain of quality in Resource B. Will improvements in this aspect of the kōhanga reo lead to, or create the conditions for, improvements in the learning, development or wellbeing of tamariki?

 

2 What is the quality of existing information?

Consider the quality and extent of existing information (including self-review and self-audit information) that already exists to inform an external evaluation of the issue. This information will be used to scope the size of the task or, in some cases, to exclude it. Consideration is given to whānau dialogue.

 

3 What level of consultation has already occurred?

Before the review, kōhanga reo whānau may have followed a process of consultation for defining their own priorities.

To assist in planning the review, identify which of the following groups have been involved in consultation on the review priorities and decide who ERO should talk to in the review. Key groups include:

• whānau members;
• the kaupapa kaimahi or alternative representative;
• the licensee;
• whānau management;
• kaiako;
• tamariki; and
• other groups.

 

4 Has a need for improvement already been identified?

Identify whether self-review results, including dialogue within the whānau, indicate that:

• the kōhanga reo is already performing well in relation to this area; or
• there is a need for improvement.

ERO will usually aim to achieve a balance between these two categories in a review. For areas where the kōhanga reo is performing well, the focus will be on validating the results of self review and identifying good practice. Where there is a need, ERO will work with the whānau to formulate strategies for improvement. Where an action plan is in place, ERO will take this into consideration.

 

Resource B: Chain of Quality

Overview

This resource is intended to supplement Resource A to help guide discussions about the priorities for review. The Chain of Quality diagram is a basis for examining visually the relationship between the kōhanga reo processes and outcomes for tamariki. It can be used to:

• assist kōhanga reo whānau in self review; and
• assist ERO, kōhanga reo and kaupapa kaimahi or alternative representative in identifying priorities for review.

 

Chain of Quality link

The Chain of Quality diagram in this resource (page 16) shows how positive outcomes for tamariki are linked to effective processes – whānau management, professional kaiako and high quality programmes. Underpinning all the links in the chain are contextual factors including the philosophy of kōhanga reo, Te Whāriki and the involvement of communities.

Outcomes for tamariki are not always easy to assess directly. There are currently few measures for assessing the contribution that kōhanga reo whānau make to outcomes for tamariki.

Nevertheless, the focus on how well tamariki learn and develop is central to all ERO reviews. ERO’s methodology and professional practice considers how different kōhanga reo processes contribute to learning and development outcomes for tamariki.

Within a kōhanga reo context, outcomes for tamariki are likely to be influenced most directly by the quality of whānau planning, evaluation and assessment, the learning environment and interactions. This is why these areas are included as priorities for all kōhanga reo reviews (Strands A and B).

Issues such as the quality of kaiako and whānau management are important because they affect the quality of programmes, but their influence may be less direct than the quality of the programmes themselves. These issues are investigated in Strands A and B.

 

Minimum standards

The Chain of Quality does not include compliance with requirements for the physical and emotional safety of tamariki. These requirements are expressed in the kōhanga reo regulations in terms of minimum standards rather than on a quality spectrum. Ensuring that kōhanga reo whānau meet these requirements is a compliance issue and is the responsibility of whānau management.

ERO’s interest in checking minimum standards is part of the compliance matter of reviews. However, the extent to which whānau management has good systems for reviewing compliance with safety regulatory requirements is a quality issue, and could be evaluated by ERO as part of the “effective whānau management” link in the Chain of Quality (see diagram below).

 

Using the Chain of Quality

The Chain of Quality is a tool to assist kōhanga reo whānau and ERO to determine the nature of the link in the context of individual kōhanga reo.

While ERO reviews could focus on any of the links in the chain, in all cases it should be possible to trace a relationship between the issue for review and outcomes for tamariki. Areas for improvement identified in the review should lead to (or help create the conditions for) improved learning and development outcomes for tamariki.

A useful exercise for kōhanga reo whānau is to ‘fill in’ or list the key components of the links and the specific ways in which the activities in one link can influence those in the next. The process of ERO and the kōhanga reo whānau working together in tracing the link may be useful not only in deciding what to include and what to exclude, but also in clarifying the detailed specific priorities for the review.

 

Example

Whānau management of a kōhanga reo might identify Mana Aoturoa as a priority for review. Using the Chain of Quality diagram as a guide, ERO and the whānau could expand this focus and decide to evaluate Mana Tangata to determine how these two areas contribute to a high quality and stimulating learning environment, which in turn helps to improve learning and development.

For each of the links in the chain, this resource includes evaluative questions that are intended to clarify issues for review. The questions are included in this framework to inform kōhanga reo whānau about the basis on which they will be reviewed.

Not all these questions will be used in all reviews. Instead, specific questions that are relevant to a particular review will be identified by the priorities decided at the beginning of the review.

Chain of Quality
Te Kōhanga Reo Philosophy Te Korowai

 

Evaluative questions

 

Delivering positive outcomes for tamariki

How effective is the kōhanga reo in promoting and extending the use of te reo Māori by tamariki? (Mana Reo)

How effective is the kōhanga reo in promoting and developing tikanga Māori?
(Mana Atua)

How well does the kōhanga reo promote and extend the learning and development of tamariki through the provision of a high quality programme?
(Planning and Evaluation – Mana Tangata)

How effective is:

• programme planning?
• assessment?
• programme evaluation?

What is the quality of the programme experienced by tamariki? (Planning and Evaluation - Mana Tangata)

What expectations does the whānau have for the learning and development of tamariki? (Planning and Evaluation - Mana Tangata)

How effectively do programmes extend the knowledge of the world for tamariki? (Mana Aoturoa)

How effectively do programmes honour all people and respect individual uniqueness? (Mana Tangata)

How effectively do learning programmes assist tamariki to understand their environment? (Mana Whenua)

How well does the kōhanga reo promote the emotional safety and security of tamariki?

 

Learning environment

How well do the environment and resources support (Mana Tangata):

• learning and development needs of tamariki?
• physical, social and emotional needs of tamariki?

 

High quality kaiako and kaiāwhina

What is the quality of the process for curriculum development, programme delivery, quality improvement and involving whānau in the kōhanga reo? (Planning and Evaluation)

What is the quality of pedagogical practice demonstrated through interactions between kaiako/kaiāwhina and tamariki? (Mana Tangata)

How effectively do the kaiako and kaiāwhina respond to the learning needs of tamariki?
(Mana Tangata)

 

Effective whānau management

How effective is the whānau in setting direction for the kōhanga reo? (Planning and Evaluation)

How effectively does the whānau plan, manage and evaluate the operation of the kōhanga reo? (Planning and Evaluation)

How effective are self-review processes in bringing about improvements likely to have a positive impact on learning, development and wellbeing of tamariki? (Mana Tangata)

To what extent do whānau practices ensure a safe physical environment? (Mana Whenua)

How effectively does the whānau promote the holistic wellbeing of tamariki? (Mana Atua)

To what extent are opportunities provided for whānau learning? (Mana Tangata)

How effectively does the whānau ensure the language of the kōhanga reo is Māori? (Mana Reo)

How well does the whānau meet its obligations to be a good employer? (Mana Tangata)

How effective are the systems for managing performance and identifying and meeting the professional development needs of kaiako/kaiāwhina? (Mana Tangata)

 

Kōhanga reo philosophy Te Korowai

How does the kōhanga reo philosophy of Te Korowai impact on whānau management and the quality of education for tamariki?

 

Iwi, hapū, whānau

How effectively are iwi, hapū and whānau (Mana Tangata) involved in:

• the operation and decision-making of the kōhanga reo?
• setting goals for tamariki and knowing how well tamariki are progressing?

How well are iwi, hapū and whānau kept informed? (Mana Tangata)

How effective are the systems for resolving concerns and complaints?

How effective are the interactions between kaiako, kaiāwhina and whānau? (Mana Tangata)

 

Resource C: Information for whanau, iwi and hapū

Overview

One of the purposes of ERO is to provide information to whānau to assist in their decision-making. This resource is intended to highlight issues that are of interest to whānau and the wider public, and that should be addressed in ERO reports. The information needs of whānau, and the ways in which whānau can use information provided by ERO in their decision-making, vary widely.

 

Whānau involvement with the kōhanga reo

Involvement of the whānau is part of the philosophy of kōhanga reo. Kōhanga reo have a particular role in developing and educating whānau as well as tamariki. As a result, whānau whose tamariki attend kōhanga reo are likely to have a day-to-day involvement with the service and to be familiar with the programmes.

In deciding on priorities for review, ERO will take into consideration the extent to which whānau, iwi and hapū have been involved in discussing priorities for review, and will determine the extent to which discussions with whānau, iwi and hapū representatives should take place during the review.

 

Whānau choosing a kōhanga reo

For many whānau, the decision to enrol their child at a kōhanga reo is the first time they come into contact with the education system, and with ERO. Whānau may have little information about the benefits of kōhanga reo, the quality of education offered at a particular kōhanga, or how to choose between different kōhanga.

Whānau may look to ERO reports for general information on what counts as quality in kōhanga reo (the key features they should look for before deciding to enrol their child). ERO reports have an important role to play in communicating to whānau the aspects of activities and processes that are likely to have the most impact on outcomes for tamariki.

In relation to a particular kōhanga reo, whānau may ask questions such as:

• What sort of kōhanga reo is this and what distinguishes it from others (nature of the programme, iwi, hapū, structure, environment etc)?
• Is there a high quality programme?
• Is the kōhanga reo providing high quality literacy and numeracy programmes (especially for older tamariki)?
• What qualifications and experience do staff have?
• Will my tamaiti be safe?
• What is the state of whānau management?

ERO reports need to contain sufficient information to enable whānau to make judgements on these issues.

 

Meeting the needs of whānau

ERO will recognise the information needs of whānau during reviews and in reports through:

• gathering general information on the quality of education likely to be of interest to whānau
• checking compliance with regulations in all reviews; and
• including a community page in reports that draws on information obtained through each of the four review strands, explicitly addressing questions of interest to whānau.

 

Example – Information to be Included in the community page

The following list is indicative only. Not all the questions will be relevant to all reviews, some of the questions could be merged and other questions could be identified for particular kōhanga reo whānau.

Information Area	Information to Include
What sort of kōhanga reo is this?	Characteristics of the kōhanga reo and distinctive features of its programme, iwi, hapū, structure and environment.
Is there a high quality programme?	Information drawn from the quality of the programme, learning environment and interactions.
Is the kōhanga reo providing a sound foundation for learning?	If appropriate, information from the kōhanga reo priorities and planning and evaluation strands, for example on the quality of early literacy and numeracy programmes.
What is the quality of the kaiako?	Information drawn from the compliance strand on the extent to which the kōhanga is meeting requirements for staff qualifications. If appropriate, information from the kōhanga reo priorities and planning and evaluation strands, for example on how well the staff are interacting with tamariki.
What is the quality of the kaiako?	Information drawn from the compliance strand on the extent to which the kōhanga is meeting requirements for staff qualifications. If appropriate, information from the kōhanga reo priorities and planning and evaluation strands, for example on how well the staff are interacting with tamariki.
Will my tamaiti be safe?	Information drawn from the compliance strand on the extent to which the kōhanga reo is meeting safety requirements in the regulations.
What is the state of whānau management?	Information drawn from the compliance strand on the extent to which the kōhanga reo is meeting requirements and regulations.

 

Resource D: Guidelines for involvement of the kaupapa kaimahi or alternative representative in reviews

Purpose

Before the review, Te Kōhanga Reo National Trust is responsible for identifying the kaupapa kaimahi to take part in the reviews. The purpose is to provide an additional mechanism for making relevant information available to ERO and ensure the interests of the kōhanga whānau are taken into account. The whānau is able to select an alternative representative should they decide not to not use a kaupapa kaimahi.

 

The role of the kaupapa kaimahi or alternative representative

Before the review starts whānau management confirms for ERO the kaupapa kaimahi, or alternative representative, who will take part in the review.

 

Parameters related to the kaupapa kaimahi or alternative representative

The decision to include a kaupapa kaimahi or alternative representative in a review is made by the kōhanga.

Te Kōhanga Reo National Trust determines the parameters within which the kaupapa kaimahi may operate. Whānau who elect to have an alternative representative must consider his or her role.

Te Kōhanga Reo National Trust will meet all costs related to the kaupapa kaimahi. The whānau will meet all the costs should they choose an alternative representative.

The kaupapa kaimahi is responsible to both Te Kōhanga Reo National Trust and whānau management during the time they are involved in the review.

An alternative representative is responsible to whānau management during the time that they are involved in the review.

Whānau management is responsible for the kaupapa kaimahi or the alternative representative, and for providing them with access to information during and following the review.

ERO may provide opportunities for individuals to give review officers information in confidence, without the kaupapa kaimahi or alternative representative being present.

The kaupapa kaimahi will not be a designated review officer. ERO will not veto the choice made by the whānau. Where an alternative representative is nominated this person will not be designated.

 

Relationships and processes for involvement of kaupapa kaimahi

What ERO will do

• Provide whānau management with broad guidelines as shown in Resource D.
• Allow the kaupapa kaimahi or alternative representative full involvement in the review including the opportunity to:
  • attend meetings and interviews carried out as part of the review;
  • see all documents the whānau wishes to make available; and
  • participate in review team discussions.
• Consider information provided by the kaupapa kaimahi or alternative representative when drafting the report.

What the management of the kōhanga reo will do

• Decide on the framework within which the kaupapa kaimahi will be involved.
• Meet with the kaupapa kaimahi or alternative representative.
• Notify ERO of the decision to have the kaupapa kaimahi or alternative representative.                    

What the kaupapa kaimahi / alternative representative will do

• Agree to work within ERO’s timelines and procedures for carrying out the review.
• Support the whānau to complete the KWAS.
• Participate in those aspects of the review process whānau management wishes.
• Participate in meetings and review team discussions.
• The kaupapa kaimahi or alternative representative will be able to comment on ERO’s unconfirmed report through the whānau management.

 

Resource E: Developing recommendations

Overview

The purpose of developing recommendations is to aim for improvement in areas likely to have a direct and positive impact on learning and development outcomes for tamariki.

All ERO reports are expected to include some recommendations. There is no direct link between the number of recommendations in a report and the overall performance of a kōhanga reo. Recommendations focus on improvement, and all kōhanga reo are able to improve.

In some cases a single, specific course of action might not be the most useful response to a particular problem. In these cases ERO may consider providing a list of options or ideas which may be used in discussion with the kōhanga reo whānau to choose the most appropriate strategy.

Recommendations only deal with significant issues.

 

Consultation with the kōhanga reo whānau management

As much as possible, recommendations will be developed in consultation with kōhanga reo whānau management at the time of the discussion meeting to clarify emerging findings. As part of this discussion, ERO and the whānau will discuss which options and ideas are practical and achievable. If it is not possible to develop recommendations at the discussion meeting then these should be developed as soon as possible after the meeting.

The review team may still make recommendations even if the discussions with the whānau are not successful in agreeing on the areas for improvement.

 

Types of recommendations

Recommendations fall under one of the following types:

• specific corrective action;
• further investigation by the management of the kōhanga reo;
• use of external advice and support by the kōhanga reo in specific areas;
• introduction of new processes based on good practice at other kōhanga reo;
• ongoing monitoring (where the whānau already has a process to address the issue); and/o
• recommendations to other agencies (for example, Te Kōhanga Reo National Trust).

 

Resource F: National interest

Overview

The purpose of this resource is to clarify the mechanisms that ERO will use to investigate areas of national interest.

The Government has specific policies to achieve its objectives, most of which are implemented by the Ministry of Education.

One of the purposes of ERO reviews is to provide information to the Government on the extent to which these policies are successful in achieving the Government’s objectives.

However, in kōhanga reo this may not be relevant. Information required by the Government that pertains to kōhanga reo will be gathered.

 

Areas of national interest

Since 1994 ERO has aggregated information from its reviews of schools and early childhood education services and published national reports on aspects of the pre-tertiary education sector of specific interest to the Government.

ERO collects, analyses and reports information nationally for the purpose of educational improvement.

ERO reports on individual schools and early childhood services can be used by institutions to bring about immediate and local improvement. National information about the education system as a whole can be used by Government to bring about long-term and systemic educational improvement.

 

Determining areas of national interest

Each year, in discussion with its Minister and the Ministry of Education, and drawing on information from review officers and from its liaison meetings with education sector groups, ERO decides on areas for national reporting that are of current interest to Government.

ERO draws up an annual list of approximately 20 areas for study that may be amended during the year if particular topics of urgent interest to the Government arise.

In recent years ERO has produced a series of reports on What Counts as Quality in early childhood services, including kōhanga reo.

 

Collecting information of national interest

ERO uses a range of ways to gather information of interest to Government depending on the scope and focus of the information needed.

Currently ERO uses one or a combination of the following evaluative approaches:

• analysis of confirmed ERO reports on schools/early childhood services;
• specific in-depth questions asked by review officers during reviews;
• questionnaires sent directly by ERO to schools/early childhood services;
• specific studies undertaken by ERO in schools/early childhood services outside education reviews; and/or
• focus groups of review officers.

 

Collecting information of national interest during a review

Review officers carry out any investigations of areas of national interest in a similar way to that used for other aspects of a review. That is, they ask questions and seek evidence to support statements provided by the establishment. They record their evidence before evaluating and reporting it.

Depending on the area of national interest at the time of an ERO review, a kōhanga reo might be asked to provide some specific information outside the review strands through interviews or the provision of documents.

 

Areas of national interest in reports

For transparency purposes information about areas of national interest will be included in a separate section in the report. When the report is confirmed it will be analysed and the information from all reports will be aggregated by ERO.

 

Disseminating information of national interest

Reports drawing together national information are, in the first instance, provided for and discussed with the Minister responsible for ERO. They are then sent to the Secretary for Education and discussed with appropriate Ministry of Education officials with the aim of providing information for ongoing education policy development and implementation.

As the information is also of interest to the education sector, national reports are published either in booklet form or on the ERO website (www.ero.govt.nz). Published booklets are sent directly to schools or early childhood services free of charge. Multiple copies can usually be provided on request.

More than 100 of these reports have been produced, providing a national or regional picture of ERO’s review findings based on actual school or early childhood service practices.

 

Resource G

Overview

ERO’s approach to evaluation and reviews is supported by a range of tools and experience. One of these tools is a set of evaluation indicators.

The evaluation indicators have been developed in consultation with Te Kōhanga Reo National Trust. Their experience and direct involvement has provided a kōhanga-specific interpretation. It is envisaged that these will develop and amendments will be made. This is an additional resource.