The whole point of internal evaluation is to assess what is and is not working, and for whom, and then to determine what changes are needed, particularly to advance equity and excellence goals. Internal evaluation involves asking good questions, gathering fit-for-purpose data and information, and then making sense of that information. Much more than a technical process, evaluation is deeply influenced by the school's values and how it sees its role in the community. Effective internal evaluation is always driven by the motivation to improve, to do better for the students.
When internal evaluation is done well, processes are coherent and align with schools' visions and strategic goals. Leaders and teachers work collaboratively across teams, syndicates, departments, faculties and in some cases communities of learning, to ensure that the efforts that go into evaluation lead to improvement. The urgency to improve is shared by all, and can be articulated by all.
Evidence from research and the case study schools shows that there are organisational conditions that support development of the capacity to do and use evaluation for improvement and innovation. These include:
See the Domain 6 indicators for practices that are characteristic of effective evaluation, inquiry and knowledge building for improvement and innovation and Section 3 of this guide for examples of how the conditions, capability and capacity to undertake effective internal evaluation in a school can be developed. Section 6 of this guide emphasises the importance of using the same set of indicators for both internal and external evaluation.
Internal evaluations vary greatly in scope, depth and focus depending on the purpose and the context. An evaluation may be strategic, linked to vision, values, goals and targets; or it may be a business-as-usual review of, for example, the curriculum or a learning area; or it may be a response to an unforeseen (emergent) event or issue. Figure 2 shows how these different purposes can all be viewed as part of a common improvement agenda.
FIGURE 2: TYPES OF INTERNAL EVALUATION
Strategic evaluations focus on activities related to the vision, values, goals and targets of the school community. They aim to find out to what extent the vision is being realised, goals and targets achieved, and progress made. For boards of trustees and leaders strategic evaluations are a means of answering such key questions as: To what extent are all our learners experiencing success? To what extent are improvement initiatives making a difference for all learners? How can we do better? Because strategic evaluations delve into matters that affect the school as a whole, and the wider community, they need to be in-depth and they take time.
A strategic evaluation
An intermediate school and a secondary school decided to adopt a joint approach to engaging with their community. In 2007 and 2011 they carried out extensive community consultation to determine what parents and community members considered important outcomes for their children. The results led to the development of seven strategic goals that were reaffirmed in 2011 and 2013, included in the schools' strategic plans, and incorporated into annual goals and targets.
In the intermediate school, a 'story board' approach is used to report on the annual action plan. The principal's reports to the board use a 'traffic light' system to indicate how the school is tracking in relation to its different strategic goals. This reporting is to a schedule, with one strategic goal as the focus each month. This approach keeps board members informed and supports their decision making.
To monitor progress towards its goals the board seeks regular reports from school leaders, asking How well are we doing? What evidence of progress do we have? What is working well? What do we need to adjust and why?
Regular monitoring and reporting enables the board to make resourcing decisions that align with its strategic priorities.
Regular (planned) evaluations are business-as-usual evaluations or inquiries, where boards of trustees, leaders and teachers (and, where appropriate, students) gather data, monitor progress towards goals, and assess the effectiveness of programmes or interventions. They ask: To what extent do our policies and practices promote the learning and wellbeing of all students? How fully have we implemented the policies we have put in place to improve outcomes for all learners? How effective are our strategies for accelerating the progress of target learners? Business-as-usual evaluations vary in scope and depth and feed back into the school's strategic and annual plans.
A regular evaluation
The social sciences faculty came to evaluate its junior programme as part of its regular review cycle. Feedback from some senior students had highlighted an apparent lack of alignment between the junior and senior curriculums.
The first step was to find out what the students thought about the social sciences curriculum, so faculty staff developed and carried out a survey. But the resulting data was compromised (some students misunderstood questions and some completed the survey too quickly) and did not give staff the necessary depth of information. They then realised that they lacked the technical expertise to design a survey that would tell them what they needed to know.
Deciding that a different approach was needed, staff set up think tank groups that included students from each class. Teachers presented the survey data to the students and, by discussing it, were able to get in-depth answers to the survey questions.
Following the think tanks there was a lot of discussion within the faculty about the philosophy underpinning the current curriculum, the types of assessments used, and what was and was not working. Staff visited other schools to see their programmes and assess the possible relevance for them of the different approaches. A Google Doc was set up to facilitate collaborative redevelopment of the faculty's programme. A framework for planning units was agreed, and responsibilities for specific topics allocated. Much of this work took place at the faculty's regular professional learning meetings.
Because the level 5 social studies achievement objectives (AOs) are very broad, staff developed explicit curriculum links to senior social sciences subjects - links that the students could understand. A benefit of this process was that it required staff to focus closely on the AOs and ask themselves whether they were really meeting those objectives and providing a platform for student achievement in the senior years. Sometime later, when faculty staff conducted a follow-up evaluation, again using a survey and think tank, the response to the changes was overwhelming positive.
Emergent (or spontaneous) evaluations are a response to an unforeseen event or an issue picked up by routine scanning or monitoring. Possible focus questions include: What is happening? Who for? Is this okay? Should we be concerned? Why? Do we need to take a closer look? Emergent evaluations arise out of high levels of awareness about what is happening for learners.
An emergent evaluation
Leaders and teachers in a primary school found that samples of student writing gathered at the start of the school year revealed that the writing of 42 percent of year 4-6 students deteriorated over the summer holiday break. This decline in performance was observed across ethnicity, year level and teacher. An analysis of November-to-November data showed the students not making the expected progress. Leaders recognised that they needed to do something differently, but that the school had limited influence over what happened in the holidays.
Thinking about what action they might take to improve the situation the staff did some reading on the 'summer effect' but found little of use. So they decided to experiment with an entirely new strategy. This involved collecting a sample of each student's writing at the end of the year, pasting it into the front of their exercise book at the start of the new year, and then making their final learning intention for the old year their first learning intention for the new year.
Expectations are now set on day one. Teachers refer back to the previous year's writing sample and discuss with their students the quality of work that is expected of them, emphasising that this is what they are capable of. Leaders ensure that teachers have the student data from the previous year and that they have the time to determine learning strategies for the start of the year. In this way each teacher can start where the previous teacher left off, without the need to reassess students.
In the first year of implementing this strategy the leadership team rigorously monitored writing data and what teachers were doing in their classrooms, ensuring that the students were writing every day. These strategies have worked well. Now, less than 10 percent of students go backwards in their writing over summer. Explicit teaching strategies target this group with the aim of accelerating their progress.
Internal evaluation requires boards, leaders and teachers to engage in deliberate, systematic processes and reasoning, with improved outcomes for all learners as the ultimate aim.
Those involved collaborate to:
Figure 3 identifies five interconnected, learner-focused processes that are integral to effective evaluation for improvement.
Figures 4 to 8 unpack each of these processes in terms of the conditions that support their effectiveness, the reasoning involved, and the activities or actions involved.
FIGURE 3: LEARNER-FOCUSED EVALUATION PROCESSES AND REASONING
Why not start a discussion about what each of the five evaluation processes might mean in your own school community? This will clarify your thinking about evaluation and evaluation practices and help identify areas where you need to develop greater capability or capacity.
Effective evaluation requires us to think deeply about the data and information we gather and what it means in terms of priorities for action. By asking the right questions of ourselves, we will keep the focus on our learners, particularly those for whom current practice is not working. The twin imperatives of excellence and equitable outcomes should always be front and centre whatever it is that we are evaluating.
Internal evaluation is most effective when the organisational conditions are supportive and staff members are encouraged to develop the capabilities to do it well. There is no one way of developing these conditions and capabilities, but there are some actions and decisions that are likely to help.
The diagrams that follow (Figures 9 and 10) are based on the experiences of some of the case study schools as they went about developing the conditions and capabilities they needed to engage in effective internal evaluation. Consider using them in your school community to initiate discussion about the extent to which you have practices in place that promote effective evaluation and identify areas for improvement.
FIGURE 9: ORGANISATIONAL CONDITIONS FOR INTERNAL EVALUATION
How can we develop the organisational conditions to support evaluation and inquiry in our school community?
We have not yet developed coherent organisational conditions that support evaluation and inquiry
Our vision, values, goals and priorities are developed, planned and implemented as isolated and disconnected activities rather than underpinned by systematic evaluation and inquiry processes.
We have a limited range of tools and methods to gather, store and retrieve a range of valid and fit-for-purpose data.
We gather feedback from students and our community on a regular basis but are not using this well to identify priorities for inquiry and improvement.
Our efforts to change and improve outcomes for all learners need to be better aligned across the school, building on the 'pockets' of collaboration that already exist.
We are not yet allocating sufficient resources (for example, time, expertise, staffing) to support change and improvement.
Our processes for evaluation and inquiry tend to be ad hoc and disconnected.
We have limited ways of sharing and disseminating new knowledge and this is a barrier to improvement and innovation.
Examples of development/improvement actions taken by case study schools
A collaborative approach to evaluation and reporting is encouraged by involving trustees, leaders and teachers in reviewing student achievement outcomes and identifying areas for strategic focus for the next year.
We draw on an increasing range of information as a starting point for our regular curriculum/syndicate reviews: emerging issues; teacher reflection and feedback; student achievement data; student feedback through surveys and classroom observation; and research.
Student feedback is actively sought to ascertain the success of initiatives and issues for further investigation. Several methods for eliciting student feedback are being trialled including focus groups or think tanks and in-class discussions.
Leaders take time to introduce new processes to build trust necessary for engagement in meaningful inquiry. They keep workloads and change processes manageable.
Leaders have aligned our inquiry and evaluation activities, for example linking teacher appraisal and inquiry projects to maximise the impact on what is happening for identified learners.
Professional learning groups provide a forum to discuss and inquire into particular issues. Teachers explore, test and monitor their responses to their identified issues. New learnings are shared through whole staff feedback sessions.
We have coherent organisational conditions that support evaluation and inquiry
We embed evaluation into the way we plan and take action to realise our vision, values, goals and priorities.
We use appropriate tools and methods to gather, store and retrieve a range of valid and fit-for-purpose data.
We recognise the importance of student and community voice and draw on these perspectives to identify priorities for inquiry and improvement.
We have high levels of relational trust that support collaboration, risk taking, and openness to change and improvement.
We allocate sufficient resources (for example, time, expertise, staffing) to support change and improvement.
We use systematic, coherent and 'smart' evaluation and reasoning processes at all levels of our school.
We share and disseminate new knowledge in ways that promote improvement and innovation.
Figure 10: CAPABILITY AND COLLECTIVE CAPACITY
How can we build the capability and collective capacity to do and use evaluation for improvement?
We are yet to build the capability and collective capacity to do and use evaluation for improvement
Our professional learning is focused on teachers' interests and/or the expertise of available professional development providers. We are yet to engage in professional learning that develops the knowledge, skills and confidence needed for evaluation and inquiry.
We have limited or no access to expertise (internal/external) to build our capability and collective capacity in evaluation and inquiry.
Our leaders and teachers are starting to develop data literacy. We do rely on one or two people to manage/analyse and interpret data.
We are not aware of/well informed about current research evidence about what makes the biggest difference for learners.
We are not yet confident in using evaluation and inquiry processes for improvement.
Our use of evidence in decision making is limited.
Examples of development/improvement actions taken by the case study school
Staff meetings are professional forums. The focus is on building professional capability by teachers sharing readings, research and practice.
Leaders are clear about the steps in the change process, framing up questions for teachers to ask of their practice and developing tools to support the inquiry process.
Leaders adopt a more collaborative approach to working with data by increasing the use of data as a focus for discussions in syndicate meetings; conversations between teachers; and within the senior leadership team.
Leaders engage with research evidence and share relevant research with teachers to make better use of evidence in improving teaching practice.
Leaders have a clear focus on embedding evaluation and inquiry processes by supporting team leaders to work with teachers to unpack their data; deepening conversations about data and practice; de-privatising classroom practice through use of video; and extending teacher engagement in professional reading through regular discussions and debate.
We have the capability and collective capacity to do and use evaluation for improvement
We engage in professional learning, mentoring and coaching to develop the knowledge, skills and confidence needed for evaluation and inquiry.
We carefully select and use relevant expertise (internal and external) to build our capability and collective capacity in evaluation and inquiry.
Our leaders and teachers are data literate.
They ask good questions, use relevant data, clarify purposes, recognise sound and unsound evidence, understand statistical concepts, focus on interpretation, and engage in evidence-informed conversations.
When generating solutions and making decisions, we draw on current research evidence about what makes a bigger difference for learners.
Engaging in evaluation contributes to changes in our thinking and behaviour and builds inquiry 'habits of mind'.
Participation in evidence-based decision making builds our efficacy and agency.