Effective Communities of Learning | Kāhui Ako are clear about the purpose and focus of their collaboration. The learner is firmly at the centre and identifying what matters to improve their outcomes underpins vision, purpose and focus.
Working collaboratively to identify a small number of ambitious and measurable achievement challenges, that are clearly articulated and understood by students, teachers, leaders, parents and whānau provide the basis for the CoL | Kāhui Ako focus.
The starting point for identifying common goals and achievement challenges is effective internal evaluation. Doing this well at both individual, organisation and community levels requires asking the right questions and accessing relevant, transparent and timely information to inform the decisions made.
Internal evaluation is both a starting point and an ongoing process. In the first instance it requires leaders, teachers and community stakeholders to engage in the deliberate and systematic analysis of data to identify priorities for improvement.8 ERO’s 2016 publication Effective School Evaluation – How to do and use internal evaluation for improvement outlines some of the questions that can be used to help work towards determining the most appropriate challenges based on CoL | Kāhui Ako data.
|Noticing||Investigating||Collaborative sense making|
what's going on here?
For which learners?
Is this what we expected?
Is this good?
Should we be concerned? Why?
What is the problem or issue?
What do we need a closer look at?
What do we already know about this?
How do we know?
What more do we need to find out?
How might we do this?
What good questions should we ask?
What relevant and useful data do we need?
Do we need help to interrgate our data?
What is our data telling us?
Is this good enough?
WHat do we know about factors that might be driving variation?
Do we have different interpretations of the data? If so, why?
What do we need to explore further?
What can we learn from research evidence about what good looks like?
Do we need support to help make sense of our data?
Ongoing use of data on learner and institutional performance enhances collaboration by enabling participants to identify progress and know that what they are doing is having an impact. The visibility of measurable impact builds collective efficacy and ownership and promotes further improvement efforts.
|The big picture|
All endorsed achievement challenges9 focus on those areas for which there is national data in reading, writing, mathematics and NCEA. In some cases, national data together with supplementary local data has led to challenges being set in other curriculum areas such as science.
The most effective achievement challenges move beyond overall cohort targets (e.g. 85% of students reading at or above national standards) to identify the actual numbers of students in each school required to be accelerated to meet the targets set. Additionally, the more specific the data, the more clarity there is about targeted groups in the CoL | Kāhui Ako at risk of not achieving. We are finding the more recently endorsed challenges reflect a much higher level of specificity.
Targets set at Years 9 and 10 help to ensure that achievement challenges take account of the whole learner pathway. The majority of CoL | Kāhui Ako have targets set in the primary and senior secondary years, but not all challenges include the critical area of Years 9 and 10. This may be because there is a lack of national data at these levels but there is no reason why data sources based on curriculum levels or locally sourced assessment data could not be used to support establishing targets.
CoL | Kāhui Ako need to consider the impact that targeting achievement in the early years of secondary school will have on raising the number of students achieving at various NCEA levels. In addition, the transitions from early learning services to Year 1, Year 6 to 7 and Year 8 to 9 are all key transition points for consideration in setting coherent achievement challenges. There is also scope to explore agreed markers of progress at these key transition points.
For CoL | Kāhui Ako where there is no strong history of collaboration, or where there is no clear data literacy leadership in the group, the time taken to work through the data interrogation and analysis processes and to agree achievement challenges can take considerable time – in some instances, up to 18 months.
There appear a number of reasons for this including the need to establish a sufficient level of relational trust to work transparently with each other’s data, and the need for early facilitation and support to develop the necessary skills for dealing with large data sets. The latter is not surprising given that it has taken time nationally to develop the skills required to make sense of school level data and its use.
CoL | Kāhui Ako are positive about the support Ministry personnel and other facilitators have provided in helping member schools to work with data, particularly where there is insufficient data analysis capability to work independently. The proposed use of ‘expert partners’ working alongside CoL | Kāhui Ako will provide valuable support during the establishment phase to build capability and to support the relational conditions necessary to enable frank conversations about data.
We found that some of the earlier forming CoL | Kāhui Ako expressed frustration at having to resubmit achievement challenges because these did not meet ‘Ministry requirements’. It is likely that the issue was more about not knowing how to use data to specify an effective achievement challenge. The redrafted challenges resulting from targeted support illustrate the need to build capability to undertake this critical part of developing the focus of the CoL | Kāhui Ako.
More recently established CoL | Kähui Ako are benefitting from the support of lead advisers and the availability of examples of endorsed challenges from other CoL | Kähui Ako.
In a few instances there was confusion between what is perceived by CoL | Kāhui Ako as having to work with a ‘narrow’ data set when developing achievement challenges and the autonomy some believe they should have in determining challenges outside the scope of the national data set.
While some CoL | Kāhui Ako are keen to broaden the scope of their challenges, a strong focus on raising achievement levels in literacy and mathematics is a precursor to students achieving in the broader curriculum. This is not clearly understood by some and is an area where early intervention or facilitation leads to a more appropriate focus on high leverage, measurable challenges.
Where there is compelling evidence/data in curriculum areas other than literacy or mathematics or in such areas as the key competencies, there is no reason why CoL | Kāhui Ako cannot develop relevant challenges. If robust internal evaluation and data analysis points to a priority area of improvement and there is baseline evidence from which to determine progress and achievement, there may be a strong reason for its inclusion.
Some CoL | Kāhui Ako need support to understand the difference between evidence- based targets and the learning conditions and pedagogies that are necessary to enact these challenges. There is evidence of some CoL | Kāhui Ako setting additional targets/goals to complement what they see as ‘academic’. These include areas such as individualising/personalising learning, readiness for learning, digital learning, deep learning and strengthening wellbeing conditions. While these all have merit, they sit more appropriately as part of the ‘how’ not the ‘what’ of setting targets.
For the most part it is principals, senior staff and in some cases teachers, who are most closely involved in the collaborative activities of data analysis and internal evaluation. This is not surprising given that educators are most closely involved with identifying what is working and what needs improvement.
We found that a few CoL | Kāhui Ako had used their initial resources to employ an external facilitator or expert. Alongside support received from the Ministry such as Student Achievement Practitioners, this was described as being a very useful process. The brief usually embraced support to develop collaborative processes in order to better enquire into data. CoL | Kāhui Ako that have employed this approach talk about having a deeper understanding of what sits behind their agreed challenges.
The extent to which boards of trustees are actively involved in the development of achievement challenges varies considerably. Trustees are more likely to be kept informed about the process by their school principal than through direct involvement. In some cases the board chair or a delegated trustee attends the discussions about achievement challenges, which usually follow initial analysis by principals or a delegated sub-group of professionals. In a few effective cases – the board, parents and community representatives take part in discussions about the tentative achievement challenges before these are submitted for endorsement.
Achievement challenges are enacted at school level, so board involvement and understanding is critical to successful implementation. Through its stewardship role, boards have strategic oversight on behalf of the CoL | Kāhui Ako about how well their own school is contributing to the overall plan to raise student achievement. Hence the need for trustees to be clear about the accountability role they have within their CoL | Kāhui Ako.
There is potential scope for early learning services considering participation in CoL | Kāhui Ako to be involved in discussion about achievement challenges. Given that the majority of challenges involve a focus on literacy and mathematics, CoL | Kāhui Ako members would benefit from including evidence of what children entering school bring with them at transition and a more robust understanding how the curriculum is structured in early learning services.