Background

Context

From January 2020 all schools are expected to be implementing the new Digital Technologies (DT) curriculum content for students in Years 1 to 10. The curriculum content incorporates two new strands in the Technology learning area: Computational Thinking and Designing and Developing Digital Outcomes. These strands have staged progress outcomes to indicate the minimum achievement expected for students operating at different levels of the New Zealand Curriculum. The rationale for the new curriculum content is to ensure that all students up to Year 10 experience sufficient opportunities to develop their understanding and capabilities about digital technologies. The intent is they become designers and not just users of technology and equip them for their future.

In Term 3, 2018, ERO undertook an evaluation to provide a high-level overview of schools’ awareness of and readiness to implement the new content. This looked at 10 percent of schools through a random-sample survey (see ERO’s report It’s Early Days for the New Digital Technologies Curriculum Content). Since 2018, the Ministry of Education (the Ministry) has provided a number of initiatives to raise awareness of change and to support schools to implement the new curriculum content.

At the time this survey was undertaken in 2018 only 7 percent of schools felt they had sufficient knowledge and skills to start implementation. While most leaders (95 percent) felt their teachers would be somewhat confident to start working with the content by January 2020, there were several barriers to progression. Some of these challenges were a result of difficulty navigating online resources, and the delayed provision of appropriate support programmes from the Ministry. The Ministry’s package of professional support covers a range of needs. The professional development (PD) options include:

  • Digital fluency(Digital fluency PD, available since 2017, is not specific to the implementation of the DT curriculum content)
  • Kia Takitū ā-Matihiko | Digital Readiness Programme
  • Tailored DT and Hangarau Matihiko (HM) professional learning and support

Schools making most headway were already working with their staff to familiarise them with the requirements, sharing readings, accessing external PD from a variety of sources, and often had someone who ‘championed’ the DT curriculum content in the school.
ERO’s formative evaluation was to inform the Ministry of strengths and areas for improvement in their support processes. Findings were shared with the Ministry in December 2018 and they have used these to action improvements this year.

In Term 1, 2019, ERO undertook case studies of six schools from the initial survey to unpack how those schools were preparing to implement the new curriculum content.

Evaluation Approach

The overall evaluation used a theory of change (see Appendix) that visualised the basis of ERO’s evaluation approach. The theory of change lays out the three stages of awareness, acquisition and application of the new content by schools. While acknowledging that the theory of change presents a linear model of change, it was important to assess how implementation progressed in practice for schools. ERO’s 2018 DT curriculum content survey tested whether the assumptions made at the time of developing the theory of change hold up. Schools also identified what helped or hindered their progress to implementation.

Within this theory-based approach, the evaluation sought to understand how some schools were responding to the DT curriculum content implementation, and the changes that were occurring as a result of such implementation.

Following ERO’s 2018 survey, it was apparent schools were at different points in their journey towards implementation readiness. In this subsequent case study evaluation, ERO’s focus drilled deeper into questions around the enablers and barriers to awareness, foundational knowledge and early implementation for schools during their journey.

The evaluation explored the following key evaluation questions (KEQs) and evaluative criteria:

  • How were schools moving towards 2020 DT curriculum content implementation?
  • What made DT curriculum content implementation work well in schools? What was working well (or not) for schools? (Effectiveness and impact)
  • What were the barriers and enablers that made the difference between successful and challenging implementation and results? (Relevance)

A cascading set of sub-evaluation questions and prompts include:

Was awareness and support for knowledge acquisition facilitated? If so, how?

  • What explains varying levels of school readiness and understanding of DT curriculum content? (ERO’s DT curriculum content 2018 survey showed that having many teachers aware is not the same as school readiness.)
    • Did leaders, champions, in-school prioritisation and/or expectations explain this?
  • What might explain varying levels of awareness and engagement among teachers?
    • Was there clarity in understanding the difference between digital fluency and the DT curriculum content?
    • How was DT curriculum content viewed within the larger technology learning area?
    • Were they aware of content, pedagogical content knowledge, progress outcomes (POs), links within technology and links across the curriculum?
  • In what ways did needs identification for teachers vary and why?
  • To what extent did contextual factors help or hinder implementation?

Was the implementation aligned to ensure that the knowledge gained was applied in practice? If so, how and in what ways?

  • What were teachers finding most useful when applying the new content knowledge?
  • Pedagogy: how were the key competencies used to support learning in this area?
  • How did teachers apply DT curriculum content in their teaching?
  • How were schools integrating DT curriculum content into their curriculum?

Have collaboration and networks helped in knowledge acquisition and application? If so, how?

  • What opportunities did teachers have to share ideas, work and interact with others to gather information, and consider a range of approaches?
  • Were there formal or informal networks that teachers tap into and how did that work – for whom, under what circumstances and why?

Case Selection

ERO chose a purposive sample of schools for the case studies. The sample exemplifies the different points that schools were at in their journey to implement DT curriculum content. ERO used the DT curriculum content 2018 survey results to place the sample schools on a continuum of implementation.

The key criteria for case selection was schools’ levels of awareness, engagement and application of the knowledge of DT curriculum content in their teaching practice. The Ministry did not establish any explicit success criteria at the start of the implementation about what constitutes successful implementation from a school perspective. However, two short-term outcomes identified by the Ministry were that:

  • schools and kura have integrated the Digital Technologies (DT) or Hangarau Matihiko (HM) components of the technology or Hangarau learning areas in their local curriculum
  • teachers and kaiako know how to teach the new content.

Both of these outcomes were enmeshed into the theory of change, which identifies short-term implementation outcomes.

ERO’s six case studies focused on different points in the journey – some schools were early adopters who had progressed further than others still starting out. We considered other factors for selection such as school type, location, roll size and the presence of curriculum champions, where relevant, to ensure a variety of contexts. In the ERO DT curriculum content 2018 survey, the presence of curriculum champions was shown to be strongly related to schools with higher awareness and engagement.

On your marks… | …get set… | …go!

This case study phase drilled deeper into questions around the enablers and barriers to awareness, foundational knowledge and early implementation for schools. The case studies exemplified a range of initial and current progress journeys that schools were on, drawing on the key themes identified in the baseline survey. ERO determined the pre-conditions, if any, that led to successful journeys, and the characteristics of the schools which had progressed further into their journey of implementing the DT curriculum content.

ERO identified stages of readiness similar to a three-command start when racing: on your marks (get on your lane/spot), get set (get to starting position), go! (take off). In each state of readiness, we had two schools – a contributing and a composite/secondary school. These were mostly in urban locations, barring one secondary-urban area school. For each school’s most recent ERO review, four were well placed, and two were very well placed.