Academic wellbeing and attitudes to learning of Key Stage 2 pupils following the pandemic

22nd November 2022 by Timo Hannay [link]

In a continuation of our collaboration with Hodder Education and the Nuffield Foundation – this time also with Nottingham Trent University – we have been looking at academic wellbeing and attitudes among children in Key Stage 2 (ages 7-11) in the wake of the pandemic.

The resulting white paper is out today. Key findings include:

• Academic wellbeing has dropped across all dimensions (positivity, motivation, resilience, and self-efficacy) since the pre-pandemic period
• Pupils in Year 3 have seen the largest reductions in wellbeing
• The majority of children now feel some vulnerability in self-efficacy
• Across all year groups, girls feel more positive than boys
• Across all year groups, boys reported feeling greater self-efficacy than girls
• Pupils in the North of England gave more satisfactory responses across all dimensions of academic wellbeing

To find out more, we encourage you to read the full report.

As always, we welcome your feedback and suggestions: [email protected]

Omicron, Eunice and Heatmageddon: Oak National Academy usage during 2021-2022

7th November 2022 by Timo Hannay [link]

Update 10th November 2022: We have slightly amended some of the wording above Figure 7 to refer to 'lessons' rather than 'pupils'. This more accurately reflects the fact that video lessons may be used by teachers as well as pupils, especially outside lockdown.

One of many remarkable outcomes of the COVID-19 pandemic was the emergence in April 2020 of Oak National Academy, a free website providing video lessons and online quizzes for schoolchildren of all ages, especially those studying at home under the English national curriculum. Although the worst of COVID-19 now seems to be behind us (probably) and schools were more or less fully open throughout the last academic year, Oak continues to pursue and expand its mission (though not without some pushback from private-sector educational suppliers). To this end, it has now been reconstituted as an independent public body under the auspices of the Department for Education (DfE).

Since 2021, SchoolDash has been lucky enough to work with Oak and make use of its rich data to look at online learning activity during lockdown, as well as more general educational questions, such as learning effectiveness by subject and age.

This post looks at Oak usage data during the 2021-2022 academic year (ie, autumn 2021 to summer 2022) in order to better understand online learning in the post-pandemic world. Most of the results come from a study commissioned by Oak to support their internal decision-making and public impact evaluations, but the views expressed here are those of SchoolDash alone.

In summary, we find that:

• While Oak received less use than during the early-2021 lockdown, it remained popular by almost any standard. Furthermore, usage tended to increase markedly during periods of disruption to schools, such as the COVID-19 omicron wave, Storm Eunice and the July 2022 heatwave.
• Coverage of secondary schools (74%) was much higher than primary schools (37%). It also tended to be higher among state schools and urban schools, though this is at least in part because such schools tend to be larger.
• Allowing for their different pupil populations, teacher and pupil usage tended to be higher in poorer areas.
• Pupils in poorer localities were slightly less likely to watch video lessons all the way through and were also less likely to use a computer rather than a mobile phone or tablet device.
• The type of device being used had a large effect on median session duration. Those on mobile phones were only about 25% as long as those on computers or tablet devices. There were no obvious differences between pupils in poorer or more affluent areas when they were using similar devices.
• Across England, Google Classroom appears to have been a more popular online teaching environment than Microsoft Teams, particularly in the south, but with exceptions in some parts of the country.

Continuing online education

Figure 1 shows daily usage statistics for Oak during the 2021-2022 academic year (up to and including June 2022). There was a clear weekly cycle, with the highest levels of activity usually occurring early in the week and the lowest levels at weekends. There were also dips during school holidays, especially at Christmas and at Easter.

We previously found that during lockdown Oak sometimes delivered over 10,000 teacher downloads a day. While it only rarely exceeded that threshold in the last academic year, it regularly displayed usage of about half that level, especially during 2022. This suggests that teachers are still making heavy use of Oak in preparing for in-school lessons.

By contrast, teacher shares (ie, when a teacher shares an Oak lesson with their pupils) were typically a few hundred each day – with the notable exceptions of some big spikes at the beginning of the spring and summer terms. This was much lower than during lockdown, when teacher shares were regularly in the thousands – presumably a reflection of the fact that most teaching was being held face to face. (You can also see separate data for sharing to Google Classroom and Microsoft Teams, a distinction we will return to below.)

Similarly, lesson starts by pupils were 'only' in the tens of thousands each day compared to the millions seen during the early-2021 lockdown. Note also the surge in lesson starts during mid-January 2022, which coincided with the COVID-19 omicron wave, and the large spike on 18th February 2022, which was the day that Storm Eunice hit the UK.

(Use the menu below to view different activity metrics. Hover over the lines to see corresponding data values.)

Figure 2 shows further detail for the period around the COVID-19 omicron wave. Teacher downloads and pupil lesson starts (blue lines) both peaked at the same time as DfE-reported COVID-related absences at schools (red columns). For statistically minded readers, the linear correlation coefficient between COVID absences and lesson starts is 0.83.

(Use the menu below to view different activity metrics. Hover over the lines and columns to see corresponding values.)

Figure 3 shows later usage data from July 2022. This coincided with an unprecedented heatwave in the UK, which peaked on 18th and 19th July. During this period, many pupils remained at home and some schools switched to online-only teaching to prevent pupils and staff from having to commute. While teacher downloads showed a normal late-summer pattern of gradual decline, pupil lesson starts increased by about 50% compared to the week before. Taken together with the findings for the COVID-19 omicron wave and Storm Eunice, this suggests that Oak remained a popular resource during acute educational disruptions.

(Use the menu below to view different activity metrics. Hover over the lines to see corresponding data values.)

Teacher cover

So much for overall usage, what kinds of schools use Oak? This is something we can examine courtesy of the fact that teachers downloading or sharing an Oak resource are asked to pick their school from a list. Figure 4 shows the proportions of schools of various kinds that made at least some use of Oak during the 2021-2022 academic year (specifically, 1st September 2021 to 30th June 2021).

Across all schools, this was about 45%, though coverage was much higher among secondary schools (74%) than primary schools (37%). Looking across both phases, Oak showed highest coverage among state schools, academies, single-academy trusts, urban schools, especially in London, and those with higher levels of disadvantage or English as an Additional Language (EAL). By contrast, there were no clear trends by Ofsted rating or faith status.

Note that there are some differences by phase. For example, unlike primary schools, secondary schools tend to show higher coverage among MATs and schools with higher Ofsted ratings. They also show a bigger disparity between state and independent schools.

However, we need to take care when interpreting these results. Large schools are more likely to have used Oak simply because they have more teachers, so the probability of at least one of them having accessed an Oak resource is higher. Since secondary schools, state schools and urban schools all tend to be larger, this effect surely explains at least some of results seen. In the analyses that follow below, we will control for this by looking at activity levels relative to the size of the pupil population in each group of schools.

(Use the menus below to explore other school phases and types. Hover over the columns to see underlying values and sample sizes.)

The sharing type

Figure 5 shows activity in the form of 'teacher shares per 1,000 pupils' (to allow for the fact that some groups of schools are bigger than others). Once again, we see that use among secondary schools tended to be much higher than among primary schools.

Focusing on secondary schools, the highest relative levels of teacher activity tended to be among state schools, academies, large MATs and smaller, more rural schools. Other trends were more ambiguous.

(Use the menus below to explore other school categorisations and phases. Hover over the columns to see underlying values and sample sizes.)

Active children

Pupil activity on Oak is anonymous, so we cannot analyse it by school or even school type. But we do know the rough locations – in the form of top-level postcodes – from which pupils (and teachers) are accessing the service. This allows us to look at usage by region and local deprivation level, as shown in Figure 6.

Note that as for Figure 5 above, we are controlling for pupil population, but in this case it's only approximate because there's no guarantee that pupils attend a school in the same postcode as the one in which they live. However, at this national level we expect any local inconsistencies to more or less even out. To put it another way, pupils accessing Oak from a particular region or deprivation quintile will generally be attending a school in the same region and a similar deprivation quintile.

Looking at trends by region, London is most active in terms of teacher downloads and pupil lesson starts, while the West Midlands pips it in terms of teacher shares. The trends by local deprivation are more clearcut: across teacher downloads, teacher shares and pupil lesson starts, relative activity tends to be higher in poorer areas.

(Use the menus below to explore other school categorisations and phases. Hover over the columns to see underlying values and sample sizes.)

Seeing it through

Another way of assessing lesson engagement is shown in Figure 7. Across all lessons, just over 70% of videos were watched to at least the halfway point and just under 60% were watched to the end. Note that the way in which video lessons are used seems to have changed somewhat since the pandemic. Although they are still used to provide remote learning for absent pupils, portions of videos are now also used to reinforce class learning or to serve as revision aids. As a result, partial video views are not necessarily an indication that the material is not being used as intended.

That said, differences across regions showed no clear pattern, but there was a consistent, if modest, decline with increasing deprivation.

(Use the menus below to switch between these metrics and categorisations. Hover over the columns to see underlying values and sample sizes.)

Technology matters

Another potential source of disparity between different local areas, schools and individuals is access to technology. In particular, do children have ready access to the kind of device that makes online lessons engaging, or at least bearable? Figure 8 shows the proportions of Oak lesson starts by device type. Overall, around 70%-90% of lessons were started on a computer, and this proportion tended to rise over time. In contrast, mobile phones accounted for only 10%-20% of lesson starts, and tablet devices for well under 10%. Furthermore, both of those proportions tended to fall over time. Phones became more common relative to computers at weekends, while use of tablet devices appeared to increase during school holidays, especially at Christmas.

Figure 9 shows how computer usage varied by local deprivation level. In general, more affluent areas showed higher proportions of lesson started on a computer, though the difference between the most affluent quintile (83%) and the most deprived one (79%) was relatively modest.

Figure 10 shows median session durations by device type. These are are normalised so that the average value for computer users is 100. Sessions on computers and tablets were similar in duration, while those on mobile phones it were typically only about a quarter as long. Session lengths tended to rise towards the end of the school year and were more volatile for tablets than for other devices, probably because the sample sizes for these were smaller.

Figure 11 shows median session durations by device type and local deprivation level. Within each device type, there was no obvious trend by deprivation. This suggests that all children, regardless of local deprivation levels, engaged similarly with Oak given access to similar devices.

Finally, Figure 12 shows the proportions of teacher shares that were made using Google Classroom and Microsoft Teams (by far the common commercial platforms on which teachers share Oak lessons with pupils). Across England, Google was more heavily used, especially in the south, but Microsoft is slightly more popular in the East Midlands and Yorkshire and The Humber. This is similar to the situation we saw in early 2021, though if anything, the north-south differences between Microsoft and Google seem to have become even starker since then.

Oak was born of lockdown, but even a year or more after the last widespread school closures in England it remains a popular service. As well as delivering a free resource for schools and pupils, the data it generates provide a window into ongoing educational activities and disparities.

Special thanks to our collaborators at Oak National Academy for supporting this analysis and allowing us to publish the results. As always, we welcome your feedback: [email protected].