School science technician recruiting under lockdown

20th July 2020 by Timo Hannay [link]

Throughout the COVID-19 lockdown, we have been working closely with the Gatsby Foundation and Teacher Tapp to track and analyse the resulting disruption to teacher recruitment. But teachers are not the only people whom schools need to hire: support staff play critical roles too. For this reason, we also monitor hiring of technicians – particularly important in science. As for our teacher recruitment analyses, what follows is based in information gathered from the websites of secondary schools, sixth-form colleges and further-education colleges in England¹. This work, too, has been generously supported by the Gatsby Foundation.

Our main findings relating to school science technicians are:

• Recruitment activity in 2019-2020 was very similar to previous years until mid-March 2020, when it slowed abruptly. By mid-July, the cumulative deficit across the whole academic year compared to 12 months earlier was 400 advertisements.
• During the lockdown period from mid-March to mid-July alone, there were 350 fewer advertisements (-55% compared to the same period a year earlier).
• This decline was broadly based: every region and almost all school types showed year-on-year declines. However, reductions in activity tended to be greater among large schools and those with high proportions of disadvantaged pupils.
• There were similar trends across other technician disciplines, but science, as the biggest category, showed by far the largest decline in absolute terms.

Technical interruption

Figure 1 compares the numbers of school technician advertisements in three consecutive academic years. The seasonal cycles in weekly numbers during in 2017-2018 (blue line) and 2018-2019 (green line) were very similar, with peaks in September, May and – especially – June. (Note that this is somewhat different to the seasonality for teachers.)

From September to early March, the pattern for 2019-2020 (red line) was broadly similar, but since mid-March activity has been unusually low. This is most easily seen by looking at cumulative numbers. Between 2017-2018 and 2018-2019, there was an overall increase of 260 technician ads (+8%). During 2019-2020, activity was initially in line with previous years, but fell away from mid-March, and by mid-July the cumulative deficit compared to the same period a year earlier was nearly 800 advertisement/s (-25%). Among science technicians alone, the cumulative deficit was 400 adverts (-29%) compared to a year earlier.

(Use the menus below to view weekly or cumulative data, and to select different subject areas. Click on the figure legend to hide or view individual academic years.)

Figure 2 shows the year-on-year changes between 2018-2019 and 2019-2020. Daily differences show wide day-to-day swings (blue line), which can be smoothed by calculating a seven-day moving average (red line). These show a dip in September 2019, followed by a broadly flat period during the rest of the autumn and winter. However, between early March and mid-July 2020 – coinciding with the COVID-19 lockdown and peak technician recruiting season – there was a large decline. (The brief recovery in mid-April is an artefact caused by the different timing of Easter.) Consistent with what we saw in Figure 1, the cumulative the shortfall in number of technician adverts across all subjects is almost 800, and in science alone is 400, with around 350 of these 'missing' from the period between March and July.

(Use the menus below to view daily or cumulative, and to select different subject areas. Click on the figure legend to hide or view individual data sets.)

Figure 3 shows year-on-year changes in school technician recruiting activity for different subjects during the periods before (blue columns) and after lockdown (red columns)². In percentage terms, advertisement numbers declined by 51% across all subjects. Music and Drama showed the largest percentage decline during lockdown (-73%), followed by Science (-55%). However, Science showed by far the largest fall in terms of absolute numbers of advertisements (-350). On this measure, Art and Design Technology came second, with almost 180 fewer advertisements between mid-March and mid-July.

(Use the menu below to switch between percentages and numbers of advertisements. Hover over the columns to see corresponding data values.)

Figure 4 examines changes by school type and location. Looking at regional differences across all subjects, every part of the country showed lower activity during March-July compared to a year earlier, and all except the East Midlands showed a bigger decline than in the period from September to March.

Note that the 'Multi-school groups' category contains jobs posted by schools that recruit collectively, usually as part of a multi-academy trust. Because these positions are not assignable to an individual school, they are treated separately. This category grew strongly (up 26% year on year) in the period from September to March, mainly because an increasing number of schools entered this group. But even here, activity fell in the period from March to July compared to the same period a year earlier (-18%).

There are hints of patterns by school type, though they are somewhat inconclusive. For example, schools with larger numbers of disadvantaged pupils showed bigger declines, but those in poorer areas didn't and, if anything, the pattern for low-attaining pupils seems to run the other way. Large schools showed bigger declines than small ones, but so did rural schools, which tend to be smaller (and, be warned, represent a very small sample size). Local authority-maintained schools showed larger declines than academies and (especially) free schools. There is no clear trend by Ofsted rating.

Science technician adverts show similar patterns to those described above to all subjects, which is unsurprising since they represent by far the largest subject group.

(Use the menus below to explore different school groups and subjects. Click on the figure legend to hide or display time periods. Hover over the columns to see corresponding data values and numbers of advertisements.)

All in all

The reductions in technician recruitment described here are, if anything, even greater than those that we have seen for teacher recruitment. In that case, we were able to use Teacher Tapp surveys to show that the decline appears to have been caused by a combination of disruption to recruitment activities (which is very bad news for schools) and a fall in teacher turnover (which is somewhat less problematic and might even be good news for some schools). We are not able to distinguish between those possibilites here, but it seems reasonable to assume that a combination of these two effects is also at play in technician recruitment. If so then a degree of understaffing in September seems likely.

Much attention is rightly being paid to the immediate academic and psychological effects of the lockdown on schoolchildren. But this should not distract from the effects on school staffing, which are likely to reverberate into the next academic year and beyond. We will continue to monitor those trends on this blog and in our Jobs section.

Footnotes:

1. This process does not capture all vacant positions because: (a) not all positions are advertised on school websites, (b) even when they are, they are not necessarily presented in a way that can be automatically indexed, and (c) websites are sometimes unresponsive or otherwise unavailable. For this reason, the data presented should be thought of as being based not on a comprehensive list of all vacancies but on a subset. However, positions have been detected for well over 90% of schools and these are broadly representative of the overall population of schools.
2. In fact, the cutoff date – Friday 13th March 2020 – slightly precedes school lockdowns, which were announced on Wednesday 18th March and took effect from Monday 23rd March. This is because the data suggest that schools changed their behaviour slightly early, in anticipation of the lockdown, with the decline in recruiting activity beginning on Friday 13th March.

Who stands to lose from GCSE grade assignments?

10th July 2020 by Timo Hannay [link]

Update: Here is coverage of the issue, and our analysis, from Schools Week.

Following this year's cancellation of GCSEs and other public exams amid the COVID-19 pandemic, the Department for Education (DfE) has decided that students will be assigned grades through a process involving their schools, examination boards and Ofqual. In order to prevent grade inflation and ensure consistency across schools, Ofqual has decided to moderate grade assignments by taking into account students' prior attainment at primary school and the recent performance of their secondary school, but not to allow for any anticipated increases in school effectiveness compared to previous years. Consequently, grade moderation will not be influenced by anything that pupils have done during their secondary years, only by their performance at primary school and their choice of secondary school.

Yet while individual school performance tends to be broadly consistent from year to year, there are exceptions. This post analyses recent cases in which mainstream, non-selective state secondary schools in England have shown large year-on-year improvements in performance. In other words, exactly the kind of effect that Ofqual proposes to ignore in their 2020 grade moderation process.

This analysis was commissioned by the Academies Enterprise Trust (AET), a large multi-academy trust that operates nearly 60 primary and secondary schools across England, and the results are published here with their kind permission. AET were concerned about the implications of Ofqual's policy for rapidly improving schools, including some of their own, and wanted to understand the likely scale of the problem. However, the interpretations and views presented here are those of SchoolDash.

We conclude that:

• Large year-on-year increases in school performance are relatively rare but consistently occur in a small proportion of schools. In 2019, 15 schools representing nearly 1,600 Year 11 students showed year-on-year improvements of 0.8 of a grade or more in their Progress 8 scores. A total of 45 schools representing just under 5,400 Year 11 pupils showed improvements of at least 0.6 of a grade.
• Most improvements of this kind do not appear to be mere statistical fluctuations. Of the 11 schools that showed improvements of at least 0.8 in 2018, all of them (100%) sustained most or all of these increases into 2019. Of the 51 that showed improvements of at least 0.6 in 2018, 40 (78%) sustained these into 2019.
• Rapidly improving schools come in a variety of types, but they are disproportionately likely to have low prior Progress 8 scores, low Ofsted ratings and to have undergone recent changes in leadership.

Progress report

We have looked at recent school performance data in order to determine how common it is for schools to show large improvements in performance from one year to the next. This analysis uses the DfE's Progress 8 measure of GCSE performance since it controls for differences in pupils' prior attainment levels, which is a major source of variation in school-level GCSE performance. Using Progress 8 therefore allows us to focus more directly on changes to the performance of the school itself rather than the pupils it attracts.

Put simply, Progress 8 indicates whether a school is achieving higher or lower GCSE grades than its pupils' performance at primary school would suggest. The national mean value is set to zero. A Progress 8 score of 1 indicates that a school is achieving an average of one grade higher in each contributing GCSE; a score of -1 indicates that it is achieving an average of one grade lower. In practice, most schools have scores well inside this range (ie, a fraction of a grade either side of zero). Note that because it is a zero-sum score, Progress 8 doesn’t capture anything about the country's absolute level of academic performance, only the relative performance of schools within it.

Progress 8 scores for any given school fluctuate from year to year, but usually in a relatively narrow range. However, each year a few schools show very large changes. Figure 1 shows data for non-selective state secondary schools in England (of which there over 3,000). In 2019, 15 of these schools showed increases in Progress 8 of at least 0.8 between 2018 and 2019, while 45 schools showed increases of at least 0.6. These represent small proportions of all schools (0.5% and 1.4%, respectively), but nevertheless account for thousands of Year 11 pupils (just under 1,600 and 5,400, respectively).

One potential reason to discount this effect is that it might be caused by random statistical fluctuations rather than genuine improvements in school performance. However, this appears not to be the case. In 2018, very similar numbers of schools showed improvements (blue columns) and for the vast majority these improvements were mostly or wholly sustained in the following year (red columns).

If anything, the proportion of schools showing sustained improvements goes up with the size of the improvement. Of the 11 schools that showed Progress 8 increases of at least 0.8 in 2018, all of them sustained most or all of this (which is to say, maintained an improvement of at least 0.4 over their 2017 score) in the following year, 2019. For the other columns shown, the proportions are about 70-80%. This suggests that the increases reflect genuine school improvements rather than statistical artefacts.

Subjective information

Progress 8 measures a range of subjects, but even within an individual school, different subjects can display different patterns of performance. Figure 2 shows data for the maths and English elements of Progress 8. Compared to the results we saw above for overall Progress 8 scores, these involve slightly larger numbers of schools, but as before, the year-on-year improvements are sustained in most of them (in the range 80-95%).

The groups of schools showing specific improvements in maths or English are not only more numerous than those showing improvements in overall Progress 8 performance, they can be different schools altogether. Figure 3 shows the maths and English elements for all 94 schools with Progress 8 improvements of at least 0.5 between 2017 and 2018. (As we saw above in Figure 1, 72, or 77%, of these sustained their improvements into 2019.)

Some of these showed improvements mainly in maths (red dots), others mainly in English (green dots), and others still showed major improvements in neither (blue dots), indicating that their Progress 8 scores were bolstered by improved grades in other subjects. Thus large year-on-year increases in Progress 8 are often driven mainly by improvements in particular subjects.

Click here to display all groups; click on the legend below to turn individual groups on or off.

A turn for the worse

There is of course a comparable – though not identical – effect at the other end of the scale, with some schools showing large year-on-year decreases in performance, as shown in Figure 4.

While similar numbers of schools and pupils are involved, the situation is not quite symmetrical. In this case the changes are less likely to be sustained, with proportions in the 60%-75% range. Also, the percentages tend to decrease for larger declines in performance, presumably in part because these are more likely to prompt intervention.

Signs of success

Schools showing large year-on-year increases in performance come in a wide variety of types, and their relatively small number makes this difficult to analyse rigorously. However, three features stand out:

• Most obviously, they tend to start with low Progress 8 scores. The pre-improvement average for the 11 schools that showed increases of at least 0.8 in 2018 was -0.90. For the 51 schools that showed improvements of at least 0.6, it was -0.62. The same is not true of schools showing rapid decline, which tend to have initial Progress 8 scores much closer to the national average or even slightly higher.
• They often have low Ofsted ratings. Of the 11 schools showing Progress 8 increases of at least 0.8 in 2018, 70% were rated 'Requires Improvement' or lower and none were rated 'Outstanding'. (The corresponding proportions for all schools were 24% and 20%, respectively.) This trend gets weaker when we include schools with smaller year-on-year changes, but even among those showing Progress 8 increases of 0.6 or more, 48% were rated 'Requires Improvement' or below – double the proportion among all schools. This trend does not apply to schools showing rapid decline, which have distributions of Ofsted ratings very similar to those for all schools.
• They are more likely to have had recent changes in leadership. Among schools showing Progress 8 increases of at least 0.8 in 2018, 82% have recorded a change in headteacher¹ since 2017 compared to 45% for all schools. The proportion for schools showing improvements of 0.6 or more is 65%. In this case, a similar pattern is evident among schools showing large declines in Progress 8 score.

Rapid, sustained year-on-year progress is a regular occurrence among England's secondary schools, albeit involving a relatively small number of them during any given year. The DfE has said that in the current unprecedented situation its "priority will be making sure no pupil is disadvantaged by the cancellation of this year’s exams" . Based on this analysis, any moderation policy that does not allow for these effects would risk disadvantaging thousands of students. At least attempting to identify which schools are likely to have shown rapid improvement this year seems like the only fair thing to do. As they say, past performance is no guarantee of future results.

Footnotes:

1. This refers to the headteacher named in the DfE's public records for the school. It can change for a variety of reasons, including alterations to management structure, such as the appointment of an executive headteacher, as well as straightforward replacements of one head with another.