Key changes for September 2023

Overview

We have made some changes to the way we show learner progress in personalised assessments reports. This helps support a better understanding of progression under Curriculum for Wales.

We have incorporated 'learning time’ into the progress and age-standardised scores. This will now show learners’ progression through each school year, as well as between school years.

We have also re-standardised all personalised assessment subjects. We have re-evaluated how the national cohort is performing based on the most recent data.

This means that you will see some changes in reports for assessments from September 2023. This will not affect assessments taken up to the end of the 2022 to 2023 school year.

The detailed information on learners’ skills should remain the prime focus when you plan for learner progression. Age-standardised scores and progress scores should not be used in isolation.

Changes from September 2023

The addition of learning time

From September 2023, progress and age-standardised scores for assessments will take account of ‘learning time’. This will support a better understanding of progression under Curriculum for Wales.

This means that learners’ scores adjust relative to:

the date they complete an assessment
the typical development observed over the course of the school year

Re-standardisation of personal assessments

We have re-standardised the personalised assessments.

This means that the reference cohort used to calculate the progress and age-standardised scores has been updated.

We will use the 2022 to 2023 cohort of learners as the new reference cohort from now on. Scores for assessments from September 2023 will be calculated in line with the re-standardisation.

Reports

We have updated the progress curves on reports to reflect the re-standardisation. These changes are relevant for all assessments taken from September 2023.

Benefits of the changes for schools and learners

The inclusion of learning time and the re-standardisation will:

provide schools with the most reliable, up-to-date information to support learner progression planning
make it easier to identify whether learners are making consistent progress
make it easier for practitioners to compare the progress of learners who take assessments at different times of the year
provide progress curves that reflect the most recent data available
provide better distribution of progress and age-standardised scores on group reports. This will make it easier for teachers to distinguish individual learners’ attainment
follow international best practice in assessment which is to update reference cohorts on a regular basis to reflect the change in the spread of learner attainment over time
bring all assessment subjects in line using the same reference cohort (our previous reference cohorts were a mix of pre/post-COVID)

Rationale for re-standardising in summer 2023

The personalised assessments were introduced incrementally over a period of 4 years. The progress and age-standardised scores for each subject were generated:

based on different cohorts
at different times

The standardisation for Numeracy (Procedural) took place in 2019 before the COVID pandemic. The standardisation for Reading and Numeracy (Reasoning) used data post-COVID from the 2020 to 2021 school year.

The national level of attainment for all subjects is likely to have changed since the previous standardisations. The re-standardisation provides an opportunity to account for these changes. It will present the most accurate picture for all 4 subjects.

Changes in reports

Assessments taken from September 2023

Learners will receive progress and age-standardised scores from the re-standardisation, referenced against the spread of scores of learners across Wales in the new reference year.

We have changed the shapes of the progress curves in the reports for each subject to show the spread of attainment in the new reference year, and the typical development of learners over the course of a school year.

Inclusion of ‘learning time’

Progress and age-standardised scores will now account for learning time. This means:

learners who maintain the same or similar scores in their assessments over time are making consistent progress
learners need to show higher attainment in absolute terms later in the school year to achieve the same score they received earlier in the year
the points for all previous and new assessments will be shown on each new progress chart

Reports up to and including 2022 to 2023

Historical data, up to and including school year 2022 to 2023 will stay the same.

There will be no changes to scores for assessments already taken. They will remain plotted on all individual and group progress reports.

From September 2023, the shape of the progress curve will be based on the new standardisation. Therefore the position of the assessment points on the chart may appear slightly different to previous reports.

How to compare progress for this year with previous years

In this transition year, we will include ‘learning time’ for the first time. In assessments where the learner demonstrates similar attainment:

those taken early in the 2023 to 2024 school year will tend to result in higher scores relative to autumn 2022
assessments taken later on in the school year will tend to result in lower scores than users might expect relative to summer 2023

When comparing attainment and reports it’s important to remember that from September 2023 onwards:

progress and age-standardised scores account for ‘learning time’
the progress curves in the chart on learner progress and group progress reports have been updated to reflect the most recent data

When comparing these curves to the curves on previous years’ reports, you will find that this year’s curve will be:

slightly higher at the start of the year
slightly lower at the end of the year

This represents the most recent progress data.

Glossary

Standardisation

Standardisation is the process of working out progress and age-standardised scores.

These scores provide an easy reference for where learners’ attainment sits relative to others in their cohort. For example, learners with progress scores of 1000 have demonstrated attainment at the national average.

We compare learners’ attainment to the attainment of the ‘reference cohort’ when the standardisation took place. From September 2023, the reference cohort for all subjects is made up of learners who took the assessments in the 2022 to 2023 school year.

Over time, the spread of learner attainment on any assessment changes. It is good practice in the delivery of assessments to re-standardise every few years.

This ensures that scores continue to accurately reflect where a learner sits within their cohort. A re-standardisation re-evaluates the level of attainment for each progress and age-standardised score. It re-assesses the attainment that the average learner (and learners at all other levels) is demonstrating.

We calculate the progress score by comparing a learner’s outcome with the outcomes for other learners in their year group across Wales.

The score range is 950 to 1050 and the average is 1000. This score is provided at the bottom of all group reports accessed by schools. It is used to track progress on the curve in the chart on the individual learner and group progress reports.

We work out the age-standardised score by comparing a learner’s assessment score with the scores of other pupils in Wales who were born in the same year and month. The score range is 70 to 130 and the average is 100. The age-standardised score from the learner’s most recent assessment is shown on their progress report.

Learning time

Learning time refers to the point in the school year at which a learner completes an assessment. It allows for the progress a learner makes within the year as well as between subsequent years.

Learners taking an assessment early in the school year would usually do less well than if they took an assessment towards the end of the school year. This is because of the benefit of an additional year of learning.

In the new approach, the progress and age-standardised score will take account of learning time. This means learners have their score adjusted relative to:

the date they complete the assessment and
the typical development observed in the reference cohort over the course of the school year

The scores have been updated to account for learning time in order to:

support a better understanding of progression under Curriculum for Wales
provide schools with improved information to support learner progression and make it easier to see whether learners are making consistent progress
make it easier for teachers to compare the progress of learners who take assessments at different times of the year

Following these changes, learners who maintain the same or similar scores in their assessments over time are making consistent progress.