Annex A: Assessment criteria and level definitions

Overview

This annex sets out the generic criteria for assessing submissions and the definitions of the starred levels in the overall quality profiles and each of the sub-profiles (for CKU, E&I and SPRE).

The expert panels will use their professional judgement to form each quality profile to be awarded to each submission, taking into account all the evidence presented at unit level for the sub-panels, and at institutional level for the IL SPRE Panel. The primary outcome of the assessment will be an overall quality profile awarded to each submission, incorporating each sub-profile from the unit-level assessment and the sub-profile from the institutional-level SPRE assessment. The overall profile will show the proportion of the submission that meets each starred level in the profile.

The Panel criteria explains in more detail how the expert panels will apply the assessment criteria and interpret the level definitions in developing the sub-profiles.

The method for combining the sub-profiles into the overall quality profile is explained in Annex A: Quality profiles in the Guidance.

The definitions of the starred levels in the overall quality profile are:

• four star: quality that is world-leading in terms of originality, significance and rigour
• three star: quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence
• two star: quality that is recognised internationally in terms of originality, significance and rigour
• one star: quality that is recognised nationally in terms of originality, significance and rigour
• unclassified: quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment
• ineligible: an output or an impact case study that is not eligible for submission to the exercise

For each of the three elements of the assessment at unit level – CKU, E&I, SPRE – sub-panels will develop a sub-profile, showing the proportion of the submission that meets each of the four starred quality levels. For the institutional-level SPRE assessment, the IL SPRE Panel will develop an IL SPRE sub-profile showing the proportion of the submission that meets each of the four starred quality levels. The assessment criteria and the definitions of the starred levels for the sub-profiles are set out below.

CKU sub-profile: criteria and definitions of starred levels

The criteria for assessing the quality of outputs are ‘originality, significance and rigour’:

• four star: quality that is world-leading in terms of originality, significance and rigour
• three star: quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence
• two star: quality that is recognised internationally in terms of originality, significance and rigour
• one star: quality that is recognised nationally in terms of originality, significance and rigour
• unclassified: quality that falls below the standard of nationally recognised work
• ineligible: an output that is not eligible for submission to the exercise

E&I sub-profile: criteria and definitions of starred levels

The criteria for assessing ICS are ‘reach’ and ‘significance’. In assessing the impact described within a case study, the panel will form an overall view about its ‘reach and significance’ taken as a whole, rather than assess ‘reach and significance’ separately:

• four star: outstanding impacts in terms of their reach and significance
• three star: very considerable impacts in terms of their reach and significance
• two star: considerable impacts in terms of their reach and significance
• one star: recognised but modest impacts in terms of their reach and significance
• unclassified: the impacts are of little or no reach and significance
• ineligible: an impact case study that is not eligible for submission to the exercise

SPRE sub-profile: criteria and definitions of starred levels

SPRE will be assessed in terms of ‘vitality and sustainability’ at both institutional and unit level. Panels will consider both the ‘vitality and sustainability’ of the submitted unit (for the UL assessment) or the institution overall (for the IL assessment), including the approach to Engagement and Impact, and the contribution to the ‘vitality and sustainability’ of the wider research base:

• four star: an approach to Strategy, People and Research Environment that is conducive to producing research of world-leading quality and enabling outstanding engagement and impact, in terms of its vitality and sustainability
• three star: an approach to Strategy, People and Research Environment that is conducive to producing research of internationally excellent quality and enabling very considerable engagement and impact, in terms of its vitality and sustainability
• two star: an approach to Strategy, People and Research Environment that is conducive to producing research of internationally recognised quality and enabling considerable engagement and impact, in terms of its vitality and sustainability
• one star: an approach to Strategy, People and Research Environment that is conducive to producing research of nationally recognised quality and enabling recognised but modest engagement and impact, in terms of its vitality and sustainability
• unclassified: an approach to Strategy, People and Research Environment that is not conducive to producing research of nationally recognised quality or enabling engagement and impact of reach and significance

Notes on the criteria and definitions of the starred levels

‘World-leading’ quality denotes an absolute standard of quality in each unit of assessment. ‘World-leading’, ‘internationally’ and ‘nationally’ in this context refer to quality standards. They do not refer to the nature or geographical scope of particular subjects, nor to where the research is carried out nor its place of dissemination. For example, research which is focused within one part of the UK might be of ‘world-leading’ standard. Equally, work with an international focus might not be of ‘world-leading’, ‘internationally excellent’ or ‘internationally recognised’ standard.

The criterion of ‘reach’ for ICS does not refer specifically to a geographic scale. Sub-panels will consider a number of dimensions to the ‘reach’ of impacts as appropriate to the nature of the research and its impacts. For example, an impact located within one region of the UK might be judged as ‘outstanding’ (graded as four star). Equally, an impact with international reach might not be judged as ‘outstanding’, ‘very considerable’ or ‘considerable’.

The CKU sub-profile for a submission where none of the submitted research outputs meet the one-star threshold will be 100 per cent unclassified.

The CKU sub-profile for a submission where none of the submitted research outputs are eligible for submission to the exercise, or where none of the submitted research outputs contain work that meets the definition of research for REF 2029 will not be awarded a quality profile.