Section 4 – Contributions to Knowledge and Understanding (CKU) guidance

Overview

This policy section outlines the rules that should be followed when selecting outputs for submission to REF 2029.

It provides guidance on the eligibility and selection of outputs for submission in REF 2029. It covers:

• Technical definitions of eligible outputs and diverse outputs
• The definition of substantive link
• Information about the different types and timings of relationships that demonstrate a substantive link
• Information about required, additional, and panel specific output data
• Guidance on how to assign an output to a unit of assessment (UoA)
• Information about specific types of output including: confidential outputs, outputs in languages other than English and interdisciplinary outputs
• Information about reserve lists and the circumstances when a reserve item might be submitted
• An overview of the unit reduction request process
• Information about citation data and access to outputs
• An overview of the role of the Code of Practice (CoP) in relation to the CKU policy module
• The audit process

Introduction

The Contributions to Knowledge and Understanding (CKU) element of REF 2029 captures information about each UoA’s various and diverse contributions to knowledge and understanding through the assessment of:

• a pool of submitted research outputs – the number of outputs is determined by the volume measure, multiplied by 2.5 and rounded to the nearest whole number
• elements of the disciplinary-level evidence statement (DLS)

Together, these two components are expected to account for 50% of the overall quality profile. This is subject to change depending on the outcome of the PCE pilot and this paragraph may be updated.

Full detail on the weighting of the disciplinary level evidence statement will be provided in due course.

The funding bodies welcome the submission of a very wide range of output types that are consistent with discipline-specific and interdisciplinary approaches to scholarly communication.

Key changes for REF 2029

Decoupling

REF 2029 builds on the partial decoupling of outputs from individuals introduced in REF 2021 and will further cement the move towards assessing the performance of disciplinary areas within the HEI, rather than the outputs of individual researchers. Furthermore, this will allow institutions more flexibility as decoupling fundamentally separates the contracts used to calculate the volume measure from the specific outputs included in a submission. This means that firstly, eligible research outputs do not have to be authored by individuals whose contracts contribute to an institution’s overall submission volume. Secondly, these outputs do not need to be submitted to the UoA associated with the author’s volume-contributing contract.

The outcomes of REF are crucial to the sector, as they directly inform the allocation of significant block-grant research funding to Higher Education Institutions (HEIs) and provide accountability for the utilisation of public funds. Decoupling helps to ensure that the size of a HEI’s submission remains proportionate to the scale of their research activity, whilst also giving institutions more flexibility to create an equitable and inclusive approach to capturing a diverse range of research, research communities and research excellence wherever it is found.

Recognising the paramount importance of people within the research sector, the policy aims to achieve a crucial balance. It seeks to acknowledge individuals as the essential connection between institutions and the research outputs they submit, while aligning assessment with institutional level funding allocation and providing appropriate recognition for research enabled by the institution.

No collection of staff details

HEIs will not submit individual staff information to REF 2029, other than the HESA ID demonstrating a substantive link to the HEI. The volume measure in REF 2029 will capture the information relating to staff meeting the same eligibility criteria as those submitted to REF 2021 with the data gathered directly from the Higher Education Statistics Agency (HESA), rather than a staff census list.

With the removal of a staff census in a decoupled framework, eligibility of outputs will be based on the existence of a substantive link between the HEI and the research, rather than the inclusion of an author on a staff census list.

Introduction of substantive link

Institutions must be able to demonstrate a substantive link to any submitted output. This is distinct from the substantive connection between a staff member and a UoA that was required in REF 2021.

A substantive link describes the relationship between an HEI and research, submitted in the form of an output.

HEIs may submit any output that also meets wider eligibility criteria, to which they can demonstrate a substantive link. This:

• allows a wider range of outputs to be submitted to a UoA
• ensures that only outputs where the HEI has enabled the research are submitted
• encourages HEIs to hire staff based on future work at the institution
• provides appropriate recognition for research directly enabled by the institution
• addresses concerns regarding the “import” of staff with eligible outputs during the submission period, where those outputs have not been enabled by the submitting unit

Removal of the REF 2021 minimum or maximum output requirement

In a change from REF 2021 there will also be no minimum or maximum output requirements for individual staff members.

Institutions are required to demonstrate the representativeness of their output submissions. See sub-section 7.

CKU elements of the disciplinary level evidence statements

Details of the requirements relating to the CKU elements of the disciplinary level evidence statement will be published at a later date.

New process for unit reduction requests

There will be no individual level output reduction request process for REF 2029 and a new process for unit reduction requests will be introduced for the exercise. This approach helps reduce the burden of the exercise.

Disciplinary level evidence statements

The disciplinary evidence statements will contain multiple sections which will contribute to all three quality sub-profiles of the assessment (Contributions to Knowledge and Understanding, Engagement and Impact, and People Culture and Environment).

The statement will take the form of a structured questionnaire and focus on quantitative and qualitative evidence.

There will be strict word limits, and we do not envisage long narrative components.

As outlined in the Initial Decisions additional guidance will be published separately on the content of the disciplinary level evidence statements.

Eligible outputs

HEIs must decide which outputs to select for submission, in accordance with this guidance and its REF 2029 CoP.

The number of outputs required in a submission is determined by multiplying the volume measure by 2.5 at UoA level. The result is then rounded to the nearest whole number, with .5 results rounded up. The full detail of this is in the Volume Measure guidance.

Joint and multiple submissions will continue to be supported in REF 2029, full policy and process will be published in due course.

The decoupling of individual staff members and outputs in REF 2029, and the introduction of the substantive link, is intended to increase flexibility for institutions in building the portfolio of outputs for submission.

To aid institutions in promoting equality, complying with legislation and avoiding discrimination, institutions must document and apply robust, fair and transparent processes for the selection of outputs in their REF 2029 CoP. These must demonstrate how they have taken into account equality, diversity and inclusion considerations.

There will be an opportunity in the disciplinary level evidence statement for institutions to contextualise the distribution of outputs, drawing on the processes described in the CoP.

Core requirements of eligible outputs

Each output included in the submission must:

• be the product of research, briefly defined as a process of investigation leading to new insights, effectively shared
• have first been brought into the public domain during the publication period 1 January 2021 to 31 December 2028 or, if a confidential report, lodged with the body to whom it is confidential during this same period
• have a substantive link to the HEI making the submission
• for outputs within scope of the Open Access policy, meet the requirements of that policy   

COVID-19 delayed outputs

Outputs that were delayed due to the impact of COVID-19 but submitted to REF 2021 are not eligible for submission to REF 2029, even if they meet all the other core requirements of eligible outputs.

Definition of research

The REF 2021 definition of research is unchanged. For the purposes of the REF, research is defined as a process of investigation leading to new insights, effectively shared.

This includes:

• work of direct relevance to the needs of commerce, industry, culture, society, and/or to the public and voluntary sectors
• scholarship, which is defined for the REF as the creation, development and maintenance of the intellectual infrastructure of subjects and disciplines, in forms such as dictionaries, scholarly editions, catalogues and contributions to major research databases
• the invention and generation of ideas, images, performances, artefacts including design, where these lead to new or substantially improved insights
• the use of existing knowledge in experimental development to produce new or substantially improved materials, devices, products and processes, including design and construction

Effectively shared includes research that is published, disseminated or made publicly available in the form of assessable research outputs, and confidential reports.

Exclusions from the definition of research

The definition of research excludes:

• the routine testing and routine analysis of materials, components and processes such as for the maintenance of national standards, as distinct from the development of new analytical techniques
• the development of teaching materials that do not embody original research

Exclusions

Editorships of journals and other activities associated with the dissemination of research findings should not be submitted.

Diverse outputs

Research involves a diverse set of research practices, which lead to a wide range of diverse outputs.

Diversity of output forms refers to the representation and recognition in each REF 2029 UoA and main panel of a broad range of research outputs, expressed in the most appropriate way for the research itself and its audiences and partners.

Such outputs may include innovative formats, accessible reporting formats, and modes of expression that may depart from historical conventions within specific fields and contexts. 

Examples of diverse output types include, but go well beyond, journal articles, conference papers, monographs and book chapters, to include, as appropriate to each UoA:

• non-textual (artefacts, audio, visual, multi-modal)
• creative, performance and practice-led outputs
• reviews and research synthesis
• software and code
• datasets
• translations and critical editions
• reagents
• multi-component outputs
• policy summaries and analyses
• different types of reports

A glossary of output categories and collection formats will be published.  Further guidance on types of outputs will be published later in the Autumn in an update to REF 2021 Guidance on Submissions Annex K, which offers a good indication of the potential range of outputs.

The assessment of diverse outputs will recognise rigorous, significant and original research across multiple purposes, designs, methodologies, analytical approaches and modes of communication.  

An underpinning principle of the REF is that all forms of research output and research practice will be assessed on a fair and equal basis. Sub-panels will not regard any particular form of output as of greater or lesser quality than another per se, and all subject matter will be assessed equally, without bias.  

Institutions that wish to submit outputs produced in languages other than English are welcome to do so. Such outputs will be assessed equitably, as described in sub-section 16.

Reviews, textbooks or edited works (including editions of texts and translations) may be included if they embody research as defined in paragraph 5.3.

Significant material in common

Where two or more research outputs in a submission include significant material in common (for example, a journal article that also appears as a chapter in a book) the sub-panels will assess each output taking account of the common material only once.

Where a sub-panel judges they do not contain sufficiently distinct material and should be treated as a single output, an unclassified score would be given to the ‘missing’ output.

Substantive link

Definition of a substantive link

A substantive link describes the relationship between an HEI and research, submitted in the form of an output.

The HEI must have enabled the research leading to an output that is first made publicly available during the output eligibility period.

A substantive link must be evidenced through an eligible employment relationship with an author (or equivalent) who has made a significant research contribution to the output.

Output eligibility period

Outputs must be first made publicly available between 1 January 2021 and 31 December 2028.

Where the final form of an output was delayed due to COVID-19 beyond 31 December 2020 and was subsequently published by 31 March 2021 and submitted to REF 2021, that output cannot be resubmitted to REF 2029.

Where the date of imprint on a publication lies outside the publication period but the actual date of appearance is within the publication period, evidence of the actual date on which it became publicly available will be required for data verification purposes, such as a letter from the publisher.

Eligible employment relationships

Eligible employment relationships are defined as:

• A minimum of 0.2 FTE for at least 12 months continuous employment (as defined by Section 210 of the Employment Rights Act 1996 (ERA 1996) or the Employment Rights (Northern Ireland) Order 1996) by the HEI making the submission. Continuous employment may be demonstrated by a sequence of consecutive contracts AND
• The role descriptor includes an explicit expectation of research activity (as distinct from scholarship) within the role.

Eligible employment relationships and the point outputs are made publicly available

An eligible employment relationship must have occurred, either:

At the point at which the submitted output was first made publicly available,

• An output may first be made publicly available at any point during the eligible employment relationship OR

Before the submitted output was first made publicly available, when the research resulting in the output was carried out.

• For most outputs this will be January 2021 to December 2028
• This period may be extended prior to January 2021 for longform and other diverse outputs. This will be finalised by the panels during the criteria setting phase
• Such cases must be evidenced through a limited set of indicators (see paragraph 6.6.2.)

Eligible employment relationships that occur after the point at which an output was first made publicly available cannot constitute a substantive link.

Time limited period

The time limited period prior to the date of the output first being made publicly available, during which research resulting in the output was carried out, will be defined by panels during the criteria setting period and updated here. It is anticipated that it will be two years for the majority of outputs and up to five years for longform and other diverse outputs.

Set of indicators to evidence substantive link

The main panels will define the indicators that will be requested to evidence a substantive link where it occurs before the submitted output was first made publicly available, and they will be updated here.

Likely indicators include, but are not limited to:

• evidence of internal research support (for example, funding for research materials, technical or research support, conference attendance)
• evidence of work in progress presentations (internally and externally)
• evidence of an external grant to support a relevant program of research

Exclusions

Outputs sole-authored by, or where the only substantive link to the institutions is through the following groups, will not be eligible for submission:

• postgraduate research students (including PhD theses)
• visiting or honorary staff
• staff on teaching only contracts (that is, those returned to the HESA Staff record as ACEMPFUN.01)
• individuals employed on academic or other contracts with no explicit expectation of research within the job role, technical teaching or support only contract

Outputs where the substantive link occurred before the submitted output was made publicly available, will not be eligible for submission where the author was subject to compulsory redundancy.

HEIs must set out their approaches to submitting eligible outputs by staff made redundant in their CoP.

Exceptional circumstances

In exceptional, clearly justified and pre-cleared cases, outputs sole-authored by staff on teaching-only or other non-research contracts may be submitted, where:

• the institution has provided allocated time and funding to support that specific research activity

Where HEIs wish to submit outputs of this type, they are required to outline a transparent and consistently applied approach in their CoP, before seeking clearance from the REF director for submission.

Further details on the process to request clearance to submit these outputs will be published as part of future guidance regarding special requests.

Outputs authored during an excluded contract

Outputs authored by individuals while on an excluded contract may be eligible for submission if the individual subsequently moved to an eligible contract (that is, Teaching and Research or Research only) during the REF period, subject to the following conditions:

• the contracts were both held at the same institution and meet the definition of continuous employment (as defined by Section 210 of the Employment Rights Act 1996 (ERA 1996) or the Employment Rights (Northern Ireland) Order 1996)
• all other eligibility criteria for an eligible employment relationship are met within the REF period (1 January 2021 to 31 December 2028)

Where HEIs submit outputs of this type, they are required to outline a transparent and consistently applied approach in their CoP.

Exceptions

The funding bodies are committed to encouraging diversity in research, with the assessment framework recognising all types of research and forms of output. In support of this commitment, and to mitigate an unintended consequence that research methods or output formats that typically take longer to produce are disinvested in by institutions, the funding bodies consider that it is appropriate there is an exception in place in REF 2029 to the requirement for an eligible employment relationship to have occurred either during or before the point at which an output has been made first publicly available.  

This exception will apply to longform and/or long process outputs only, and will allow the submission of these output types in REF 2029 where an eligible employment relationship occurs after the point that the output is first made publicly available, for a time-limited period.

Additional guidance will be published as part of the Panel Criteria and Working Methods on the outputs that meet the definition of longform and/or long-process and on the time-limited period that will apply. The guidance will be summarised here. This will be informed by further engagement with sector organisations and representatives. 

Representative submissions

The expectation is that submissions include the highest quality research across all the disciplines in which an HEI is active.

Measures have been put in place to mitigate concerns that full decoupling could lead to the underrepresentation of research areas and research groups.

Submissions should be representative of the research undertaken by the submitting unit during the REF period. They should reflect the range of research activity in the UoA.

A submitting unit comprises the collection of subject structures, research groups and research activity in an HEI, submitting to a UOA.

Further detail will be provided on the definition of a submitting unit in the context of the People, Culture and Environment assessment element.

The representativeness of submissions will be assessed via the CKU Disciplinary Level Evidence Statements (DLS) and HEIs will describe their procedures for achieving a representative submission in their CoP.

Panels will assess the extent that assessment criteria are met and will apply a grade for each submission, with the representativeness of submissions benchmarked within UoAs and moderated between UoAs, to ensure consistency, while considering disciplinary difference.

The important issues of research diversity, demographics and career stages will be assessed as part of the wider disciplinary level evidence statements (DLS).

A representative submission will:

• provide a description of the comprehensive scope of the research activity in the submitting unit. This should be consistent with the research strategies and structures of the submitting unit and the unit’s research strategies
• contain outputs that represent the range of research activity in a submitting unit over the assessment period and provide a clear explanation for the omission of any areas of research activity not represented and how this aligns with the research strategy outlined in the DLS

Full guidance on panel working methods and criteria (for example, scoring convention and methods), and on REF 2029 disciplinary level evidence statements (for example, weighting of representativeness) will be developed and published after working with the REF panels.

Representation of research activity

Representation of research activity will be assessed as part of the CKU element of the DLS and will be assessed on the extent that:

• submitted outputs are reflective of the research strategies and structures of the submitting unit over the REF period, including contributions to large-scale research programs, networks, or national/global initiatives
• submitted outputs reflect the diversity of subject areas that are pursued within the UoA
• submitted outputs reflect the diversity of research practices and outputs produced within the disciplinary area, including the extent to which different and diverse types of research are included in the submission

Code of Practice (CoP)

HEIs are required to set out in their Code of Practice (CoP) their process for ensuring that their output submissions are representative of the research being undertaken within UoAs and their research strategies. This will ensure that processes are transparent, robust, and equitable.

The representativeness of submissions will be assessed as part of the REF 2029 disciplinary level evidence statements. As such, the CoP will describe a HEI’s approach to achieving representativeness in submissions, as set out in their disciplinary level evidence statements.

HEIs will be required to publish analysis of the extent of the success of their approach in their CoP, this will reflect the evidence submitted in the REF 2029 DLS.

Disciplinary level evidence statements (DLS)

The representativeness of an institution’s output submission will be assessed at the disciplinary level, via REF 2029 disciplinary level evidence statements (DLS).

The DLS contribute to the assessment score for each assessment element. Guidance on the full content of the DLS, weighting and the approach to assessment will be finalised with REF panels.

Representation of research activity will be assessed at UoA level as part of the unit’s overall CKU assessment score.

The full guidance for disciplinary level evidence statements will be published at a later date.

Significant research contribution (SRC)

In cases of multi-author outputs, it may be necessary to demonstrate that a significant research contribution (SRC) was made by an author with an eligible employment relationship to the HEI.

It is anticipated that the approach to demonstrating SRC will broadly align with the approach used in REF 2021.

Panel guidance on SRC

Panels will be asked to confirm their approach to SRC, including approaches to edited collections/special issues, where author contributions may be absent, and identify disciplinary differences to set requirements around SRC. 

Required output data

For each output submitted, enough information should be given to enable the REF team and panels to determine precisely what is being listed, whether it is a product of sole or multiple authorship or production, in what form it exists and where it may be found.

The following are required for each output:

Output number

Output number sequentially from one for each output listed in a submission. This number is for administrative convenience of referencing only.

Date of output

Date of output, specifically the calendar year in which the output became publicly available.

For outputs where the substantive link is through a former staff member, the month in which the output first became publicly available will also be required.

Type of output

Outputs should be categorised into the following broad types (there will be a number of specific data requirements in common for each output type). Further guidance on types of outputs and examples of each output type, will be published later in the Autumn in an update to REF 2021 Guidance on Submissions Annex K, which offers a good indication of the potential range of outputs.

Title of the output

If the output has no title, a description is required.

HESA Staff Identifier

Where the substantive link is evidenced by an eligible employment relationship where the staff member has a HESA Staff Identifier in the Staff record, this must be supplied. If the eligible employment relationship is evidenced by a staff member without an associated HESA Staff Identifier, this must be indicated.

Additional output data

Each of the following is required where applicable to the output:

Digital object identifier (DOI)

Where available the DOI of the output should be included.

Former staff

A flag to indicate whether the output was authored by a member of staff who is no longer employed by the HEI at the submission date.

Pre-publication substantive link

A flag to indicate that the substantive link was pre-publication of the output, but within the relevant time limited period when the research resulting in the output was carried out.

Open access

For in-scope outputs only, whether the output is compliant with the open access requirements, has an applicable exception, or is not compliant.

Request for cross-referral

A request to the sub-panel to consider cross-referring the output to another sub-panel for advice.

Research group

The research group to which the research output is assigned, if applicable. This is not a mandatory field, and neither the presence nor absence of a research group is assumed.

Research specialism

Where requested in the Panel criteria, output allocation information to be provided to assist in allocating outputs to appropriate readers.

Co-authors

In the cases of co-authored or multi-authored outputs, the number of additional co-authors.

Disciplinary identifier flags

These flags will be determined by sub-panels and will be updated here.

A brief abstract

Required for outputs in languages other than English (see sub-section 16).

Pending publication

A flag to indicate that the output is due for publication between the submission deadline and the end of the publication period and therefore may have an associated output on the reserve list (see sub-section 18).

Request to ‘multiple-weight’ the output

For outputs of extended scale and scope, the submitting institution may request that the sub-panel weights the output as more than one output, (see sub-section 19). Associated output/s may be submitted on the reserve list.

Supplementary information

The DOI (or other URL, if no DOI is available) for any supplementary information published alongside an output.

Confidential output

Whether the output should be omitted from the published data for specific reasons, such as commercial sensitivity or security.

Panel additional data requirements

Where the panels require additional information relating to outputs for the assessment of research in their UoA, this will be published in the Panel criteria. A summary will be added below.

Assigning outputs to UoAs

In REF 2029, the allocation of outputs to UoAs will follow a revised approach that reflects the decoupling of contracts and outputs. This means that outputs are not tied to specific researchers and can be submitted by the institution if a substantive link can be evidenced.

Where an author’s contract has been associated with a UoA, there is no requirement for the output to be submitted to the same UoA. The following principles and guidelines will govern the allocation of outputs to UoAs:  

Institutional responsibility

HEIs are responsible for mapping outputs to the UoAs to which they are associated. This involves careful consideration of the output’s subject matter, the UoA descriptors, and the overall research strategy of the institution.  

UoA descriptors

Outputs must align with the disciplinary areas defined by the UoA descriptors and set out in the Panel criteria and working methods.

Output assignment flexibility

Outputs do not need to be returned to the UoA that is associated with the author’s contract of employment, including in cases where the author’s contract is included in the volume measure.

This allows for greater flexibility in allocating outputs, especially for interdisciplinary research or outputs with relevance to multiple UoAs.

Single submission

An output cannot be included more than once within the same UoA submission by one HEI. This prevents double-counting and ensures that each output is assessed only once. This does not affect outputs which have been selected for multiple weighting.

Interdisciplinary research

REF 2029 encourages the submission of interdisciplinary and transdisciplinary outputs.

Submission to multiple UoAs

An output can be included in the submission to multiple UoAs at the same institution, however the output must meaningfully align with the disciplinary areas defined by each of the UoA’s descriptors. It is anticipated that UoA descriptors will be sufficiently broad to allow interdisciplinary and transdisciplinary outputs to be meaningfully aligned with the appropriate disciplinary areas.

An output will be assessed separately for each UoA it is submitted to and may score differently in each case.

Submission to more than two UoAs will increase the likelihood of audit, to ensure that it meaningfully aligns with the UoA descriptors for each UoA to which it is submitted.

No Interdisciplinary Research (IDR) flag

The IDR flag used in REF 2021 will not be used in REF 2029. This decision was made based on feedback that the flag was ineffective in capturing the interdisciplinary nature of research.  

Cross-referral

Cross-referral remains an important tool for assessment. This is a request to the sub-panel to consider cross-referring the output to another sub-panel for advice. This does not indicate incorrect allocation to a UoA.

Co-authored/co-produced outputs 

For co-authored outputs, the number of other authors will be required. Regardless of the number of authors listed on an output, a co-authored output listed in a submission will count as a single output in the assessment. Co-authored/co-produced outputs will not be counted pro-rata.

Where a co-authored output is eligible for return in different submissions (whether from the same HEI or different HEIs), the output may be returned in any or all of these submissions.

Disciplinary identifiers

Where sub-panels identify a need, disciplinary identifier flags will be created.

Details of any disciplinary identifiers will be added here.

Languages

While the majority of outputs for assessment will be submitted in English, for research outputs in a language other than English, different assessment processes apply based on the language’s legal and assessment status.

Welsh and British Sign Language (BSL)

Outputs submitted in Welsh or BSL will be assessed directly in their original form, without requiring translation or interpretation. Panels will ensure appropriate expertise is available to assess these outputs within the REF process.

Irish, Irish Sign Language, Ulster Scots, Scottish Gaelic, and Scots

Outputs submitted in these languages will be assessed in translation where necessary, ensuring accessibility for panels while maintaining recognition of linguistic diversity. Specialist advisers fluent in these languages will be consulted as required.

Abstracts for outputs in a language other than English

For research outputs in a language other than English (including outputs submitted in the medium of Welsh and BSL), a short abstract in English should be provided to describe the content and nature of the work (maximum 100 words).

Panels will use this abstract to identify appropriate assessment, including (where required) external specialist advisers to whom the work may be referred.

The abstracts themselves will not form the basis for assessment. Work may be referred to external specialist advisers only where panel members and assessors are unable to assess an output in the language in which it is submitted.

Abstract exemptions

The requirement for an English abstract is waived for outputs submitted in UoA 26 (Modern Languages and Linguistics) if the output is produced in any of the languages within the remit of that UoA.

Similarly, the requirement is waived for any other UoAs that indicate in their criteria statements that they are able to assess outputs in that language directly.

Translation specialist advisors

For research outputs submitted in languages other than English (except for those submitted under UoA 26 or another UoA with direct language submission criteria) institutions will have the option to tick a box indicating whether the author considers it appropriate for the output to be assessed in translation. This ensures greater transparency and flexibility in the assessment process while respecting authorial intent and disciplinary considerations.

Outputs with delayed publication

Institutions may include a reserve output for each output expected to be made publicly available between the submission date and the end of the publication period.

Submission of reserve outputs for outputs with delayed publication

A reserve output will be associated with the specific output which is pending publication.

Assessment of reserve outputs

If the submitted output pending publication is not made publicly available on or before the 31 December 2028, the panel will assess the associated reserve output.

A reserve output will only be assessed in the event the output is not made publicly available within the publication period.

Multiple-weighting of outputs

Institutions may request that outputs of extended scale and scope be multiple-weighted (count as multiple outputs) in the assessment.

The panels will provide more information in the Panel criteria about outputs that may merit multiple-weighting in their discipline areas.

Further work will be done to explore the potential use of triple- and quadruple-weighting in REF 2029 and this section will be updated at a later date.

Process for requesting multiple-weighting of outputs

Institutions’ requests for multiple-weighting must be accompanied by a statement of up to 100 words explaining how the scale and scope of the output satisfies these criteria.

Where requesting an output to be multiple-weighted, the submitting institution must reduce the number of outputs listed in the submission by the appropriate number of outputs (unless including ‘reserve’ output/s, as described below).

The sub-panels will decide whether to multiple-weight each output that has been so requested, according to the published criteria. This decision will be separate to the panel’s judgement about the quality of that output.

Where the panel decides to multiple-weight an output, it will count as more than one output in the submission. Where the panel does not accept the case for multiple-weighting, it will count the submitted output as a single output, and grade the ‘missing’ output/s as unclassified (unless a ‘reserve’ output/s is included).

Reserve output/s

Institutions may include a ‘reserve’ output/s with each output requested for multiple-weighting.

A reserve output/s will be associated with the specific output for which multiple-weighting has been requested. If the request is declined, the panel will assess the associated reserve output/s.

A reserve output/s will only be assessed in the event that the panel does not accept the request for multiple-weighting.

Reserve pool

To support diversity of outputs and to provide flexibility in submission, reserve lists will be available for certain types of outputs.

In all circumstances a reserve output/s will be associated with a specific output within the submission.

The reserve list will be used for the following types of outputs:

Outputs with delayed publication

This includes outputs that have been accepted for publication but have not yet been made publicly available.

The associated outputs from the reserve lists in this situation would only become eligible if the output was not made publicly available before 31 December 2028.

Multiple-weighted outputs

Institutions may request that outputs of extended scale and scope be multiple-weighted (count as multiple outputs) in the assessment.

The associated output/s from the reserve list in this situation would only become eligible if the request for multiple weighting is declined.

A reserve output/s will only be assessed in the event that the panel does not accept the request for multiple-weighting.

In situations where multiple reserve outputs are submitted they must be provided in priority order, should fewer than the total number of reserve outputs need to be assessed (for example, if a multiple weight of three is requested but only double weighting is approved by the panels).

Confidential outputs

Confidential reports include any item produced for and lodged, in the publication period, with a company, government body or other research sponsor(s), but which has not been published because of its commercial or other sensitivity.

Permission to submit a confidential output

A confidential report may only be submitted if the HEI has prior permission from the sponsoring organisation that the output may be made available for assessment. HEIs will confirm permission has been secured when they make submissions.

If the REF team requests a confidential report for assessment, the HEI must make it available.

Confidential reports will only be shared with the REF team and those involved in the assessment process.

Confidential outputs and panels

All panel members, advisers, observers and others involved in the assessment process are bound by a confidentiality agreement. Therefore, it should be possible for HEIs to submit confidential reports without compromising any duty of confidentiality upon them.

There may be main or sub-panel members who HEIs believe would have a commercial (or other) conflict of interest in assessing confidential reports. HEIs will be required to name such individuals when making submissions.

Confidential outputs and REF publication

Outputs identified by institutions as confidential will not be listed as part of the published submissions.

Confidential reports submitted to the REF will be destroyed as soon as no longer required for assessment purposes.

Role of the Code of Practice (CoP)

HEIs must develop and implement a CoP for REF 2029 following the guidance  and using the template provided.

HEIs are required to set out in their CoP their process for output selection and ensuring that their output submissions are representative of the research being undertaken within UoAs.

The CoP requires institutions to set robust, transparent, and equitable and inclusive policies and procedures for the selection of outputs.

CoP and output selection

The CoP should outline the policies and procedures for the fair and transparent selection of outputs for submission.

In addition to addressing the selection of outputs for submission, these policies and procedures must include:

• policies and procedures for identifying substantive link
• policies and procedures for allocating/assigning outputs to UoAs
• procedures for ensuring submissions are representative of the research undertaken within the UoA during the REF period and the submitting unit’s research strategies

CoP complaints and investigation process

The funding bodies will approve REF CoP and provide a complaints and investigations process for when individuals believe CoP have not been followed. The complaints and investigations process will include a requirement (where appropriate) to take corrective actions on submissions where complaints are upheld.

Unit reduction requests

In exceptional circumstances, institutions may submit a request to reduce the number of outputs to be submitted to a UoA. Such requests will only be considered where there is clear evidence of issues that have impacted the unit’s ability to research productively.

Criteria for unit reduction requests

To be considered, unit reduction requests must meet the following criteria:

• Exceptional circumstances: the request must be based on exceptional circumstances that were unforeseen and outside the institution’s control. Examples could include significant and unexpected funding cuts, major restructuring, the closure of a research centre, disruption caused by disaster such as fire or significant building issues, or a collective of sustained and disproportionate equality related circumstances.
• Significant impact: the circumstances must have had a significant impact on the UoA’s ability to research productively.
• Evidence-based: the request must be supported by clear and compelling evidence, including data or details of the exceptional circumstances, and an explanation of how these circumstances have impacted the UoA.

Process for submitting unit reduction requests

Institutions wishing to submit a unit reduction request must do so to the REF team. The REF team, working with PDAP, will review each request on a case-by-case basis and will decide based on the evidence provided and the criteria outlined above.

Further detail on the process and associated deadlines will be published at a later date and updated here.

Decision on unit reduction requests

The REF team will inform the institution of its decision in writing. If the request is approved, the REF team will specify the number of outputs that must be submitted to the UoA. If the request is denied, the institution will be required to submit the full number of outputs to the UoA.

Transparency and accountability

All decisions on unit reduction requests will be made in a transparent and accountable manner. The REF team will publish an anonymised summary of requests received and the decisions made.

Citation data

Panels will develop the approach to the use of citation data. Full details will be published in the Panel criteria and will be summarised here.

Panel use of citation data

This will be updated in future.

Format of citation data

This will be updated in future.

Access to submitted outputs

Final data collection formats and validations will be published shortly after the final guidance is set.

The full guidance on access to submitted outputs is being developed and therefore this entire sub-section (sub-section 24), will be updated at a later date.

Audit

The full audit guidance for REF 2029 is being developed and therefore this entire sub-section (sub-section 25), will be updated at a later date.

REF 2029 will audit robustly. Audit may focus on, but not be limited to, the following.

Verification of output selection

HEIs must be prepared to demonstrate that all outputs included in the submission genuinely meet the eligibility criteria. The evidence for this should be tied to the documented processes in the CoP.

Verification of output UoA assignment

HEIs should be prepared to demonstrate the rationale for allocating outputs to a UoA/UoAs.

Where an output is assigned to more than two UoAs, this may increase the likelihood of audit.

Consequences of non-compliance

Unsubstantiated assignment to a UoA: if the audit identifies an output which does not align with the disciplinary areas defined by the UoA descriptors and set out in the Panel criteria, that output will be removed from that UoA’s submission and an unclassified score will be assigned for each missing output.

Ineligible outputs included: if the audit finds that ineligible outputs were included in the submission, they will be removed from the unit’s submission and an ineligible score of zero will be assigned for each missing output.