Analysis of responses to Initial decisions consultation

1. In June 2023, the Future Research Assessment Programme (FRAP), on behalf of the UK higher education funding bodies in England, Scotland, Wales and Northern Ireland, published a joint report: ‘Research Excellence Framework 2028: Initial decisions and issues for further consultation (REF 2023/01)’. We will refer to this report as the ‘Initial decisions’ throughout this document. 

2. The changes outlined in the Initial decisions aim to create an assessment exercise that will underpin: 

• a research system that produces high quality, rigorous research that is open to all 
• an inclusive and collaborative research system that supports a diversity of people, ideas, institutions, methodologies, outputs, and activities 
• an engaged and impactful research system that connects research with wider society to bring about positive socio-economic change 

3. Open from June to October 2023, the FRAP launched a consultation which invited comment from the sector on several of the initial decision proposals. 

4. In this report we present the completed analysis of the consultation, published in line with our commitment to transparency and collaboration with the research community. This report is presented following the structure of the consultation, and is organised into the following sections: 

• section 1 – Volume measure  
• section 2 – Contribution to Knowledge and Understanding (CKU) 
• section 3 – Engagement and Impact (E and I) 
• section 4 – Units of Assessment  
• section 5 – Impact of the COVID-19 pandemic  
• section 6 – Cymraeg in HEFCW  
• section 7 – Consultation statistics and questions 

5. Each section contains a short introduction, followed by the specific questions asked in the consultation, before providing an analysis of the responses received and an outline of the next steps. The questions asked in the consultation have been collated at the end of this report. 

6. Specific issues within the analysis are presented using language that reflects the number of respondents who raised that issue. It should be stressed that while a specific point may be raised by a minority of respondents, that does not mean it is not an important issue, and such views will still be considered alongside more emergent themes. Conversely, while a majority of respondents may raise an issue, those views will be considered on the strength of the argument rather than frequency of response. Policies will be balanced with other decisions made across the whole assessment framework, weighing up relative benefits and risks. 

7. It is also important to note that this report is a summary of the consultation analysis. Many of the topics discussed in this report are complex, and while great effort has been made to ensure reporting is reflective of the consultation response, it is not practical to represent the many individual perspectives in fine detail. This summary represents one output from the consultation process, and the varied perspectives gathered as part of this consultation will inform development of REF policy and discussion in the sector.   

8. Separate engagement activities were carried out to gather community views on People, Culture and Environment (PCE). While there were responses to this consultation that contained useful discussion on PCE, these comments fall beyond the scope of this report. 

9. In recognition of the complexities for Higher Education Institutions (HEIs) in adapting to proposed changes to the REF, it was confirmed in December 2023 that the next exercise would be extended to 2029. For the avoidance of confusion, this report will refer to REF 2029, unless directly quoting the Initial decisions. 

10. This report is being published as part of module 3 in the REF 2029 policy roadmap. Further policy updates will be published on the REF 2029 website going forward. 

Background 

11. In REF 2021, volume was calculated by HEIs through a process of determining eligibility and the submission of a census point staff list. The process of determining eligibility was outlined in institutional Codes of Practice, which were agreed upon on a case-by-case basis by the funding bodies. 

12. To gain a more accurate representation of a unit’s research capacity across the REF assessment period, the funding bodies intend to calculate the volume using an average full-time equivalent (FTE) of eligible staff across academic years 25/26 and 26/27, rather than based on the FTE of staff employed on a single census date. This will enable the volume measure to reflect changes across the period, such as the transition of staff from non-independent to independent researchers or changes in working patterns from year to year. 

13. Regarding volume measurement, the Initial decisions outlined that: 

• institutions will not submit staff to REF 2028 
• staff data will be gathered directly from HESA (Higher Education Statistics Agency) to determine the volume in REF 2028  
• staff eligibility criteria will be used to identify staff posts that contribute to the volume measure 
• the funding bodies will calculate the volume using an average FTE of eligible staff across academic years 25/26 and 26/27 
• institutions will be able to request an exemption from submission for very small disciplinary areas 

• institutions will be able to undertake a light-touch refresh of their REF 2021 Codes of Practice, where appropriate 

14. In the consultation we asked the following questions:  

• what practical challenges may institutions face in implementing these changes? 
• how might the funding bodies mitigate against these challenges?  
• what would be the impact of these changes on individual researchers and particularly those with protected characteristics or other underrepresented groups? 

Analysis of the responses  

15. A large minority had concerns that the HESA return date would only be a few months before the planned REF submission deadline, which left little time for amendments to submission post volume calculation. This would be especially problematic for planning related to impact case studies.  

16. A minority stated that the transition from staff lists to HESA data would require significant revision of institutional Codes of Practice. 

17. A small minority argued that there would be additional burden associated with the assignment of significant research responsibility (SRR) status to individual staff members. 

18. A small minority highlighted difficulty in mapping staff into REF specific HESA fields, given the diversity of internal HEI structure, the blur between associated disciplines and the recent increase in highly interdisciplinary research. 

Next steps 

19. This consultation was undertaken prior to the decision to delay REF 2028 to REF 2029. That decision was partly informed by concerns related to HESA data return dates. 

20. The funding bodies appreciate that HEIs require clarity of the practical use of the HESA data for REF. Policy and plans relating to the use of HESA data to calculate REF 2029 volume measures have been published alongside this report as part of REF 2029 Policy Roadmap module 3.  

21. Further decisions related to the use of HESA data are scheduled for release in module 4 of the policy roadmap, due to be published in Autumn / Winter 2024. 

Background 

22. As part of the Initial decisions for REF 2029, the REF 2021 sub-profile of “outputs” has been replaced by Contributions to Knowledge and Understanding (CKU).  

23. The Initial decisions report outlined the following decisions regarding the submission of outputs: 

• REF 2028 will fully break the link between individual staff members and unit submissions 
• institutions may submit outputs produced by any staff member where there is a demonstrable and substantive link to the submitting institution within the REF assessment period 

• institutions will be able to submit one output to multiple submissions, provided it falls within the Unit of Assessment (UoA) descriptors 
• given there is no minimum output requirement for volume-contributing staff there is no need for a process to account for individual equality-related circumstances 

24. Fully breaking the link between individual staff members and submitted outputs would simplify the output submission process, as units will no longer be required to attribute outputs to individual staff members with minima and maxima for each staff member. This reform also removes the need for the individual staff circumstances process and will enable the funding bodies to streamline and simplify unit circumstances processes. An additional intention of decoupling individual staff members from institutional submissions is to reduce the focus on individuals and to be more inclusive of research generated by individuals not on research-based contracts. 

25. Outputs will be considered eligible where there is a demonstrable and substantive link to the submitting institution within the REF assessment period. This includes outputs produced by anyone employed by the institution on a minimum 0.2 FTE contract for at least 6 months in the REF assessment period. It would be necessary only to demonstrate a link to the institution, recognising that relevant outputs (that is, those outputs that fall within the UoA descriptor) may be produced by those in multiple administrative units within the HEI. This will allow the submission of outputs produced by any staff member, including non-academic staff, where a link to the institution can be demonstrated. A specific discussion on the inclusion of sole-authored PhD students was acknowledged as a question in the consultation due to ongoing discussion within the sector.  

26. In REF 2021, any outputs that were co-authored by eligible staff were admissible within multiple submissions, regardless of whether they were from a single or multiple HEIs. The funding bodies proposed the retention of this policy alongside the introduction of the substantive link criteria. 

27. In the consultation we asked the following questions: 

• what would be the impact of these changes on individual researchers and particularly those with protected characteristics or other underrepresented groups? 
• what impact would these changes have on institutions in preparing output submissions? For example, what may be the unintended consequences of allowing the submission of outputs produced by those on non-academic or teaching-only contracts? 
• should outputs sole-authored by postgraduate research students be eligible for submission? If so, should this include PhD theses? 
• what would be appropriate indicators of a demonstrable and substantive link to the submitting institution? 

Analysis of the responses  

28. We have broken the analysis of the responses into the following sections: 

• breaking the link between individual staff members and unit submissions 
• eligibility of outputs sole-authored by PhD students  
• substantive link criteria and output portability  
• submission of co-authored outputs 

Breaking the link between individual staff members and unit submissions 

29. While respondents recognised positive impacts for individuals and institutions in decoupling individual staff members and unit submissions, there were concerns around the need for carefully linked policies throughout the wider exercise to ensure this change met the principles upon which it is based.  

30. A small majority were concerned that decoupling would allow submitting institutions to exploit staff on teaching-only contracts, making use of their research without paying contracted hours for them to conduct the research.  

31. A large minority were concerned that full decoupling might increase HEI burden through the development of required adaptions. This includes demonstrating a substantive link to an institution, processes to select outputs for submission involving a wider pool of outputs, and the potential for increased complexity of selecting outputs without linking to staff members.  

32. A minority expressed concern that the decoupling of outputs from individuals would allow submitting institutions to focus their submissions on the work of a small number of individuals. Submissions that are concentrated on the outputs of ‘research stars’ could lead to the exclusion of others, in turn prioritising certain types of research and publication practice and putting undue pressure on ‘research stars’ to carry the performance of their department or institution in the REF exercise. 

33. A minority felt that decoupling could provide an opportunity to showcase the research of staff members who might not have been eligible for submission to previous REF exercises. 

34. A small minority felt the decoupling policy could foster opportunities for increased collaboration; this was both at an institutional and individual level such as collaboration with non-academic staff, including ‘team science’, or increased collaboration with industry or practice.  

35. A small minority felt the policy would allow for more flexibility including allowing for different working models, such as secondments or working across academia and industry/practice and would allow institutions more freedom to hire staff without (traditional) research track records. 

36. A small minority were concerned that REF submissions might not be representative of the submitting unit, and that any policy would be required to ensure that issues surrounding equality, diversity, and inclusion were considered.  

37. A small minority highlighted the converse perspective, noting the opportunity for more inclusivity and/or improved EDI through allowing the inclusion of research from a broader range of staff or for a more balanced and less biased assessment. 

Eligibility of outputs sole-authored by PhD students  

38. A majority was against the inclusion of sole-authored PhD outputs (including thesis) on several grounds, including: 

• postgraduates are fee-paying students, and not staff, therefore it is unethical for HEIs to use their outputs to secure institution-level funding 
• postgraduate researchers are classed as “students” and are therefore focused on professional training, not research excellence 
• given that the recruitment and funding of postgraduates is variable between HEIs and programmes. Therefore, this could be considered an “output” of the system and unrelated to research excellence
• the inclusion of PhD theses would lead to a significant increase in burden for internal HEI submissions, REF peer review and on individual PhD students 

39. A minority argued in favour of the inclusion of sole-authored PhD outputs on the basis that:  

• inclusion in the REF would act as motivation for students to produce high quality outputs 
• inclusion could be more aligned with the REF 2029 aim to better recognise the diversity of research contributions 

Substantive link criteria and output portability  

40. A majority considered that the proposed 0.2 FTE / 6-month minimal contractual requirement would be an insufficient indicator of a demonstrable and substantive link. Concerns were raised that the current proposals would allow HEIs to hire “REFable” academics, for example those with a high quality near-to-publication portfolio, on short-term contracts, thereby allowing their submission, despite a debatable link between the HEI and output.  

41. A small majority thought that a research contract would be an acceptable substantive link between an HEI and specific output. It was noted that the definition of an eligible contract would require nuance, given the range of contract types used for researchers and the prevalence of multi-HEI contracts and secondment / sabbatical processes. 

42. A minority questioned how the substantive link provisions might impact the portability of outputs between HEIs. There were some concerns raised that the continuation of REF 2021 portability rules, combined with the substantive link, could encourage the use of short-term contracts to “buy-in” outputs.  

Submission of co-authored outputs 

43. A majority expressed clear support for the proposed co-authorship submission criteria. These supportive responses often cited the ease of continuity from REF 2021 as a significant positive factor. A large minority indicated that some form of “statement of contribution” would be needed to establish the justification for co-authorship. It was noted that such statements are already standard practice in certain disciplines, however there were also concerns that such statements are not practically auditable, and assigning contribution would impose an additional burden.  

44. A small minority specifically commented on the potential impact on interdisciplinary research (IDR). Concerns were raised that the option to make multiple submissions of co-authored outputs may encourage HEIs to prioritise the submission of cross-UoA outputs, at the expense of discipline specific research, regardless of quality. 

45. A small minority stated that further clarity is needed on if submissions should link to the relevance of the output to the UoA or the relevance of the author (staff member submitted to HESA) to the UoA. 

46. A small minority stated that further clarity is needed in whether the Main Panel D exception to multiple single-UoA submissions would continue. This exception would allow for outputs to be submitted to the same UoA if applicable to multiple co-authors. 

Next steps 

47. After initial analysis, the funding bodies announced in December 2023 that outputs sole-authored by either PhD students or teaching-only contracts would be ineligible for submission in REF 2029.  

48. The December update also confirmed that the proposed co-authorship rules would be implemented as presented in the Initial decisions. The funding bodies are cognisant of ongoing concerns related to the representativeness of outputs within submissions, and this issue will be considered during later policy refinement. 

49. The December update confirmed that further work would be undertaken to implement full decoupling, while addressing the concerns raised around EDI, submission representation and burden.  

50. Further policy on full decoupling, substantive link, and output portability is scheduled for release in module 4 of the policy roadmap, due to be published in Autumn / Winter 2024. 

 

Background 

51. From its introduction in 2014, the assessment of research impact has become recognised as a key positive outcome of the REF process. As part of the continuing evolution of REF, this sub-profile has been adjusted to “Engagement and Impact” and remains a 25% total scoring section for UoA submissions. 

52. The Initial decisions report outlined and sought the communities’ views on the following proposed changes for Engagement and Impact: 

• a structured statement will make up at least 20% of the quality sub-profile for this assessment element. The funding bodies intend to weight the statement on a sliding scale, proportionate to the number of case studies submitted to ensure that its contribution to the sub-profile is no less than that of a single impact case study
• the number of impact case studies required in each submission will be determined by the average FTE of volume-contributing staff in the unit. Institutions will be required to submit a minimum of one impact case study per disciplinary submission 
• the funding bodies propose to revise the thresholds between case study requirements 

53. Feedback from the REF 2021 panels supported including a narrative element to the assessment of impact. Given the introduction of narrative explanatory statements as part of PCE, the funding bodies decided that the Discipline Level Evidence Statement (DLES) would be used to capture elements that were previously explored in the “environmental statements” made as part of UoA submissions.  

54. In line with the reduction of the minimum impact case study to one, the initial decision proposed changes to the volume threshold boundaries that indicate the required number of impact case studies for a submission. The range between thresholds widens as volume increases, in conjunction with a reduced score contribution for each individual impact case study compared to the DLES. 

55. The funding bodies sought feedback from the community on several of these proposals, specifically the changes to the impact case study requirements and assessment format. Three linked questions were asked: 

• what will be the impact of reducing the minimum number to one?    
• what will be the impact of revising the thresholds between case study requirements?   
• to what extent do you support weighting the impact statement on a sliding scale in proportion to the number of case studies submitted? 

Analysis of the responses  

56. We have broken the analysis of the responses into the following sections: 

• minimum impact case studies reduction to one 
• proposed impact case studies volume thresholds 
• proposed “sliding scale” assessment system 

Minimum impact case studies reduction to one. 

57. A large minority stated that the reduction to one impact case study would help to reduce burden and give more flexibility in submissions for smaller units.  

58. A small minority had concerns that small teams would be under increased pressure to produce impact. A small minority raised concerns that the submission of a single ICS would result in de-facto deanonymization of impact case study authors within a small unit or niche discipline.   

59. A small minority had concerns that a minimum of one impact case study could lead to the prioritisation of “safe” research over innovative work, leading to a homogenisation of research priorities. 

Proposed impact case studies volume thresholds. 

60. A large minority had concerns that the impact case study scaling requirement for submissions above 170 FTE, proposed to be +1 ICS per 50 FTE above 170, would put larger submissions at an advantage. It was argued the largest submissions will be able to “cherry pick” their ICSs for submission.  

61. A small minority argued that the proposed thresholds would reduce the burden to produce impact case studies, especially for smaller units, which could increase the quality and diversity of submissions.  

62. A small minority noted that none of the proposed changes will mitigate against “cliff-edge” effects between volume boundaries.  

63. A small minority suggested that thresholds could be further considered, particularly at the smaller and larger ends of the scale to ensure impact case studies submitted are representative of the unit. 

Proposed “sliding scale” assessment system.  

64. A minority had concerns that a 50% DLES value would be too high for small units who may not have the administrative resource to craft such a heavily weighted statement. 

65. A minority thought that the introduction of the DLES, combined with the reduction to one impact case study for small units, would increase the REF representation of small disciplines. 

66. A small minority recognised that the reduced impact case study weighting at lower volumes could be beneficial to smaller units as it would reduce burden and enable better assessment of the non-output factors that contribute to developing impact. 

67. A small minority argued that regardless of the chosen thresholds, the impact case study contribution should always be greater than the DLES. There were similar concerns that the sliding scale changes significantly across the thresholds, yet the content is likely to be similar between submissions.  

68. A small minority highlighted the need to clarify “rigour” and “engagement” in REF guidance. 

Next steps 

69. The REF December update confirmed that the minimum number of case studies would be reduced to one for the smallest units. While it was not specifically raised as part of this consultation, it was also confirmed that the 2* minimum rating for an impact case study’s underpinning outputs would be removed as per proposals.  

70. Further policy decisions related to the impact case study number thresholds and the E and I sliding scale assessment system are scheduled for release in module 4 of the REF 2029 policy roadmap, due to be published in Autumn / Winter 2024. 

Background 

71. As part of the Initial decisions for REF 2029, the funding bodies had decided to retain the unit of assessment (UoA) structure from the previous exercise. Given the evolving nature of research, the funding bodies sought feedback from the sector on “if the UoA structure is relevant to you/your organisation, please indicate clearly any changes that you propose to the UoA structure and provide your rationale and any evidence to support your proposal”. 

Analysis of the responses 

72. Compared to other sections of this consultation, responses on UoA were largely made by specific respondents with strong interest and expertise in a single UoA or main panel. Due to this, it would not be accurate to use the same reporting language that has been done in other sections of this document. 

73. In the past, changes to UoA descriptors have been made largely on the grounds of better recognising “hidden” disciplines within the unit and clarifying the “home” for certain subjects that sit across UoAs. Concerns in relation to hidden disciplines stem from the idea that these UoAs are heavily weighted towards certain groups, for example neuroscience at the expense of psychology, and this leads to HEIs making a bias in submissions that do not accurately reflect the research going on in that UoA. Several disciplines were reported as “hidden” within the current UoA structure: 

• nursing in Allied Health Professions, Dentistry, Nursing and Pharmacy (UoA 3)  
• psychology in Psychology, Psychiatry and Neuroscience (UoA 4) 
• non-European languages in Modern languages and Linguistics (UoA 26) 
• the fields of Criminology and Midwifery (no specified UoA) 

74. In addition to hidden disciplines, there were concerns that certain disciplines warrant a specific UoA due to the specifics of their subject: 

• criminology within Main Panel C 
• technology within Main Panel C 
• ecology within Main Panel B 

75. Concerns were raised that the some UoAs contained too broad a range of disciplines. This led to a difficulty in individual units being able to understand their contribution to a submission sub-profile score, and there were additional concerns that larger UoAs have become too large to effectively assess and administer. Several suggestions were made to modify existing UoAs: 

• the separation of Finance and Accounting from Business and Management Studies (UoA 17) 
• the separation of UoA 24 into Sport and Exercise Science, and Leisure and Tourism 
• the transfer of Moving Film and Screen Studies from Music, Drama, Dance, Performing Arts, Film and Screen Studies (UoA 33) into Communication, Cultural and Media Studies, Library and Information Management (UoA 34) 

76. Concern was expressed that panels may not have the expertise to assess some outputs if UoAs comprise a broad array of disciplines, or where research is conducted by staff who work in or with practice or industry. 

77. Respondents noted concern that Main Panels C and D have a larger number of “fields of research” and overall output numbers to assess, compared to Main Panels A and B, and this contributes to higher levels of workload for those panels.  

Next steps 

78. As laid out in the Initial decisions, the funding bodies proposed the retention of the current 34 UoAs for REF 2029. As part of the December update, the funding bodies confirmed that the proposed structure would be carried forward for REF 2029. While it was acknowledged that many important issues were raised, there was not sufficient evidence that any changes to the UoA structure would outweigh the associated burden.  

79. While the main UoA structure has been decided, the issues raised will go into planning for and with panels, and engagement with relevant subject communities, moving forward. 

 

Background 

80. REF 2021 put in place several mitigations to manage the sudden disruption brought about by the COVID-19 pandemic. One of these mitigations was the introduction of “statements of impact of Covid” which could be used to contextualise the impact of the situation on that specific submission.  

81. The funding bodies intend to retain the statements on COVID-19 impact that were used in REF 2021, and to require some consideration of how Covid impacts have been addressed in output selection as part of the Codes of Practice. 

82. The following questions were posed on what COVID-19 mitigation might be warranted in REF 2029: 

• what is your view on the proposed measures to take into account the impact of the Covid pandemic? 
• what other measures should the funding bodies consider to take into account the impact of the Covid pandemic? 

Analysis of the responses 

83. A small majority considered the proposals as roughly appropriate.  

84. A minority outlined concerns about Covid’s impact (and in some cases ongoing impact) in relation to EDI and the effects on groups with different demographics. Many of these responses expressed concern that the disproportionate impact of Covid had on women, those with protected characteristics, those with parenting or caring responsibilities, and early career researchers (ECRs). 

85. A minority had wanted the REF to consider the variable impact that COVID-19 has had across disciplines. There is a view that highly collaborative and public facing research, often the humanities and arts, were disproportionally disrupted due to social distancing and travel restrictions that were applied throughout the United Kingdom. Similarly, it was noted that the emergency call-up of medical staff caused unique issues for medical and practice-based research.   

86. A small minority felt unable to comment without further information and in the absence of alternative proposals. 

87. A small minority did not consider the proposals appropriate and sought either alternative arrangements or the removal of COVID-19 mitigations from REF 2029.  

88. A small minority were concerned that smaller HEIs were disproportionally affected by pandemic related staff shortages. 

89. A small minority requested clarification on how COVID-19 mitigation statements would be used by the panels and would like clear guidance to be issued on how to prepare such statements.  

Next steps 

90. In the December update, the funding bodies confirmed that there will be no fundamental changes to the proposed measures to account for the impact of the COVID-19 pandemic. It is anticipated that these considerations will form part of a simplified unit-level reduction request process. 

 

Background 

91. The four UK funding bodies and the REF recognise the different use of languages throughout the union. As part of this consultation, the funding bodies sought to identify any consequences of the proposals that might have a disproportionate effect on the Welsh language. We asked the following questions: 

• what positive or adverse effects will the proposals have on opportunities for persons to use the Welsh language and treating the Welsh language no less favourably than the English language?  
• could the proposals be changed to increase positive effects, or decrease adverse effects on opportunities for persons to use the Welsh language and treating the Welsh language no less favourably than the English language? 

Analysis of the responses 

92. A majority agreed that the REF should continue to take measures to ensure equity between English and Welsh languages. There was no indication that the Initial decisions presented a potential disparity between the use of Welsh and English language.  

93. A small minority called on REF to expand language provisions to cover other languages that are protected within the Union, for example Gaelic.  

Next steps 

94. As a joint four nation endeavour, the REF is committed to the equal recognition of Welsh and English, as prescribed in law. The UK funding bodies will continue to consider expanded language provisions in REF 2029. 

 

Table 1 – Overall number of responses received by organisation type. 

95. Consultation responses could be provided by individuals or on behalf of organisations. Table 1 summarises the number of responses received, and the type of organisation represented by each response.  

Type of respondent	Number of responses
Total responses	260
Individuals	37
Higher Education Institution or equivalent	137
Subject body, sector body, mission group, etc. or equivalents	86

Chart 1 – National and regional breakdown 

96. Chart 1 shows a summary of the national and regional breakdown of Higher Education Institutional responses. Regions were not available for all HEIs, for example the Open University.  

Chart 1 – Organisational responses by nation and region 

Location data was obtained using the UCAS UK map of unis and colleges by region document.

Chart 2 – Transparent Approach to Costing (TRAC) group breakdown 

97. Chart 2 shows a summary of the responses by TRAC group. This analysis only includes HEIs where a TRAC group was available and is designed to give an indication of the level of engagement with the exercise by different types of institution. 

Chart 2 – Organisational responses by TRAC group 

Respondent numbers: TRAC A = 32/33 (97%), TRAC B = 22/23 (96%), TRAC C = 21/23 (91%), TRAC D = 13/15, (87%), TRAC E = 27/45 (60%), TRAC F = 14/22 (64%). Data was compiled using the following reference and TRAC definitions: Peer Groups for annual TRAC, TRAC fEC and TRAC(T) benchmarking 2020-21. Criteria (references to income are to 2012-13 data) Peer group A: Institutions with a medical school and research income* of 20% or more of total income Peer group B: All other institutions with research income* of 15% or more of total income Peer group C: Institutions with a research income* of between 5% and 15% of total income Peer group D: Institutions with a research income* less than 5% of total income and total income greater than £150M Peer group E: Institutions with a research income* less than 5% of total income and total income less than or equal to £150M Peer group F: Specialist music/arts teaching institutions.  

*Research income is defined as the funding council recurrent research grant plus the total research grants and contracts returned in the 2012-13 HESA Finance Statistics Return (FSR)

Table 2 – Responses per question: 

98. Table 2 shows the number of responses that were returned for each question: 

Question number	Total number of responses	Number of organisation responses	Number of individual responses
5	228	206	22
6	228	206	22
7	228	206	22
8	244	203	31
9	244	203	31
10	250	223	27
11	227	202	25
12	210	189	21
13	189	171	18
14	234	207	27
15	218	198	20
16	220	196	24
17	222	200	22
18	261	224	37
19	261	224	37
20	139	125	14
21	122	111	11

Consultation questions 

• Questions 1-5 were for registration purposes only. 
• Question 5: What practical challenges may institutions face in implementing these changes? 
• Question 6: How might the funding bodies mitigate against these challenges? 
• Question 7: What would be the impact of these changes on individual researchers and particularly those with protected characteristics or other underrepresented groups?  
• Question 8:  What would be the impact of these changes on individual researchers and particularly those with protected characteristics or other underrepresented groups? 
• Question 9:  What impact would these changes have on institutions in preparing output submissions? For example, what may be the unintended consequences of allowing the submission of outputs produced by those on non-academic or teaching-only contracts? 
• Question 10: Should outputs sole-authored by postgraduate research students be eligible for submission? If so, should this include PhD theses? 
• Question 11: What would be appropriate indicators of a demonstrable and substantive link to the submitting institution?  
• Question 12: Do the proposed arrangements for co-authored outputs strike the right balance between supporting collaboration and ensuring that assessment focuses on the work of the unit?  
• Question 13: Are there any further considerations around co-authored outputs that need be considered? 
• Question 14: What will be the impact of reducing the minimum number to one?    
• Question 15: What will be the impact of revising the thresholds between case study requirements?   
• Question 16: To what extent do you support weighting the impact statement on a sliding scale in proportion to the number of case studies submitted? 
• Question 17: If the UoA structure is relevant to you/your organisation, please clearly indicate any changes that you propose to the UoA structure and provide your rationale and any evidence to support your proposal.  
• Question 18: What is your view on the proposed measures to take into account the impact of the Covid pandemic? 
• Question 19: What other measures should the funding bodies consider to take into account the impact of the Covid pandemic?  
• Question 20: What positive or adverse effects will the proposals have on opportunities for persons to use the Welsh language and treating the Welsh language no less favourably than the English language?   
• Question 21: Could the proposals be changed to increase positive effects, or decrease adverse effects on opportunities for persons to use the Welsh language and treating the Welsh language no less favourably than the English language?