Sponsored content
Back to earth
A new report from Elsevier explores academia’s desire to demonstrate the real-world impact of research and set a trajectory for more holistic evaluation.
The move to incorporate more real-world impact when assessing the value of research has been gathering pace steadily for some years. A recent survey of 400 members of the research community has revealed a strong desire for this to happen – and indicates how the change could be accelerated by greater coordination on methodologies and standards, for instance.
The survey, of researchers, academic leaders and chiefs of funding bodies in seven countries, was conducted in late summer of 2023. Its findings illuminate many problems with the status quo and suggest who may be best placed to change the situation. They also explain why reform in this area, despite strong support, has progressed more slowly than other advances in the research ecosystem, such as the move to more open data sharing.
There is no doubt that the research community cares about real-world impact. Some 30% of respondents said that research should always aim to have a direct impact on the wider world. The figure rises to 82% if it includes those who agreed that research should at least aim to have an indirect impact. This level of sentiment was consistent across all fields of study.
In certain countries, the shift to a more holistic measurement of the value of research has been ongoing for some time. The transition to a world in which data is abundant has acted as a catalyst.
Severe criticism, voiced by many
The survey reveals that criticism of the status quo is more widespread than many might realize. The respondents were asked whether current methods of research assessment could be described as crude, outdated, opaque, discriminatory, or unfair – five severe criticisms of a system that distributes large amounts of funding and can terminate careers with a single decision. Each criticism was endorsed by between 13% and 20% of the respondents. Crucially, these weren’t the same few people with highly negative views: 51% of respondents expressed at least one of these five criticisms.
Given this level of discontent, why has progress towards impact-based assessment to date been so modest?
Those most enthusiastic about change are not concentrated in any one place.
The survey shows that many members of the research community are excited by the idea of focusing more on the real-world impact of research, but they aren’t concentrated in a single, vocal constituency. They are linked by mindset, not by location, role or any other factor.
What is that mindset? Those who advocate for a more holistic approach to assessment are 1.6 times more likely than average to believe strongly that their research is well understood by the public. And they are 4.2 times more likely to agree that they’re “passionate about being part of research that has a positive impact on the world.” These passionate, change-seeking people can be found in all parts of the research ecosystem.
A lack of consensus on definitions and methodologies.
It would be easy to think that real-world assessment would be straightforward in today’s data-rich society. But the problems concern coordination, alignment, and frameworks, not data. When asked for their views on the most important barriers to impact-based assessment, 56% of respondents cited a lack of common frameworks or methodologies.
Motivation for change
This issue is only going to become more pressing. Almost two-thirds (66%) of respondents agreed or strongly agreed that public pressure for research to make a tangible contribution to society will intensify. But there is a disconnect: although the funders said that they were backing researchers’ calls for change, the academic leaders and researchers did not rate them as highly motivated. The consensus among the latter group is that academic leaders are offering the most support.
When respondents were asked what they would do personally to effect change, 52% said that they would participate in a communitywide initiative. Specific campaigning actions were popular, with 42% saying that they’d spend their own time on research supporting the case for change and 40% happy to deliver a lecture. Others were keen to sign open letters and/or be interviewed by the media.
Of course, this desire for change is powering organizations such as the Coalition for Advancing Research Assessment (CoARA) and the Network for Advancing & Evaluating the Societal Impact of Science (AESIS Network). But the indications are that there’s more enthusiasm to be tapped. Supporters of change are spread widely around the ecosystem, but a breakdown of respondents shows that they range from early-stage PhD students to university vice-chancellors and those in charge of funding budgets. Once their energy is set in motion, it should have the power to transform not only research priorities but the scientific path of humankind.