The Lens
Beyond rankings: Expanding data horizons
Universities should expand their data expertise to uncover data insights and leverage knowledge to improve their strategic and operational understanding of their institutions. And who knows? This could also impact future rankings.
By Dr Jingwen Mu, Director of institutional Research & Strategic Planning, Hong Kong Baptist University
Identifying and preparing quality sustainability-related data to map research and educational activities onto the SDGs is complex and opaque.
Over more than two decades, annual world university rankings have evolved into the Olympics of higher education. While they have spurred aspirations for excellence, many institutions have stumbled at this rankings hurdle by focusing too much on the outcome. For me, rankings are not about what we achieve but how we achieve it, just as sports scientists care more about how athletes win at the Olympics, not what they win. An increased understanding of rankings data enhances our insight into our institutions, allowing us to better prepare for the future.
To go beyond the rankings outcome itself requires a more sophisticated data-informed approach. Here, I offer two indicators to illustrate what this approach can reveal. These are the Sustainability and International Research Network, introduced at the QS World University Rankings’ 20th anniversary.
Sustainability
Sustainability and the Sustainable Development Goals (SDGs) are increasingly integrated into the global higher education landscape and institutions’ strategic goals. QS pioneered efforts to weave Sustainability as a core indicator into its flagship World University Rankings. To some extent, this focuses universities on managing their sustainability data and policies relating to research, curricular and co-curricular activities, operations and engagement. Identifying and preparing quality sustainability-related data to map research and educational activities onto the SDGs is complex and opaque. For me, it has two parts: standardisation and contextualisation.
Standardisation involves adopting consistent, algorithm-based criteria and methodologies to measure and report sustainability activities. Contextualisation, on the other hand, requires adapting standardised metrics to reflect an institution's unique context and priorities. This ensures the mapping process is both relevant and meaningful.
The two-tier Auckland Approach developed with colleagues at the University of Auckland exemplifies this method. It first identifies globally recognised common SDG themes, topics, and keywords, then localises them within the New Zealand context to broaden the data horizon on local knowledge and contributions.[1]
For individual academic scholars and the institutions they work for, a sense of shared ownership in an institution’s sustainability ambition is crucial for a sustained winning strategy. A transparent and reliable SDG mapping process facilitates this co-ownership and fosters a collaborative commitment to achieving these goals.
As well as reporting sustainability-related educational and research activities in the institution’s annual Sustainability Report [2], it was also important for academic units and individuals to be informed about their sustainability impact and contribution to the university’s collective efforts. This created a sense of momentum and served to inform future sustainability strategies and action plans.
The ability of generative AI to process, analyze, and predict based on a massive amount of text makes it possible to understand the diverse voices of student experiences.
International Research Network
The International Research Network (IRN) measure adapts the Margalef Index, which is used in environmental sciences to measure the number of species represented in an ecological community[3]. The IRN looks at the diversity of an institution’s research partners in terms of their country representation.
With around 200 countries in the world, this measure rewards institutions that go the extra mile to collaborate with countries, particularly in the Global South, which are emerging in the research arena.
Taking the University of Oxford and the University of Manchester as examples, the former received a full score of 100 (i.e., No.1) on the IRN indicator, while the latter received a score of 99.7 (i.e., No.10) in QS World University Rankings 2024. The difference, despite being numerically small, is primarily explained by the fact that Oxford has sustained research collaborations (i.e., three or more joint publications) with 133 countries/regions around the world, whereas Manchester has collaborations with 118 countries/regions. The additional countries/regions on Oxford’s collaboration map are predominately developing countries in the Global South (e.g., Madagascar, Gambia, Mali, and Papua New Guinea).
This extra mile, while seemingly straightforward, usually requires strategic-level support to give it a steer. At Hong Kong Baptist University (HKBU), the context is different, but the determination to broaden the reach of the institution’s research network is the same. The Belt and Road initiative was built into HKBU’s research development strategy with recognition for academic units supporting the institution’s effort to broaden the boundaries of its research network. Surfacing and leveraging insights from the IRN metric has been one effective way to draw faculty’s attention to the institution’s priority to build Belt and Road partnerships.
The future of rankings
The Sustainability and International Research Network's examples are a window into where data will take university rankings over the next decade. University rankings will embrace disruptive technologies and more inclusive metrics.
The ability of generative AI to process, analyse and predict based on a massive amount of text makes it possible to understand the diverse voices of student experiences. This may be a structured survey, a study experience post shared by a student on Reddit, or the graduate career path depicted by a person’s LinkedIn activities. The sheer scale of AI analytics is opening new possibilities, with the rankings world about to speed up and go beyond traditional measures such as the staff-student ratio.
AI will also enable the development of rankings knowledge graphs. These graphs will connect disparate data and ranking metrics, revealing intricate intra- and inter-institutional relationships and patterns. This would essentially democratise data insights, making the meanings behind ranking outcomes accessible and meaningful to a broader audience. Imagine a day when we ask a ‘Ranking-GPT’ to unpack a strong performance in research impact, and it replies by linking it to the newly introduced open research practices and an institution’s expanded collaboration network.
Conclusion
The higher education sector has embraced data and the QS World University Rankings showcase how data can be deployed. Looking beyond the rankings data affords enormous opportunities to explore our institutional performance further and how strategic objectives can be realised. As we reflect on new technologies, the access to and application of data will not only usher in a new rankings world, but also enable institutions to go beyond rankings and embed sophisticated data insights into institutional performance. As the saying goes, the Olympics is an ordinary performance on an extraordinary day.
References
[1] https://direct.mit.edu/qss/article/doi/10.1162/qss_a_00304/120309/Evaluating-approaches-to-identifying-research
[2] https://hkbu-sustainability.hkbu.edu.hk/reports/sustainability-report-2023.html
[3] https://support.qs.com/hc/en-gb/articles/360021865579-International-Research-Network-IRN-Index
Dr Mu is the Director of institutional Research & Strategic Planning at Hong Kong Baptist University, an internationally recognised university rankings expert and a member of the QS Global Rankings Advisory Board. She was previously Senior Global Strategy Advisor to the Vice-Chancellor at the University of Auckland, New Zealand.