Scope and Themes

At the REvaluation Conference 2024 we will explore the challenges and opportunities presented by the rapidly changing landscape of research and innovation policy and its implications for assessment and evaluation. Traditionally set in the city of Vienna, Austria, from December 4-6, 2024, this conference promises to be a gathering of thought leaders, academics, and practitioners who are charting the course through these dynamic times.

Conference Overview

We find ourselves in an era where socio-technical systems are undergoing profound shifts, and research and innovation policies must adapt to address pressing global challenges. We observe altering societal needs and policy striving to support transformational change; increased policy ambition has led to advanced demands for strategic intelligence, while new data sources and methodologies offer novel opportunities and pose new forms of epistemological uncertainties.

This edition of REvaluation Conferences aptly reflects this under the motto "Navigating Times of Change", convening all stakeholder groups that need to join forces for navigating these changes: researchers, evaluators and innovators, policy makers and public administrators, as well as funders of science and innovation from many different countries and regions.

From these different perspectives, we will exchange latest conceptual and methodological advances and empirical insights and jointly explore how we can best inform and shape the process of effectively navigating or guiding through the significant challenges and opportunities of evaluating and supporting STI policies. Together we want to explore ways of finding direction, adapting to change, and making informed decisions within dynamic environments RTI policy is operating in.

Concretely, we ask ourselves questions around the evidence we need to understand transformative settings; how we need to change the way we make sense of research success; how joint action is taken at regional and European level; how evaluation systems emerge and consolidate; and how new technologies shape our work. As a cross-cutting theme we reflect what ethical considerations structure our work and what responsibility lies in evaluation. These questions build the

Five Central Conference Strands

1. Evaluation of transformation policies and dynamics in socio-technical systems:
   The first strand recognises the delicacy of evaluating policies that aim at the transformation of socio-technical systems they are targeting. We try to understand how we can establish effective evaluation approaches that provide a comprehensive view of the progress, challenges, and limitations in achieving sustainability and innovation goals. This strand seeks to uncover best practices, methodologies, and insights to guide policymakers and evaluators.
2. Redefining success and quality in basic and applied science:
   Against the background of the challenges our societies face, the role of research, success in careers of researchers, and the quality of science impact are subject to redefinition. This strand delves into innovative approaches to assess research beyond traditional metrics. It is about recognising the multidimensional nature of research output and its impact. Ways of understanding the diverse contributions of researchers and their inter-disciplinary or atypical research careers are discussed in this strand.
3. Examining instruments in European research and innovation policy:
   The European research and innovation landscape is evolving rapidly as it seeks to meet the demands of changing times. Therefore, it becomes all the more crucial to make sense of the successes of instruments and initiatives and apply evaluation approaches fit for purpose in transnational settings. Lessons learned from previous Framework Programmes and insights from the interim evaluation of Horizon Europe are subject in this strand, but also other instruments and initiatives relevant or connected to research and innovation. Envisioning the future of European Framework Programmes and approaches, emerging evaluation requirements are being considered in this strand.
4. The emergence and consolidation of evaluation systems:
   Innovation systems vary significantly among countries, and the methods for holding policy interventions accountable also vary widely. In this strand, we delve into the challenges of establishing, strengthening, and consolidating evaluation systems at the national or regional level. This strand aims to exchange experiences and strategies related to capacity building. As all entities strive to navigate times of change, enhancing the culture and expertise in evaluation is crucial for the development of research and innovation ecosystems at all levels.
5. New methods, tools and implications for evaluation in the digital era:
   The digital age is reshaping not only how we evaluate research and innovation policies but also the research processes themselves and the way scientific knowledge is created. This strand explores both, the integration of digital technologies and AI into evaluation methodologies, and the changes that evaluators need to anticipate as the research system itself is transformed by the new opportunities offered by AI. In times of change, it is crucial to stay at the forefront of methodological advancements and address ethical considerations.

In Conclusion

The REvaluation Conference 2024 is your compass in these rapidly changing times. It is an opportunity to learn, collaborate, and innovate in the realm of research and innovation policy evaluation. Join us in Vienna to collectively navigate the challenges and seize the opportunities presented by this conference volume under the motto "Navigating Times of Change." Together, we can shape the future of research and innovation policy evaluation in this dynamic landscape.