Call

This strand focuses on developing effective approaches to accompany, monitor and evaluate research and innovation policies to support socio-technical transformation. We invite contributions that address the following areas:

• State-of-the-art evaluations: Presenting the latest advancements, progress, methodologies, and limitations in evaluating the transformation of socio-technical systems towards sustainability.
• New evaluation needs at the intersection of industrial policy and the transformation agenda: Acknowledging re-industrialisation as a new challenge for evaluating transformation and integrating industrial policy to sustainability trajectories.
• Integration of adjacent policy domains: Examining strategies and approaches to incorporate and evaluate the interplay between research and innovation policies and adjacent policy areas such as health, environment, mobility, energy, ensuring comprehensive evaluation of the transformational effects.
• Targeted evaluations of RTI missions: Sharing insights into evaluating specific missions or challenge driven policies, examining their effectiveness and impact.
• Monitoring national implementation of EU missions: Comparing and contrasting the national implementation of EU missions in different countries, highlighting best practices, challenges, and lessons learned.
• Assessing transformation policies: Exploring methodologies, indicators, and frameworks to assess the design, implementation and impact of policies driving transformation through research and innovation.
• Approaches to measuring social innovation within transformative R&I policies and projects.

This strand focuses on advancing research evaluation beyond quantitative indicators. We invite contributions that address the following areas:

• Evaluating research careers beyond one-sided frames of reference: Exploring innovative approaches to evaluating research careers, considering diverse aspects beyond traditional metrics.
• Assessing the diversity of researchers and their contributions: Examining methods and frameworks for assessing the diversity of researchers and acknowledging the different types of contributions they make.
• Holistic evaluation frameworks: Presenting innovative frameworks and methodologies that capture the multidimensional nature of research output and impact, including open science and societal, economic, and environmental dimensions.
• Exchange of knowledge and experience: Insights from the work of dedicated networks like the Coalition for Advancing Research Assessment

This strand focuses on evaluating the instruments and initiatives in research and innovation at European level. We invite contributions that address the following areas:

• Lessons learned and best practices from evaluating previous Framework Programmes: Sharing insights and experiences from evaluating past Framework Programmes, highlighting successes, challenges, and lessons learned.
• Interim evaluation of Horizon Europe: insights and reflections: Presenting findings from the manifold evaluations contributing to the interim evaluation of Horizon Europe, identifying key insights, challenges, and recommendations for improvement.
• Visions for the future of FP10 and emerging evaluation requirements: Engaging participants in an open dialogue to envision the future of the European Framework Programme and discuss emerging evaluation requirements for the next iteration.
• Assessment of EU funding programmes: Evaluating the effectiveness, efficiency, and impact of EU funding programmes, including Horizon Europe and other relevant initiatives, and identifying lessons learned for future programme development.
• Policy instrument evaluation: Exploring methodologies and approaches for assessing the performance of policy instruments, such as grants, loans, and collaborative research platforms, in achieving research and innovation policy objectives.

This strand focuses on monitoring and evaluation systems in contexts of newly developing or reforming evaluation cultures. It refers to monitoring and evaluation practices, challenges and results of so-called widening countries and countries with less developed R&I systems in Europe and beyond. We invite contributions that address the following areas:

• Evaluations to understand R&I policies, programmes, instruments and initiatives from widening countries. Presentation of current or recent evaluation studies.
• Sharing experiences, challenges, and best practices in building/reforming evaluation systems: Presenting case studies, challenges, lessons learned, and practical insights on building and strengthening evaluation systems in different contexts.
• Identifying future steps and collaborations for R&I evaluation capacity development: Sharing experiences, discussing strategies, policies, and collaborations to further enhance evaluation capacity in lower-performing innovation countries.
• Strengthening evaluation infrastructure: Discussing approaches to strengthen evaluation infrastructure, including access to data sources, the establishment of evaluation agencies, networks, and collaboration platforms, to support evidence-based policymaking.

This strand explores new methods and tools for evaluation, with a specific emphasis on all forms of digitalisation and artificial intelligence (AI). We invite contributions that address the following areas:

• Digital transformation in evaluation: Discussing the integration of digital technologies and data analytics in evaluation methodologies, including big data analysis, machine learning, and data visualisation.
• Empirical studies and case examples of AI in policy evaluations: Presenting real-world experiences and case studies that demonstrate the application of AI in policy evaluations in different thematic areas, highlighting successes, challenges, and lessons learned.
• Methodological and ethical considerations and challenges when using AI in evaluations: Examining methodological concerns, ethical considerations, and challenges associated with incorporating AI techniques into policy evaluations.

This is not an open strand in the narrow sense, but gives space to additional contributions with one of the following features:

• Your contribution can be cross-cutting as it discusses ethical implications and responsibility in evaluation systems connected to one or several strands.
• Your topic does not match at all? Convince us! Briefly argue, why you believe that your extended abstract is of significant relevance to the field and to evaluation in changing times, for example:
  • Significant methodological advances
  • Specific organisational and field knowledge
  • Findings from related scientific and policy fields
  • …
• You have suggestion for a different contribution than a paper presentation, like
  • inviting contributions to a special session within one of the strands (see next point "Extra"),
  • creating a panel discussion, or
  • hosting a workshop or another form of interaction.

Submissions can be dedicated to the below focus sessions, suggested by community members:

• Assessing transformation-oriented policies in the face of uncertainty & rapid change [under Strand 1]
  • Download: RE2024_SpecialSession_AIT.pdf
    
  • Contact: Anahi Montalvo-Rojo, AIT
    
• Mission (impossible): Challenges of Research and Innovation Policies in Translation from Policy to Practice [under Strand 1]
  • Download: RE2024_SpecialSession_IHS.pdf
    
  • Contact: Magdalena Wicher, IHS
• Advancing research assessment: the future already exists, yet is challenging to reach [under Strand 2]
  
  • Download: RE2024_SpecialSession_I&E.pdf
    
  • Contact: Leonie van Drooge, Impact&Evaluation