- Deadline for abstract submission: May 15, 2024
- Notification of abstract acceptance: July 2024
- Conference dates: December 4-6 2024
- Full paper submission deadline: January 15, 2025
Call for Contributions
We invite researchers, practitioners, policymakers, and evaluators to contribute to the REvaluation Conference 2024, a premier international gathering for exploring cutting-edge approaches and methodologies in the evaluation of research and innovation policy and interventions. The conference aims to foster interdisciplinary discussions, exchange knowledge between researchers, evaluators, policymakers and funding agencies, and enhance collaboration in the evaluation community.
The REvaluation Conference convenes at the intersection of academic research and policy and practice knowledge. We welcome submissions in the form of academic contributions, policy contributions, case studies, and practical contributions from a wide range of topics related to the conference's five thematic strands (click arrows to expand):
1) Evaluation of transformation policies and dynamics in socio-technical systems
This strand focuses on developing effective approaches to accompany, monitor and evaluate research and innovation policies to support socio-technical transformation. We invite contributions that address the following areas:
- State-of-the-art evaluations: Presenting the latest advancements, progress, methodologies, and limitations in evaluating the transformation of socio-technical systems towards sustainability.
- New evaluation needs at the intersection of industrial policy and the transformation agenda: Acknowledging re-industrialisation as a new challenge for evaluating transformation and integrating industrial policy to sustainability trajectories.
- Integration of adjacent policy domains: Examining strategies and approaches to incorporate and evaluate the interplay between research and innovation policies and adjacent policy areas such as health, environment, mobility, energy, ensuring comprehensive evaluation of the transformational effects.
- Targeted evaluations of RTI missions: Sharing insights into evaluating specific missions or challenge driven policies, examining their effectiveness and impact.
- Monitoring national implementation of EU missions: Comparing and contrasting the national implementation of EU missions in different countries, highlighting best practices, challenges, and lessons learned.
- Assessing transformation policies: Exploring methodologies, indicators, and frameworks to assess the design, implementation and impact of policies driving transformation through research and innovation.
- Approaches to measuring social innovation within transformative R&I policies and projects.
2) Redefining success and quality in basic and applied science
This strand focuses on advancing research evaluation beyond quantitative indicators. We invite contributions that address the following areas:
- Evaluating research careers beyond one-sided frames of reference: Exploring innovative approaches to evaluating research careers, considering diverse aspects beyond traditional metrics.
- Assessing the diversity of researchers and their contributions: Examining methods and frameworks for assessing the diversity of researchers and acknowledging the different types of contributions they make.
- Holistic evaluation frameworks: Presenting innovative frameworks and methodologies that capture the multidimensional nature of research output and impact, including open science and societal, economic, and environmental dimensions.
- Exchange of knowledge and experience: Insights from the work of dedicated networks like the Coalition for Advancing Research Assessment
3) Examining instruments in European research and innovation policy
This strand focuses on evaluating the instruments and initiatives in research and innovation at European level. We invite contributions that address the following areas:
- Lessons learned and best practices from evaluating previous Framework Programmes: Sharing insights and experiences from evaluating past Framework Programmes, highlighting successes, challenges, and lessons learned.
- Interim evaluation of Horizon Europe: insights and reflections: Presenting findings from the manifold evaluations contributing to the interim evaluation of Horizon Europe, identifying key insights, challenges, and recommendations for improvement.
- Visions for the future of FP10 and emerging evaluation requirements: Engaging participants in an open dialogue to envision the future of the European Framework Programme and discuss emerging evaluation requirements for the next iteration.
- Assessment of EU funding programmes: Evaluating the effectiveness, efficiency, and impact of EU funding programmes, including Horizon Europe and other relevant initiatives, and identifying lessons learned for future programme development.
- Policy instrument evaluation: Exploring methodologies and approaches for assessing the performance of policy instruments, such as grants, loans, and collaborative research platforms, in achieving research and innovation policy objectives.
4) The emergence and consolidation of evaluation systems
This strand focuses on monitoring and evaluation systems in contexts of newly developing or reforming evaluation cultures. It refers to monitoring and evaluation practices, challenges and results of so-called widening countries and countries with less developed R&I systems in Europe and beyond. We invite contributions that address the following areas:
- Evaluations to understand R&I policies, programmes, instruments and initiatives from widening countries. Presentation of current or recent evaluation studies.
- Sharing experiences, challenges, and best practices in building/reforming evaluation systems: Presenting case studies, challenges, lessons learned, and practical insights on building and strengthening evaluation systems in different contexts.
- Identifying future steps and collaborations for R&I evaluation capacity development: Sharing experiences, discussing strategies, policies, and collaborations to further enhance evaluation capacity in lower-performing innovation countries.
- Strengthening evaluation infrastructure: Discussing approaches to strengthen evaluation infrastructure, including access to data sources, the establishment of evaluation agencies, networks, and collaboration platforms, to support evidence-based policymaking.
5) New methods, tools and implications for evaluation in the digital era
This strand explores new methods and tools for evaluation, with a specific emphasis on all forms of digitalisation and artificial intelligence (AI). We invite contributions that address the following areas:
- Digital transformation in evaluation: Discussing the integration of digital technologies and data analytics in evaluation methodologies, including big data analysis, machine learning, and data visualisation.
- Empirical studies and case examples of AI in policy evaluations: Presenting real-world experiences and case studies that demonstrate the application of AI in policy evaluations in different thematic areas, highlighting successes, challenges, and lessons learned.
- Methodological and ethical considerations and challenges when using AI in evaluations: Examining methodological concerns, ethical considerations, and challenges associated with incorporating AI techniques into policy evaluations.
0) Strand X
This is not an open strand in the narrow sense, but gives space to additional contributions with one of the following features:
- Your contribution can be cross-cutting as it discusses ethical implications and responsibility in evaluation systems connected to one or several strands.
- Your topic does not match at all? Convince us! Briefly argue, why you believe that your extended abstract is of significant relevance to the field and to evaluation in changing times, for example:
- Significant methodological advances
- Specific organisational and field knowledge
- Findings from related scientific and policy fields
- You have suggestion for a different contribution than a paper presentation, like
- inviting contributions to a special session within one of the strands (see next point "Extra"),
- creating a panel discussion, or
- hosting a workshop or another form of interaction.
Calls for Special Sessions within the 5 strands
Submissions can be dedicated to the below focus sessions, suggested by community members:
- Assessing transformation-oriented policies in the face of uncertainty & rapid change [under Strand 1]
- Mission (impossible): Challenges of Research and Innovation Policies in Translation from Policy to Practice [under Strand 1]
- Advancing research assessment: the future already exists, yet is challenging to reach [under Strand 2]
- Extended abstracts should be submitted via email@example.com by May 15, 2024.
- Extended abstracts should clearly indicate the relevance to one of the five thematic strands in 2-4 pages. If a contribution does not fit into any of the strands, please convince us in a few sentences how it is yet relevant in the context of the conference.
- Extended abstracts for all types of contributions (academic papers, policy papers and case studies) should be prepared following the provided conference template and guidelines.
- Submissions will undergo a peer-review process conducted by an international panel of academic and evaluation experts.
- Authors of accepted extended abstracts will have the opportunity to present their work in presentation sessions or poster sessions.
- Accepted submissions can be included in the conference proceedings (subject to author consent). The REvaluation Conference Proceedings will be published as a special issue of the fteval Journal for Research and Technology Policy Evaluation, covering both, academic and practical paper contributions.
- In case you would like to contribute with/to workshops, a panel discussion, or a dedicated session within one of the strands, please directly contact the organisers (firstname.lastname@example.org).
We look forward to receiving your valuable contributions!
Isabella Wagner, Austrian Platform for Research and Technology Policy Evaluation (fteval)
Philippe Larédo, Institut Francilien Recherche Innovation Société (IFRIS)
Jakob Edler, Fraunhofer Institut für System- und Innovationsforschung
Wolfgang Polt, Joanneum Research, POLICIES
Klaus Schuch, PROFEEDBACK Project, COST Action CA20112