The conference is dedicated to conceptual and methodological advances in R&I policy evaluation.
Core Strand #1 concerns all developments that have taken place and on all types of effects of R&I policies.
Strand #2 puts a spotlight on the effects of the COVID-19 crisis on R&I policy, in particular on awarding practices.
In Strand #3 we ask how to conceptualise and assess the resilience of R&I systems and their contributions to more resilient economies and societies.
Strand #4 deals with the nature and the evaluative challenges and practices of transformation-oriented R&I policy.
#1) Conceptual and methodological advances in R&I policy evaluation
In this core session of the conference we focus on methodological challenges as well as methodological advances in R&I policy evaluation in general.
The focus is not thematic, but conceptual and methodological and includes all sorts of effects of R&I policies.
Interesting questions to be addressed (but not restricted to) include the following:
- How can outcomes and impacts of R&I policy interventions be better conceptualised, traced and measured? How can evaluation methodologies be further developed to measure outcomes (instead of outputs only)?
- Which new indicators (and data sources) are available or needed to assess R&I policies beyond the traditional scientific impact?
- Which experiences with novel approaches can be shared?
- How to conceptualise and evaluate impact generation at the regional, national and European level?
- How to achieve more aligned European approaches and standards in R&I policy assessments and evaluation? How to achieve better comparability?
- How to make reasonable and effective use of new data sources and methods, such as – but not confined to – the application of Artificial Intelligence, machine learning and big data in R&I policy evaluation or the use of public registers and their combination?
We also encourage contributions on how foresight and forward-looking ex-ante assessments support not only the design and implementation of R&I policies but also their evaluation.
Furthermore, we encourage also contributions dealing with monitoring and impact of STI activities using granular data. Empirical contributions are welcomed be they based on advanced spatial- or micro-econometric methods, experimental designs, Big Data/Data Mining approaches, network analysis, etc.
Possible thematic examples include (among others):
- Causal analysis of STI initiatives/instruments
- Linked micro-data
- Challenge-driven STI initiatives (e.g environmental)
- Impact of digitalisation initiatives
- Effectiveness of R&D support instruments
- Analysis of science and education
#2) What can we learn from fast-track COVID-19 R&I policies?
Due to its urgency, the COVID-19 crisis has sometimes suspended or adapted traditional funding awarding practices. Call priorities were set quickly without intensive consultation processes, and application and selection processes were in some cases immensely accelerated.
Interesting questions to be addressed (but not restricted to the issue of ‘fast track’ developments) include the following:
- What effects did these changes have, e.g. on the design of funding programmes and calls?
- How did the new procedures affect the nature and content of proposals, the selection processes and in particular, the relationships between track record and proposal content or the type of risks taken?
- Did we witness the selection of more heterodox approaches? In other words: Did the induced changes support variation, or were the low-hanging fruits supported in the end. How have these process changes been perceived by those in charge of rewarding research funds?
- How were these initiatives received by researchers?
- What lessons can be derived both for future funding activity or in view of future crises? What will remain?
#3) Evaluation of the resilience of R&I systems and their contribution to public resilience
Session #3 focuses on conceptual, methodological and practical evaluation challenges as regards the
- protection of resilience of R&I systems
- and the contribution of R&I to protect public resilience in a broader sense.
Interesting questions to be addressed (but not restricted to) include the following:
- How vulnerable are R&I systems? Are there lessons to be learned from the COVID-19 crisis?
- How can resilience of R&I systems be assessed?
- What does technology sovereignty and access to critical technologies mean in terms of the resilience agenda and how can technology sovereignty and access to critical technologies be measured?
- Which experiences with novel approaches to address the evaluation of the resilience agenda can be shared?
- How can R&I contributions to more resilient economies and societies be measured and where are the evaluative system boundaries?
#4) Understanding and evaluating transformative R&I policy
A fundamental challenge for current and future evaluation activities is to understand how transformation-oriented policy-making is affecting the purpose, accountability and accomplishment of R&I. Thus, on one hand this session facilitates learning from
- early transformative R&I policy experimentation and showcases (rationales, drivers, knowledge basis, processes, formats and measurement of achievements)
- and from transformative policies of other policy domains (e.g. health, climate change, transport, energy, security, etc.)
On the other hand, we particularly aim to share practices of how to evaluate R&I policy demands stemming from the transformation agenda.
Interesting questions to be addressed (but not restricted to) include the following:
- What does successful transformational R&I policy mean?
- What are the specific challenges, bottlenecks and traps of transformation oriented R&I polices and which tensions emerge?
- How does transformative R&I policy interact with other policies?
- What can we learn from evaluations dealing with transformative R&I policy requirements?
- How can new questions related to transformation-oriented R&I policy be conceptually and methodologically approached?
- Where are the evaluative system boundaries (what to evaluate and where to stop when focussing on R&I policies)?
- Which are relevant measurement categories (processes, outputs, outcomes and impacts) for transformative R&I policy?
- What is the specific role of inter-disciplinarity and trans-disciplinarity (in research, innovation and evaluation) in the evaluation of transformative R&I policy? What competences are needed?
- How do new methodological approaches interact or contradict with ‘traditional’ measurements and what are the implications on a system’s level?
- How to identify and assess non-intended effects of transformative R&I policies?
- How to trace and measure rebound effects?
- What is the (changing) role of (legal/regulatory) frameworks?
- How does the role of funding and innovation agencies change?
- How are the impacts of policy on transformation framed at different levels and scales (e.g. broad challenges, concrete missions, combination of different impact dimensions, different territorial scales)?
- How should transformative city-based, regional, national and European R&I policies interact?
- What is the definition of European added value in transformation-focused R&I policies?