Although there has been huge growth in academic-policy engagement activity, there is no evidence that this is leading to an increase in evidence use by decision-makers. There is an almost total lack of empirical evidence about what works, and little clarity about what the goals of these activities might be. Future activities should focus on specific aims, and collect data about their impacts.
Over the past three years, Transforming Evidence have been collaborating with policy and evidence experts to gather empirical evidence about interventions and activities which aim to promote academic-policy engagement. We found that while this has a long history, with the oldest initiatives dating back centuries, there has been a huge expansion in the number of these initiatives in the last 15 years, crossing policy sectors and national boundaries.
What sorts of activities did we find? Over 513 organisations worldwide, carrying out 1923 individual activities were included in our dataset. We categorised the activities according to their aims – for example, whether they were seeking to disseminate research, build decision-makers skills, provide opportunities for professional networking and so on.
Why has there been this large expansion? Funders and researchers are increasingly keen to have, and to demonstrate their impact. Decision-makers are also keen to draw on evidence and expertise to support their work. In short, all academic-policy engagement is often felt and seen to be a good thing.
What have we learned about what works? Not much. Only 3% of the initiatives we found had publicly-available evaluations. So at present it is not possible to say much about what types of activities are more likely to lead to policy or practice impact, or to social changes. Perhaps even more concerningly, virtually no activities had a clear goal which would make it possible to see if they were having an impact.
What might this look like? One much-discussed benefit for academics who are seconded into government or who hold fellowships in policy settings, is becoming familiar with the policy environment and learning about how policy machinery works. Fellows also benefit by building networks and relationships which outlast their time as colleagues. We could map this as a theory of change, with the Fellowship as the intervention (Yellow), the immediate outcomes in green, and the longer term impacts in blue.
What we need to know is: Does this intervention lead to these short and long term outcomes? Are there other interventions which could more easily achieve these outcomes? What are the best interventions for different outcomes, and what are the costs and benefits of these approaches?
Can we do better? Almost certainly. We are a long way from being able to even pose these questions about existing activities, with the current mass of activity contributing to increased business and complexity, rather than to a robust evidence base about what works. The Transforming Evidence collaboration is conducting empirical research to try and build this evidence base – watch this space.
