Dr. Rossella Salandra and Prof. Ammon Salter, School of Management, University of Bath
Innovate UK’s goal is to drive productivity and economic growth by supporting businesses to develop and realise the potential of new ideas. They both connect organisations to help turn ideas into commercially successful products, and service and fund business and research collaborations to accelerate innovation and drive business investment into R&D.
Through the funding process Innovate UK aims to support ideas delivered by individuals or teams with the highest potential to develop innovation, in order to allocate funding to the strongest projects.
Yet, the evaluation of innovation projects remains ‘more art than science’. Funding proposals cover a wide range of technological areas, and it can sometimes be difficult to assess the potential technical and business value of the ideas, even by experts in the field. Assessors will tend to rely on their experience to assess the quality of the proposal and the anticipated outcomes. They may also display a range of implicit biases that may favour some individuals or groups rather than others, leading to a sub-optimal allocation of funding.
An ongoing project for the Innovation Caucus, led by a research team including the two of us and members of staff at Innovate UK, seeks to help Innovate UK better understand the assessment process.
Building upon the work already done by Innovate UK, and their deep knowledge of the organisation of the funding system, the project seeks to develop insights about potential sources of bias and about the effectiveness of the assessment system.
To date, we have worked collaboratively with Innovate UK to access and organise data on Innovate UK applicants and assessors, collected from the Innovation Funding Service (IFS).
What are the characteristics of Innovate UK applicants?
We investigated the features of individual applications / applicants (e.g., gender), and of the applying organisations (e.g., location and size) in relation to the type of call, acceptance rates, and average assessors’ scores.
While most applications are led by men, there was no evidence that women-led applications had a lower success rate. In terms of the geography of applications, there was a strong concentration from England, especially London and the South East. Again, we found no evidence of geographical preference in selection, as the success rate for applications from different Home Nations in the UK was roughly similar. When looking at firm size, we found that small firms were responsible for a large share of applications, and they also had a lower success rate. However, this might have been expected, given the administrative capabilities of large and medium size firms to develop applications.
What are the features of Innovate UK assessors?
We then considered various assessors-level factors, such as their gender and location. The assessors’ pool was found to mirror the imbalances in the applicants’ pool, with an over-representation of men and assessors from the South of England. Despite this, we found little evidence of bias against women-led applications. We also found that non-London based applications were scored more highly than London-based applications by London based assessors. This would suggest the current imbalances in the assessor population are not leading to any structural disadvantages for women or for applications from outside London.
What next?
The research opens up a range of questions that could spur further investigation. For example, at the level of the applicants, the current analysis focuses only on lead applicants. It would be interesting to examine the role of the entire application team and their characteristics. At the level of the assessors, accessing information on the workload of the assessors, and the timing of their assessments might provide insights into how assessments are shaped by other parts of the selection process.
Altogether, greater understanding of the evaluators and the factors that shape their assessment might help to develop richer and more refined approaches for the treatment of R&D grant applications.