Quantitative indicators for qualitative assessment
Research at public institutions in the Netherlands is evaluated every six years. For these evaluations, the Strategic Evaluation Protocol (SEP) is applied. The SEP states (p. 6): "The main goal of a Strategy Evaluation Protocol (SEP) evaluation is to evaluate a research unit in light of its own aims and strategy". And (p. 7): "assessment is grounded in a narrative argument and supported by evidence of the scientific achievements of the unit".
The protocol does not prescribe in detail along what dimensions research is evaluated, but emphasizes that research units are free to focus on what is important to them. But such a story needs to be backed up with verifiable evidence. Here, the skills and knowledge of team Research Intelligence can help. We know what sources of data are available, how data can be interpreted, and what indicators are considered responsible. Also, we can present the data in interactive dashboards that allow you to inspect the data from different points of view.
Transparency and flexibility
Transparency and reproducibility are not only important qualities of research at VU, but also for research evaluation. In April 2024, VU was one of the first signatories of the Barcelona Declaration, committing to the advancement of open research information. Where possible, team Research Intelligence uses open data for analyses already. This ensures that conclusions are verifiable. Using open data also allows us to work flexibly, and to create insight tailored to a specific research unit.
Procedure and costs
We work in close cooperation with the research unit (or units) that will be evaluated. We always have an initial meeting to discuss expectations and possibilities. Subsequently, team Research Intelligence will send you a proposal describing the indicators that should be included, a time line with meetings to discuss progress, and other issues that may have been discussed. This will also include an estimate of costs. Costs will depend largely on the details, but to give an indication: costs for previous evaluations ranged from EUR 5,000 for a dashboard for a single research unit with few custom indicators, to EUR 18,000 for a consortium of departments, in which team Research Intelligence had a coordinating role.
