Advisory report 7: Conceptual framework for setting up KPI's in function of policy goals
On the first of March 2019, VARIO received a request for advice from Flemish minister Philippe Muyters to “set up a qualitative and measurable set of output parameters and KPI's for the benefit of the next Flemish Government, which express the Flemish ambition to be one of the European leaders in innovation".
Get in touch with:
Summary
In this advisory report, VARIO provides a conceptual framework for setting up KPI's and output parameters. The conceptual framework consists of five steps:
Step 1: The starting point should be a long-term strategy with the policy objectives of the Flemish Government. These objectives typically go beyond consecutive legislatures, are transversal and serve as an anchor point for the coalition agreement. (A limited number of) KPI's should be linked to these objectives, ideally with specific milestones/targets per legislature.
Step 2: The strategic policy objectives of the Flemish Government must be translated (not just adopted) into objectives per policy domain with associated KPIs.
Step 3: The objectives per policy domain should be translated into instruments/structures/actors/ programmes. KPIs and output parameters should then be linked to these at a disaggregated level. Several instruments/structures/actors/programmes contribute to the same objective.
In Step 1, Step 2 and Step 3 it is important that KPI's are complementary and lead to synergy; setting up one KPI should not compromise the achievement of another.
Step 4: Evaluations should be carried out at appropriate times (ex-ante and ex-post) as to whether/what the different instruments/structures/actors/programmes together contribute to the achievement of the objectives at the policy domain level. KPIs and output parameters are not objectives but a means to meet a higher goal.
Step 5: At appropriate times, it should be evaluated whether/how the different objectives per policy domain together contribute to the long-term strategy and objectives of the Flemish Government.
VARIO recommends that the current strong programme- and structure-driven evaluation culture is adjusted, and that more attention is paid to systemic evaluations (steps 4 and step 5). There is also a need for more standardised and systemic impact analyses.
In order to apply this conceptual framework at the level of policy areas and individual instruments/structures/actors/programmes, it is important to further develop the current monitoring tools through, among other things, an open data platform and better (IT) data infrastructure. In addition, the public administration itself must have enough substantive expertise to correctly interpret the data and evaluation reports and to translate them into policy.