Matrix Insight

The aim of the evaluation was to gather a wide range of evidence of the impact of EU support actions on Research Infrastructures in Europe. The overall objectives of the evaluation were to:

• Assess the pertinence of the EU support schemes and the Added Value of European action;
• Gain an overview of the impact that the EU support actions have had on scientific communities, re-search infrastructures, research policy, the economy, industry and wider society; and
• Analyse the structuring effect of support actions with regards to the European Research Area and provide Commission with recommendations for further Community actions regarding RIs.

To effectively address these objectives, the study adopted a before-after evaluative framework for the systematic assessment of impact and pertinence. A mix of methods for the collection and analysis of evidence were used. The study involved a Delphi survey; Rapid Evidence Assessment of existing literature; an on-line survey of project coordinators and participants; a detailed review of project descriptions of work; interviews with stakeholders; and 30 structured case studies involving 176 interviews. Statistical analysis from the case study sample and project survey data allowed the drawing of conclusions about the impacts of the programme as a whole.

Using multi-criteria analysis, the evaluation provided robust evidence that scientific impacts and impacts on Research Infrastructures had been achieved but that the evidence around wider societal and economic impacts was weak.

The study has informed long-term and short-term RI strategy at European level and has already led to changes in assessment criteria used for proposals funded under FP7.

Matrix undertook an analysis of data collected as part of an Ex Post evaluation and Impact Assessment of projects which had been completed under the FP5 “Competitive and Sustainable Growth” programme. The study looked specifically at projects pertaining to Key Action 1 and New Materials which related to programmes in the area of materials research and industrial processes and organisation. The assignment consisted of an extensive statistical analysis of data collected from around 800 different projects, with a focus on estimating longer term impacts of the projects and identification of correlations between success/failure and key features of the projects. The assignment also included a qualitative analysis in conjunction with the statistical analysis.

Matrix led an evaluation of the impact of the ERA-NET scheme and related ERA-NET actions under FP6 in collaboration with Rambøll.
The aim was to understand to what extent this support had an effect on the landscape of publicly funded national/regional research programmes and whether it had a structuring effect in certain sectors or countries with a view to formulating future policy.
Matrix adopted an evaluative framework for the systematic assessment of impacts generated by the scheme ex-post of implementation. We used a mixed methods approach for data collection and analysis integrating both qualitative and quantitative methods. The evidence consisted of primary data collected through two extensive quantitative surveys of ERA-NET coordinators and participants, as well as face-to-face and telephone interviews with stakeholders 156 stakeholders in 15 countries. In addition, a number of secondary data sources were used. This information combined provided the basis for various impact, economic and descriptive network analyses.

Matrix was commissioned by DG JLS of the European Commission to assess the effectiveness of EU Member State practices in the identification, tracing and confiscation of criminal assets. Matrix undertook a literature review on primary sources and cited documents from key correspondents, as well as a survey among stakeholders in all Member States.

This project demonstrates Matrix’s experience of working with stakeholders from multiple European countries and coordinating a project across the EU. It illustrates our understanding and knowledge of organisational arrangements, technology and skills involved in identifying, tracing, freezing and confiscating criminal assets; and awareness of the threats to successful performance in this field especially where international inter agency cooperation is required.

This exploratory study for DG MARKT investigates evidence of price and service differentiation in four sectors across the Single Market in order to trace the economic drivers of cross-border differentiation and in view of the implementation of the Services Directive by end 2009. The study consists of interviews with companies, trade associations, national authorities and consumer associations to typologise drivers of differentiation and investigate their economic implications. The aim of the study is to provide input into guidelines for the Member States in their transposition of Article 20(2) on non-discrimination.

Matrix carried out a study of customs controls of drug precursors in the 27 Member States of the European Union plus Croatia on behalf of DG TAXUD. The aim of the study was to explore the situation in Member States Customs administrations with regard to drug precursor controls and to identify appropriate solutions that could be employed to cut down the supply of precursors used for the illicit manufacture of Ecstasy and amphetamines in the Community.

We conducted surveys in all Member MS Customs Administrations and Croatia, by air, land, rail and sea, and field visits to two-thirds of the countries to collect additional information and to clarify survey responses. This determined the capacity of different Customs administrations to detect high risk consignments in general and of carrying out customs controls on drug precursors in particular as well as to assess the political priority of doing so and actual levels of enforcement. Based on the findings from the study, Matrix drafted an EU-wide strategy on drug precursors.

This project demonstrates Matrix’s experience of working with stakeholders from multiple European countries and coordinating a project across the EU. It illustrates our knowledge of law enforcement in EU countries and expertise in the collation and analysis of large volumes of survey data and statistics. This study also demonstrates our experience of assessing the actual levels of law enforcement across a number of countries.

Matrix was commissioned by DG JLS to examine the extent of NPO abuse for financial criminal purpose, and provide a more detailed analysis of the most frequent and serious criminal activities both in terms of volume, value and associated harms.
Evidence was collected from written documents and from experts to analyse the most frequent and serious types of financial criminal activity in the non-profit sector; analyse the volume and value of such offences where possible; provide a general estimation of total annual costs (direct and indirect) in the non-profit sector at EU-level caused by financial criminal abuse of NPOs; and identify focal points to be addressed at policy and other levels (such as appropriate elements of self or public regulation or law enforcement investigation) that could help drive policy responses to reduce NPO vulnerability to financial criminal abuse. The methodology included: interviews with experts, desk research, and a Delphi study with sample countries.
This project demonstrates Matrix’s experience of working with stakeholders from multiple European countries and coordinating a project across the EU. It illustrates our understanding and knowledge of financial crime, fraud, money laundering and terrorism, policing, counter-terrorism and the governance of NPOs in these respects.

The European Trade Union Institute provides a range of services (training courses, research publications, technical advice etc.) to the European Trade Union Confederation and by extension national confederations and European industry federations. It also acts as a resource centre for European policy stakeholders, researchers and technical institutes – and is an important player on the European arena for the trade union movement. The purpose of the Institute is to provide support for mobilising the European trade union movement and developing a European trade union identity.

The evaluation pointed to several things that could increase the impact of the work of the Institute, particularly by giving them feedback on what the trade union movement needs to participate in Social Dialogue and how to better engage with the trade union movement. By following Matrix recommendations, this project will enable the Institute to make decisions that could make it a better world for millions of workers by anything from health and safety issues to fair employment contracts and labour market equality.

In the UK, Matrix are part of a framework contract for the National Institute for Clinical Excellence (NICE) which aims to train interested international stakeholders (including EU Member States) in different aspects of the NICE methodology including assessment of cost-effectiveness of public health interventions.

In light of new staff regulations of the European Communities introduced in 2004 Matrix were asked to undertake a comparative study of the remuneration of officials of the European institutions, an update on a study completed in 1999. A number of public, private and international organisations were asked to participate in the study, providing data on remuneration structures, job descriptions and detailed salary data related to generic job titles.

Matrix are part of a consortium with Rambøll and Eureval commissioned by the Secretariat General to evaluate the system of EU decentralised agencies involving all current and past agencies. The results of this evaluation will feed into a strategic review of the agency system and its further development. Matrix are responsible for part of the Agency visits, a survey to the Management Boards of all 26 Agencies, as well as analysis and reporting. This evaluation is providing Matrix with unique insight into the EU Agency system and ways in which Agencies can increase their impact, effectiveness and efficiency.