Matrix Evidence

The Scottish Government, on behalf of the National Drugs Evidence Group, has published a review of the drugs evidence base. The review aims was to highlight where the evidence base is already strong, what the evidence tells us and where the knowledge gaps are in relation to Scotland’s National Drugs Strategy, The Road to Recovery: A New Approach to Tackling Scotland’s Drug Problem.

The review looks at both the published evidence and the policy context in which the strategy sits. It provides the link between the evidence around addictions and the wider social, health and economic context in which recovery occurs. The review set out to assess what we know about ‘what works’ in drugs recovery and identify the core questions that need to be answered when assessing the effectiveness and impact of The Road to Recovery.

Tags: evidence-based policy, systematic reviews
1 Comment »

The Australian Productivity Commission organised a roundtable discussion on the issue of strengthening evidence-based policy in 2009. Participants included government officials, academics, consultants and representatives of non-government organisations. Keynote addresses were presented by Dr Ron Haskins, Senior Fellow of the Brookings Institution, and Professor Jeffrey Smith of the University of Michigan.

The roundtable discussed the principles of the evidence-based policy movement and reviewed how well Australian use of evidence conformed to best practice. It then considered how to improve the availability of quality evidence, and reviewed possible institutional developments to embed good use of evidence more firmly into policy-making.

The roundtable proceedings have now been published. They include papers by the speakers and a summary of the key points covered in the discussion sessions, and a series of background papers prepared by Commission staff and provided to roundtable participants.

I was particularly drawn by a paper entitled ‘Evidence-based policy: summon the randomistas’ by Andrew Leigh of the Research School of Economics at the Australian National University

Tags: evidence-based policy, randomised controlled trials, systematic reviews
No Comments »

A group of highly respected scientists has published a paper in the Lancet calling for all governments to introduce mandatory use of research evidence to assess the likely effects of all public programmes before they are launched, and impact evaluations to monitor effectiveness post launch.

The paper cites as evidence of the political appetite for such a move the US government’s decision to ring-fence $1.1 billion for comparative research (including systematic reviews and clinical trials) as part of its $787 billion economic stimulus plan. Governments, they claim, have a political and ethical obligation to make effective decisions.

The paper goes on to cite the example of the Mexican Government which has passed legislation establishing a National Council for the Evaluation of Social Development Policy (CONEVAL). The Council is an independent public agency with a budget of around US$10 million. It not only conducts evaluations, but then publishes the results in the Official Journal of Evaluation.

Tags: evidence-based policy, systematic reviews
2 Comments »

An increasing part of the review work we do incorporates economic evaluations. People want to know, not just what works, but what the relative costs and benefits are likely to be when implementing effective interventions, medical or social.

A recent paper published by Rob Anderson, from Exeter University, looks at how useful systematic reviews of economic evaluations are in the context of health care. He argues that synthesising results across lots of different economic evaluations is risky for three reasons: (1) economic data is very context dependent, so it’s dangerous to generalise across settings; (2) assuming results can be generalised is crucial if the findings are going to inform policy decisions; and (3) decision analytic modelling, one of the main techniques economists use, is itself a synthesis tool, so doing additional systematic reviewing may be redundant.

The paper does not claim synthesising economic data is not useful or informative; it’s just that there may be more effective ways of doing it than systematic reviews.

Tags: economics, meta-analysis, systematic reviews
8 Comments »

Matrix Evidence exists to provide systematic and transparent reviews for decision makers. It should therefore come as no surprise that we are always pleased when we find debate about the merits of systematic reviews versus less rigorous efforts at evidence synthesis. Ben Goldacre’s Bad Science blog has recently featured just such a discussion. His piece demonstrates very clearly just what can happen when less systematic approaches to evidence synthesis are used to inform decision making. You can find the details by clicking here

Tags: systematic reviews
No Comments »