For anyone interested in evaluation of social policy, the US Government Accountability Office (GAO) is an extremely useful source of robust evidence.
The GAO was recently asked to take a look at an initiative delivered by the Coalition for Evidence-Based Policy. The Top Tier evidence initiative involved the CEP reviewing the evidence associated with 63 different interventions; of those 63, the CEP rated 6 as top tier, defined in Congressional legislation as ‘well-designed randomized controlled trials [showing] sizeable, sustained effects on important ….outcomes’.
The GAO concluded that RCTs are the best way of providing robust evidence of intervention effectiveness. However, they also noted that RCTs are often difficult to conduct in the context of social policy evaluation. For that reason, reliance on only evidence from RCTs would severely limit the evidence available to policy makers. Therefore the GAO concluded that several rigorous alternatives should be considered including quasi-experimental designs and statistical analysis of observational data.
Other important points raised by the GAO included: (i) the decision to adopt an intervention will be informed not just by evidence of effectiveness, but also by data on costs and benefits, and suitability to local communities, and (ii) improved evaluation would help identify effective interventions.