Matrix Evidence

Although not widely reported in the press, we noted with interest more policy on the hoof from Nick Clegg at the 2011 LibDem September conference.

The idea of providing catch-up summer classes for children “most in need” prior to starting secondary school may have some intuitive appeal, but where similar schemes have been properly evaluated, questions have been raised as to their effectiveness. Is it a good idea to invest £50m of tax-payers money in a scheme with no proven efficacy?

One of the most well-known evaluation studies (amongst students of social policy at any rate) is the Cambridge Summerville Youth Study:

http://www.childtrends.org/lifecourse/programs/cambridge.htm

It was a community based program for children and adolescents in eastern Massachusetts in the 1950’s. Children in the program received a counsellor who visited with them and their family twice a month. These counselors are on call for problems that the family may have been having and referred children to a variety of different programs including tutoring, medical treatment, psychiatric treatment, summer camps, Boy Scouts, YMCA, or other community programs.

Unusually, the programme was robustly evaluated using a randomized controlled trail methodology. The results from a 30 year follow-up showed conclusively that the programme had no impacts on juvenile arrest rates measured by official or unofficial records. The programme also had no impacts on adult arrest rates. There were no differences between the two groups in the number of serious crimes committed, age at when a first crime was committed, age when first committing a serious crime, or age after no serious crime was committed. A larger proportion of criminals from the treatment group went on to commit additional crimes than their counterparts in the control group.

Other studies have come up with similar findings. Generally they conclude that programmes like this, unless they are of very high quality (well-staffed by trained professionals), tend to act as ‘bad behaviour academies’, where like-minded children get together to exchange ideas on how to be anti-social.

Without wanting to dismiss the idea of summer schools purely on the basis of a flawed programme implemented a generation ago, it would be nice to see a commitment to ring-fence part of the proposed £50m budget to some robust (RCT) evaluation.

Tags: Criminal justice, deprivation, education, evidence-based policy, randomised controlled trials
1 Comment »

Research summary outlining key findings from a review of the current context for geographically focussed community regeneration in Scotland, the impact of previous interventions, and future challenges for area based regeneration.

Main Findings:

• The approach since 1996 has been to use relatively modest programmes as ‘catalysts’ to test new ways of working and to develop appropriate local solutions to problems. But there is little evidence that this has brought about significant change in the way that mainstream funds are used.
• Throughout, the programmes have been carefully evaluated. But a recurring theme has been the lack of local data to set a baseline and to measure progress – and that there has not been an agreed standard set of measurable indicators.
• Community engagement has been an important part of the approach. But the increased link to wider approaches and to community planning appears to have made effective community engagement more difficult to sustain in some areas.
• Partnership working has been an important theme. More recent approaches have tied regeneration activity into community planning. There has been a growing focus on outcomes – with clearer and more comprehensive links between national and local outcomes.
• Across Britain there is a lack of solid evidence of the overall impact of geographically targeted programmes on multiple deprivation. Such evidence as there is suggests that the gap between the most deprived areas in Scotland and the rest has not closed in any substantial way.
• There has often been an imbalance between physical, social and economic programmes. They have been run by different organisations with different priorities.
• The moves in Scotland to more flexible approaches; a reduction in the number of funding streams (and a removal of much ring fencing) and an outcomes focused approach are mirrored in England and Wales.

You can find the full report at:

www.scotland.gov.uk/Publications/2009/07/07132920/0

Tags: deprivation, regeneration
No Comments »