Matrix Evidence

It’s that time of year again – British Crime Survey data are released, and the media misinterpret them. This time the problem is that old chestnut ’statistical significance’. Journalists have looked at the figure for all crime, seen that the survey reports a rise of 4%, and then written their copy. A typical line, this one from the BBC is ‘ Separate figures from the British Crime Survey (BCS) – measuring people’s experiences of crime – suggest crime went up by 4%. Not wanting to single out the BBC, the Telegraph did no better, reporting that ‘the British Crime Survey showed an increase’ [in crime]. The Telegraph went one better though. Having failed to interpret the BCS data correctly, their article goes on to cast doubt on the survey data (collected via face-to-face interviews with over 40,000 people), preferring instead to rely on what we can only characterise as ‘common sense’: ‘People are not stupid. They know that it stretches credulity for official figures to suggest that 2011 was a good year for crime-fighting.’. Who needs data when we can just rely on people’s impressions?

Top marks to the Guardian on this occasion though, who correctly reported that the survey figures show ‘no statistically significant change’ in overall crime.

Is the concept of statistical significance really so difficult to grasp that it eludes the cream of British journalism?

Tags: Criminal justice, journalism, media
No Comments »

Although not widely reported in the press, we noted with interest more policy on the hoof from Nick Clegg at the 2011 LibDem September conference.

The idea of providing catch-up summer classes for children “most in need” prior to starting secondary school may have some intuitive appeal, but where similar schemes have been properly evaluated, questions have been raised as to their effectiveness. Is it a good idea to invest £50m of tax-payers money in a scheme with no proven efficacy?

One of the most well-known evaluation studies (amongst students of social policy at any rate) is the Cambridge Summerville Youth Study:

http://www.childtrends.org/lifecourse/programs/cambridge.htm

It was a community based program for children and adolescents in eastern Massachusetts in the 1950’s. Children in the program received a counsellor who visited with them and their family twice a month. These counselors are on call for problems that the family may have been having and referred children to a variety of different programs including tutoring, medical treatment, psychiatric treatment, summer camps, Boy Scouts, YMCA, or other community programs.

Unusually, the programme was robustly evaluated using a randomized controlled trail methodology. The results from a 30 year follow-up showed conclusively that the programme had no impacts on juvenile arrest rates measured by official or unofficial records. The programme also had no impacts on adult arrest rates. There were no differences between the two groups in the number of serious crimes committed, age at when a first crime was committed, age when first committing a serious crime, or age after no serious crime was committed. A larger proportion of criminals from the treatment group went on to commit additional crimes than their counterparts in the control group.

Other studies have come up with similar findings. Generally they conclude that programmes like this, unless they are of very high quality (well-staffed by trained professionals), tend to act as ‘bad behaviour academies’, where like-minded children get together to exchange ideas on how to be anti-social.

Without wanting to dismiss the idea of summer schools purely on the basis of a flawed programme implemented a generation ago, it would be nice to see a commitment to ring-fence part of the proposed £50m budget to some robust (RCT) evaluation.

Tags: Criminal justice, deprivation, education, evidence-based policy, randomised controlled trials
1 Comment »

Home Secretary Theresa May has asked National Statistician, Jil Matheson, to review how crime statistics are collected and published. The review aims to improve transparency and public trust in the numbers.
In response, we have published a short guide to the statistics as an example of what might be done to help the public gain a better understanding of crime and the criminal justice system.

Using the government’s own British Crime Survey and police recorded crime data, our booklet reports that crime in England and Wales fell by 50% between a peak in 1995 and 2009/10—nearly 10 million fewer crimes were committed in 2009/10 compared to 1995. The downward trend is continuing despite the recession, which many experts predicted would see a rise in the figures.

The Matrix summary also shows just how the police fare when it comes to detecting crimes. In 2009/10, 1.3 million offences were brought to justice, a rise of 19% on 1999/00—and this at a time when the numbers of offences being committed was falling. But still fewer than one in three crimes reported to the police end up with a suspect being identified. For all crimes, not just those reported to the police, detection rates fall to less than 13%.

Also striking is the fact that young people under the age of 20 account for a huge proportion of crime in England and Wales. Nearly 6% of all 17-year-olds have been found guilty or cautioned for committing offences. At a time when local authorities are looking to make savings by cutting services to young people, these figures suggest they might want to consider the impact cutting youth services may have on local crime rates.

Tags: Criminal justice
9 Comments »

I came across this on the intriguingly entitled Jack of Kent Blog, providing a skeptical and liberal view on public debates and legal issues.

The Jack of Kent Blog is written by Allen Green, a freelance legal and policy writer working in London.
This particular post explores definitions of policy, calling on such diverse sources as Samuel Johnson (“the art of government”) and RA Butler (“the art of the possible”), although I have always been lead to believe the latter was a quote from Otto Von Bismarck.

Either way, it’s an interesting blog, and well worth the visit to anyone interested in the art, or otherwise, of politics.

Tags: Criminal justice, evidence-based policy
No Comments »

‘Random attacks by thugs every 30 seconds as stranger assaults soar in binge Britain’. Thoughtful analysis of the crime statistics by Daily Mail journalists provides yet another eye catching headline. The press in general have a dreadful record when it comes to the objective analysis of crime statistics. My particular favourite was criticism of the British Crime Survey (BCS) not including murder victims; the sense of outrage was only slightly tempered when it was pointed out that the BCS is a self-report survey!

However, help is at hand. A group of reporters in the US have published a handy guide to help our friends in the media get to grips with the complexities of crime statistics. Entitled Understanding Crime Statistics: A Reporter’s Guide, it sets out some of the basic principles in the effective reporting of crime data. Let’s hope their colleagues on this side of the pond get the message (and a copy of the book).

On the subject of crime statistics, an online journal, the Gotham Gazette, has provided an interesting analysis of why crime figures have reduced dramatically in New York. Whilst short on definitive answers, an increase in the use of stop and search has been suggested by some to have reduced gun carrying. Civil liberties campaigners are not impressed with the five-fold increase in use of powers to stop and search since 2002.

Another site keeping a watchful eye on the use and misuse of crime figures by politicians in the US is Politifact, a site run by the St Petersburg Times.

Tags: Criminal justice, evidence-based policy
No Comments »

Drug policy and evidence have been a very hot topic in the UK over the last few days. The Centre for Crime and Justice published details of a lecture Professor David Nutt, chair of the Advisory Council on the Misuse of Drugs gave some months ago. In the lecture, Professor Nutt suggested the government’s drug policy was not entirely consistent with the available evidence on drug related harms.  As a consequence, the Home Secretary asked Professor David Nutt to resign. Not surprisingly, the scientific community have largely cried ‘foul’; the actions of the Home Secretary have been seen by some as not entirely consistent with a commitment to evidence-based policy. For more on this story, click here.In the meantime, in the US (I’m at the American Society of Criminology conference in Philadelphia this week) the debate on just how effective the ‘war on drugs’ has been continues. Some politicians are suggesting that until the US decriminalises drug possession, they will not be able to effectively address their prison overcrowding problem, and the huge expense their criminal justice system incurs. I came across a very interesting article on CBS News that explores the arguments, and looks at the impact that decriminalising all classes of drugs has had in Portugal.  

Tags: Criminal justice, evidence-based policy
No Comments »

This recent report published by the National Institute of Justice (NIJ) in the US summarises the available research evidence on domestic violence. The intended audience is criminal justice practitioners, in particular police officers, prosecutors and judges. You can download a copy of the report here.

The report covers both the kind of rigorous empirical studies typically funded by the NIJ, and less rigorous sources such as newspaper investigations into prosecution practices in the US. The implications discussed in the report are based on available evidence, so are not a comprehensive listing of all policies and practices widely recognized as being effective.

Of course, the usual caveats apply when drawing conclusions from one territory and applying them to another. That said, the report makes for interesting reading.

For a response to the report from a policing perspective, have a look here.

Tags: Criminal justice
No Comments »