Getting Slack with the figures

The Mail today takes a brave and commendable stand in its leader column against the Home Office's 'record of massaging figures'. It's referring to Jacqui Smith setting a new police target for increasing the public level of confidence in the police force. Elsewhere today, the Mail rages against the Government for 'cherry picking' data in what it calls a 'spin row'. So, let's have a quick look at how strong the Mail's commitment to statistical integrity is, huh?

In fact, let's start with the legendary James Slack's version of the Smith/police confidence story: Only 38% have any faith in our police... so Jacqui Smith is promising to scrap all their targets. At least, that's how the headline reads now. It used to read like this:



...but for some reason the headline was changed from the national average (46%) to use the single lowest figure for one force (38%, Lincolnshire). Some might say that using the Lincolnshire figure in the headline to say 'Only 38% have any faith in our police', is misleading. Some might say that deliberately changing an accurate headline to a more inaccurate but coincidentally more dramatic one suggests a kind of penchant for sensationalism which doesn't quite square with the Mail's occasional condemnations of politicians and their disgusting spin.

James Slack, it seems, is having a field day with statistics (something which may not be much of a surprise to people who read 5cc's regular posts about him). He lurrrves a good stat, does Slack. His other article today is a masterpiece: A decade of delinquency: Teen robberies, violence and drug crime soar to record levels.

'Record levels' seems to be something of a stretch, possibly because 'record levels' is just something journalists have a habit of throwing into articles in the same way as other people say 'fuck' or 'err...'. So don't be too hard on Slack, it seems to be just a kind of journalistic tic that doesn't actually mean anything. Let's examine these claims by looking at the figures he's taking it from (pdf file).

The headline claims that 'teen robberies' have reached 'record levels'. Slack points out that such robberies have 'surg[ed] by 29 per cent between 2005 and last year to 6,669'. Firstly, it's actually 6,699 (30 higher than Slack claims!), a typo which made it difficult to find, but what's interesting is the 'offence trends' table on page 15 of the report (18 of the pdf). This shows that, while it's true that this is higher than 2004/5 when the report started, it's actually gone down from 6,855 in 2006/7. Now, it seems to me that if robberies are slightly fewer in 2007/8 than they were in the previous year, it might not be quite correct to say that this was a 'record level', unless 'record' can somehow mean 'the second highest out of four data points', which runs contrary to my understanding.

His second headline claim suggests that 'violence' has reached record levels, a claim he expands on in the article:

Offences of violence against the person surged by a fifth from 44,988 to 53,930

Again, true for the two years he's comparing, but with the number-recognition skills gifted to me by my l33t Grammar School education, I can confirm that 53,930 is actually lower than 56,226 (the figure for the previous year, 2006/7). It's also lower than 54,661, which was the figure for 2005/6. So, in the four years recorded by this survey, the 'record levels' figure Slack boasts about in the headline is actually only the third-highest number. Still, if you remove half the data and pretend that most of the time between 2005 and 2007 didn't happen, it's really quite a shocking 'record', eh?

To give him some credit, drug offences for 2007/8 were actually higher than any of the three previous years. Of the 9 categories of offence in that table, only two reached their highest in the most recent year (drug offences and breaches of statutory order), while two categories actually show record lows (motoring offences and burglary). By my reckoning, anyway, that makes the headline claim accurate on one out of its three claims.

Statistics are fucking great fun like that. Here's an alternative headline and introduction to the article you could make from the same figures, which, while being utterly stupid, ridiculously selective with the data, and completely misleading, is still marginally more accurate than the Mail's own effort (as a result of not making two demonstrably false claims in the headline). I've taken the first three paragraphs of Slack's piece and replaced a few words:

A year of tranquility: Teen burglaries and motoring offences plunge to record lows

A decline in crime credited to Labour's 'year of juvenile responsibility' has seen a huge decrease in youth robbery and violence.

The number of offences committed by youngsters aged ten to 17 in the last year was 277,986 - a decrease of 17,143 (5.8%) from 2006/07.

A report from the Youth Justice Board yesterday charted robberies falling by 2.2 per cent between 2007 and last year to 6,669, including just 61 carried out by children aged eleven, a significant decrease from the 73 recorded the previous year.

Picking the right base year for comparison is one of the key weapons in this kind of statistical spin; Slack goes for 2004/5 in this case (the first year the figures were collected), but is there any real doubt that he'd be comparing 2007/8 to 2006/7 if those were the figures showing a massive increase? Of course, after reading this, I find out that it doesn't fucking matter what the figures say. Here's a comment on the Mail article from 'Gjones' in Nuneaton:

I seriously question the accuracy of YJB figures. The YJB's board members are selected by Jack Straw, the Secretary of State. This article picks out the negative statistics, but much of those published in the report show an improvement upon previous years - an improvement I don't believe in.

There you have it. If the numbers show an improvement, it'll be because the figures have been altered politically. This is an interesting claim, because the YJB report itself actually also uses 2004/5 as its base year for a lot of the comparisons in the text summaries rather than doing it year-on-year, which would make the figures look better. Despite this, the comments like Gjones' suggest that people will believe the stats that back up their preconceptions. If the numbers show shit is getting worse, then they're probably right! Which brings us full circle to the Mail's epic bit of win-win bet-hedging in its leader:

[Smith's] apparent conversion followed a shocking poll which showed that fewer than half of us had confidence in the police to deal with crime and disorder.

So from now on, the only target would be to get this approval rating up to more than 60 per cent.

Given the recent Home Office record of massaging figures, it would be impossible to have any confidence in the integrity of the poll next time it is conducted.

Right now the figures on police confidence are a 'shocking poll', one you can confidently use to base headlines around. If it reaches Smith's target levels, we'll know it's bollocks and lies. If it doesn't, its integrity will have been preserved.

Isn't it great to finally read a paper that hates spin?