A couple of months ago, the Telegraph published a superb piece of shoddy journalism by claiming there were only 100 cod left in the North Sea. This came from the Sunday Times’ equally misguided claim that there were 100 adult cod left. This in turn was picked up by other mainstream media outlets, no doubt triggering a run on local fish and chip shops across the UK before our favourite fish was declared extinct. DEFRA (Department for Environment, Food and Rural Affairs) was quick to publish a release saying the Sunday Times was off by a staggering 21 million fish.

How on earth do respectable media companies like this get basic maths and science so wrong? Worse, why does this seem to be a systemic issue throughout all of media. Why do journalists struggle with the most basic understanding of numbers?

This has always frustrated me but after recently reading Ben Goldacre’s Bad Science, it became very apparent that this incompetence with mathematics had effects that stretched far beyond frustrating the reader. Much of the damage lies in the misunderstanding or underappreciation of the underlying statistics.

Yet a brief look at the history of statistics reveals it wasn’t always like this.

History of Statistics

While mathematics and philosophers have poured over numbers for centuries, probability studies arose alongside the rise in popularity of gambling. Statistics subsequently evolved as a way for astronomers to measure star movements and their error margins. Yet it took some time before mathematicians and sociologists made the leap and realized they could apply those same techniques more generally. Indeed, the word statistics comes from ‘information about the state‘.

Interestingly, when statistics were first applied to the general population to understand elements such as height distribution, what we would now call the “average” was viewed as the perfect human (Quetelet’s l’homme moyen), with the rest of us being “errors” on on either side of this. This was taken further as statistics became more broadly understood by academics of the time who started to question whether statistics actually affected determinism, free will and jeapordised the very existence of God (hint: it doesn’t. But I digress…). Suddenly people were fascinated that mathematics could describe (the then) modern society. Was crime caused by innate wickedness in individuals or was there a consistency in the probabilities of certain crimes occuring every year? In 1860, Prince Albert lamented that statistics may lead to “…the destruction of true religion, as it deprives, in man’s estimations the Almighty of his powers of free self-determination, making His word a mere machine.”

Statistics was seen as almost heretical in its power — a precursor to similar accusations leveled at modern day advances such as bioengineering, high-energy particle physics and artificial intelligence.

So what has happened in the 150 or so years since we started using statistics and probabilities to describe our lives? It seems that over time, as the media has evolved from the respectable broadsheets to red-top tabloids to celebrity gossip glossies and 24 hour online media, the need to create sensationalist headlines has pressured journalists into increasingly ridiculous claims.

Some can be reasonably innocuous — for example, the cod claim could damage people’s perception of the fish causing economic impacts on fisherman — but many more are a lot more dangerous (fatal even) in their misguided claims. If we are to believe the Daily Mail, pretty much anything and everything has either cured or caused cancer at some point during its print run. Endorsements of pseudo-medicinal products such as antioxidants and brain/body supplements are often found in a whole gamut of media (from reputable sources such as the BBC to the usual culprits).

These badly written articles are often the byproduct of either accidental misunderstanding or, far worse, deliberate misrepresentation by pharmaceutical companies, alternative therapists, nutritionists, homeopathists and the like.

As a quick aside, it is probably worth mentioning I’m not attacking alternative/complementary medicines in particular, just the misappropriation (or utter lack of) scientific data to back up claims of their effectiveness. In fact, my mum arguably co-wrote the first ever book on complementary medicine and the law back in 1996!

The migration of these spurious claims into mainstream media often comes from so-called Churnalism — the journalistic practice of rehashing press releases complete with any numerical claims. As this worldwide game of Chinese Whispers progresses, the numbers either get obfuscated, their meaning reinterpreted or downright altered to make more sensational headlines.

There is also a huge amount of psychology behind all of this. Humans are pattern-finders not calculators, we have an innate ability to understand proportions (Which tree has more food? Does that tribe have more warriors than us?). We are so good at it, we see patterns where there are none. We are inherently drawn towards things that reaffirm our beliefs and avoid those that don’t. We confuse correlation and causality (a pet hate of mine!). Worst of all, we struggle with an intuition of numbers.

Here’s a little test, read the paragraph below and have a little think:

 You want to randomly pick a certain number of people so that there is a 50% chance that two people will have the same birthday. How many people do you need to pick?

Well, if we wanted a 100% chance that two people had the same birthday, the number of people would obviously have to be 367 (366 days for all possible birthdates, including leap years + 1). So if we want half that probability?

Hands up who said 23? Twenty-three! If you’re anything like me, this makes no intuitive sense whatsoever but the maths add up. While understandable, this lack of intuition is no excuse for mainstream media outlets with a readership in the millions to consistently misrepresent scientific data.

As an example, in 2007 the Independent stated that cannabis was 25x stronger than the resin sold about 10 years prior (by THC concentration). In the 1980s, cannabis was 14x stronger in 1986 than in 1970. So, as Goldacre beautifully put in Bad Science:

“[This] sets you thinking. If it was fourteen times stronger in 1986 than in 1970, and it’s twenty-five times stronger today than at the beginning of the 1990s, does that mean it’s now 350 times stronger than in 1970? That’s not even a crystal in a plant pot. It’s impossible. It would require more THC to be present in the plant than the total volume of space taken up by the plant itself. It would require matter to be condensed into super-dense quark-gluon-plasma cannabis. For God’s sake don’t tell the Independent such a thing is possible.”

Or another similar example where the Mirror claimed that “Use of [Cocaine] by children doubles in a year” with the so-called doubling being utterly dependent on the way the Mirror interpreted the data. Firstly, within the government data this was based upon, the Mirror rounded children’s cocaine usage at 1.4% in 2004 down and 1.9% in 2005 up to 1% and 2% respectively to get their “doubling” figure. So the relative increase was actually 35.7% with the absolute risk increase being 0.5%! In short, from data in a survey of 9,000 children where 45 more kids said “Yes” to the question “Did you take cocaine in the past year?” led to a headline that proclaimed cocaine use had doubled!

Now, I’m not advocated or downplaying cocaine usage in children, nor am I saying an increase of 45 is insignificant, but sensationalist headlines like this cause public outcry and affect government policies at the highest levels.

What truly baffles me is that if you turn to the financial pages in many of these same papers, you are confronted with reams of raw statistical data. Turn to the sports pages and the fans are treated like intelligent readers with all the statistical information detailing the minutae of their favourite sports team they could possibly wish for. Yet turn to the science and technology pages and we’re all suddenly treated like we’re bloody idiots — important details are glazed over, explanations are dumbed down to the point of irrelevance and, worst of all, the misuse (or plain misunderstanding) of the data leads to public scares and a mistrust of scientific data.

In our age of increasing knowledge specialisation coupled with an innate desire for instant gratification, we must learn to both question what we read in the mainstream media and understand the meaning behind the numbers. We must learn to question the sensationalist headlines and understand the difference between true science and the quackery, Churnalism and straight-up bullshit* that pervades much of what we read.

Without this, we will stop trusting and grow cynical about the amazing work that people fair smarter than ourselves do — the doctors, rocket scientists, computer scientists, physicists etc. This are the same people who are dependent on public funding to further their research and as result (directly or indirectly) enrich all our lives.

Please, for the love of cod, don’t believe every statistic you read in the news…

Further reading

Ben Goldacre’s Bad Science is a superb place to start, but Critical Mass by Phillip Ball and Alex’s Adventures in Numberland by Alex Bellos are also very interesting reads that deal with the history of mathematics and numbers.

* As an aside, Goldacre has a great definition for bullshit and clearly differentiates it from lying.