Mainstream Media and Maths: We’re All Going to DIE!

A couple of months ago, the Telegraph published a superb piece of shoddy journalism by claiming there were only 100 cod left in the North Sea. This came from the Sunday Times’ equally misguided claim that there were 100 adult cod left. This in turn was picked up by other mainstream media outlets, no doubt triggering a run on local fish and chip shops across the UK before our favourite fish was declared extinct. DEFRA (Department for Environment, Food and Rural Affairs) was quick to publish a release saying the Sunday Times was off by a staggering 21 million fish.

How on earth do respectable media companies like this get basic maths and science so wrong? Worse, why does this seem to be a systemic issue throughout all of media. Why do journalists struggle with the most basic understanding of numbers?

This has always frustrated me but after recently reading Ben Goldacre’s Bad Science, it became very apparent that this incompetence with mathematics had effects that stretched far beyond frustrating the reader. Much of the damage lies in the misunderstanding or underappreciation of the underlying statistics.

Yet a brief look at the history of statistics reveals it wasn’t always like this. Continue reading

Homosexuality: Nature’s True Altruists?

A few months ago, I was talking with a friend about Universal Darwism and the power of genetics, and without hesitation she asked, “Well, what about homosexuality? Surely that would have been removed from the gene pool centuries ago?”. As the evolutionary pragmatist, I completely agreed with her. If being gay was indeed nature over nurture, it would have been removed from the gene pool millenia ago, but the liberal in me felt that homosexuality must be genetic (as opposed to a life choice etc.). This conflict (along with the obvious observation that homosexuality is still pervasive in society) had me utterly stumped.

Let’s tackle the problem of evolving homosexuality out of the gene pool first — why did I believe that homosexuality should have been breed out? Imagine two groups, Group A, which makes up 99.9% of the population and Group B, which makes up the remaining 0.1%. Now imagine that Group B is a mere 1% more efficient at reproducing than Group A. The chart below shows how the two populations would progress (as a percentage of the total population) if Group A had 2 children each and Group B at 2.02 (1% more).

Unbelieveably, despite only making up 0.1% of the initial population, it takes as little as 700 generations for Group B to equal Group A and under 1,400 for their positions to be reversed! Anyone with a financial/economic background will appreciate the power of compound interest here, the same phenomena is exacerbated when applied to evolution, given both the size of the populations and years over which we look at the effects.

So what does this have to do with homosexuality? Simple — homosexual partners cannot reproduce (naturally), therefore cannot pass their “gay gene” down to their children. So, as in the example above, if a 1% decrease in reproductive capability leads to practical extinction within 1,400 generations, how has homosexuality (with a 100% decrease in reproductive capability!) not been erased after 50,000 years? Continue reading

The Unimportance of Immediacy

I’ve barely touched Twitter in the last 3 months — it isn’t that I’m not tweeting, I’m not even checking it. This seemed to happen shortly after I read The Shallows by Nicholas Carr, another great book talking about humanity’s adolescent approach to the Internet at the moment. While the overall tone of the book was a little too “doomsday” for me, he had some fantastic ideas on the importance that immediacy has gained in the last few years.

24 hour news, Twitter feeds, Facebook statuses, news aggregators and near real-time search engines are pumping information at us at a pace we’ve never experienced before. As consumers, we’re all expecting and demanding this sort of information, as we’d check Facebook and expect the latest news from our friends, check the BBC News website for minute-by-minute updates etc. However, over time this has had the undesired fact that this immediacy (or at least desire for) is now pervasive.

How many people have noticed their attention span has significantly decreased over the last few years? How many people feel like they’re undergoing withdrawal symptoms when they leave their phone at home? How many of you actively crave your information fix?

I do. Continue reading

Kohonen Neural Network Experimentations

A few weeks ago I updated my Kohonen neural network code to support circular rows and columns as well as some simple additional visualizations which allowed for some interesting experimentation.

Circular rows and columns follow a simple premise — the neighbourhood influence effect of training can now wrap around both row and column, allowing for circular, cylindrical and torodial geometries.

Think of it like this, a network with a single row wrapped around allows for a network to learn a circular topology. A nice, simple example of this is the Travelling Salesman Problem as you need one continuous, circular route to be determined. Below there is an example of a Kohonen network run over 350 iterations. The initial phase starts to spread the nodes out over the map, with the second phase making more localized adjustments:

Kohonen-based TSP Solver

It is worth noting that the Kohonen-based solver isn’t perfect, sometimes choosing interim junctions where there is no city, but it is an interesting, reasonably practical use of a different kind of Kohonen topology. Continue reading

The Consumerization of IT (Journalism)

There has been a storm on Twitter and (due to Robert Scoble’s involvement) Google+ recently over bloggers, journalists and the relationship between the two. It seems that Dan Lyons, a technology journalist for Newsweek, was upset — “Hit men, click whores and paid apologists: Welcome to the Silicon Valley cesspool” — and took to his personal blog to vent. Aiming at Michael Arrington and MG Siegler will inevitably draw fire but a week or so later, Lyons has further drawn on the ire of the social media elite by accusing Robert Scoble of trying to employ similar tactics. Scoble very publically rebutted Lyons’ arguments but the argument continues…

For me, a lot of this seems to boil down to one thing: the consumerization of technology journalism. Continue reading

Justice, Capuchin Monkeys and Investment Bankers

Capuchin Monkey in Costa Rica (Copyright © 2011, James Matthews)

As with most of my posts, this is inspired after reading a thought-provoking book. This time, it was The Wisdom of the Crowds by James Surowiecki. The book spends a lot of time discussing how large crowds of people can make (deliberately or inadvertently) the best (or most optimal/utilitarian) decision for a given problem. This whole and rather contentious topic is something best left for another blog post, however, there were several parts of the book that covered some more psychological aspects of group behaviour that I wanted to explore.

One area that was especially interesting was the concept of justice. It turns out that humans aren’t the only species that have a concept of justice. In 2003, scientists proved that capuchin monkeys would protest if they saw another monkey “paid” more than them for the same task. This scenario was created by training the monkeys to swap stones for cucumber (the pay). Then scientists then arbitrarily chose one of the monkeys to receive a grape instead of cucumber. The other monkeys would grow indignant, sometimes refusing to take their cucumber, other times taking it and refusing to eat it, other times refusing to continue working (bringing stones). Continue reading

Universal Darwinism: The Evolution of Everything?

After reading Richard Dawkins’ The Selfish Gene, John Cribben’s In Search of the Multiverse and Philip Balls’ Critical Mass, my interest in evolution as a generic or more universal concept has been revived. Is evolution a concept much broader than Darwin ever envisaged? Can it apply to human behaviours? Natural structures? How about our entire universe?

With the current socio-political climate in the US being driven more and more toward the extreme right, where so-called “respected” politicians harp on about intelligent design and other such bullshit, I found it interesting to see that evolution may extend from explaining how our genes have changed over the millena, to actually understanding everything from our place in the universe to the inherent behaviours that we exhibit. Darwin’s work may have uncovered a greater universal truth. As Daniel Dennett once said:

“If I were to give an award for the single best idea anyone has ever had, I’d give it to Darwin, ahead of Newton and Einstein and everyone else. In a single stroke, the idea of evolution by natural selection unifies the realm of life, meaning, and purpose with the realm of space and time, cause and effect, mechanism and physical law.”

My first proper exposure to the theory of evolution beyond basic biology class was when I was about 17 and learnt about genetic algorithms (GA) when writing the Generation5 website I put together for the ThinkQuest competition along with Samuel Hsuing and Edward Kao.

AH-64D Apache Longbow

The AH-64D Longbow’s radar has genetically evolved algorithms powering it.

Sam had written an article (which I later expanded upon) about using a GA to solve a diophantine equation. I found it amazing that computer scientists had taken Darwin’s idea of “survival of the fittest” and applied it to something as abstract as solving mathematically equations. Not only that, it was bloody efficient at doing it!

Two years later, I interviewed Steve Smith, one of the engineers behind that massive radar that sits atop the AH-64D Apache Longbow (right). The Apache’s radar can automatically detect the target from the radar signature, and the software that powers this intelligence was evolved via genetic programming.

At the time though, the deeper meaning behind all this “cool technology” never really dawned on me. Fast forward many years and my fascination with genetic algorithms remained. I was stunned by the ability of evolution to seemingly solve huge problems if you could simply assign a fitness to any given solution. Now with that said, this post isn’t meant as a lesson on genetic algorithms as I’ve written plenty in the past (including this bad boy if you’re feeling adventurous). Continue reading

Consumer IT: Services and Simplicity

People often ask me my opinions on various bits of technology they’re considering buying and 2011 has been a year of lots and lots of technology, so I thought I’d write up what technology has impacted me the most over the last 12 months.

Principally, I’ve noticed two trends evolving in the way I approach my personal and professional technology: services and simplicity. Continue reading

The Hivemind, Groupthink, Social Media and Individuality

I recently finished the fantastic book by Jaron Lanier, You Are Not a Gadget and one fairly central theme was how humans interact en masse. Much of the Internet often centres on this idea of the “hivemind” and harnessing little quanta of intelligence from a vast number of Internet-connected people to some end. Wikipedia is a prime example of this — lots of (often) anonymous people creating, editing and tweaking articles about…well, pretty much everything.

The hivemind was always an idea that both intrigued and perplexed me. I love the idea that we can harness intelligence in a similar way to harness spare compute cycles (i.e., SETI) and emergent or self-organizational behaviour continues to fascinate me. However, I’ve never been completely comfortable with how viable this is taken at an human-intellectual level. A compute cycle is a known entity − if it changes, it scales in size which affords you more work. Human intelligence is very much an unknown, in both scale and quality.

Wikipedia seems like a great example of how this might work. However, the anonymity behind Wikipedia makes it hard to ascertain how much of it is truly the hivemind at work, versus several experts or fans creating information that is subsequently updated as time moves forward (aside: he makes an interesting parallel between Wikipedia and the Bible). Much of Lanier’s arguments against Wikipedia seem aimed more at the cultural − search engines increasingly point to Wikipedia as the first listing, taking relevance away from other peripheral sites. As Wikipedia aims to be encyclopedic in nature, human opinions, insights and extremism is (often) missing from entries. Rightly or wrongly, it is these thoughts and opinions that gives us our rich and diverse global cultures.

The concern is that “hivemind” projects combined with the cloud-computing Overlords’ search algorithms is leading the human race down a path that inherently limits rather than frees the information we have readily accessible to us. While I have a small issue with his choice of words, the sentiment is beautifully summarized as:

“We should not seek to make the pack mentality efficient. We should seek to inspire the phenomena of individual intelligence.” – Jaron Lanier

Continue reading

Evolving Cooperative IPD Strategies

As I continue to play with the IPD, I created code to genetically evolve IPD strategies to see if cooperation could be borne out of random behaviours. I used the standard GA I’ve created in Wintermute, with each agent represented by the five weights detailed in my last post and with fitness calculated as the average points earned in each bout (this was subtracted from 5 in order to allow the GA to search for a minimum). I then created a population of 500 agents with a mutation and elitism rate of 0.5% per generation.

It took me a while to tweak the GA to start working, but I finally got it with fascinating results. Here is a chart of the distribution of strategies along with the best fitness for each iteration:

Continue reading