I recently finished the fantastic book by Jaron Lanier, You Are Not a Gadget and one fairly central theme was how humans interact en masse. Much of the Internet often centres on this idea of the “hivemind” and harnessing little quanta of intelligence from a vast number of Internet-connected people to some end. Wikipedia is a prime example of this — lots of (often) anonymous people creating, editing and tweaking articles about…well, pretty much everything.

The hivemind was always an idea that both intrigued and perplexed me. I love the idea that we can harness intelligence in a similar way to harness spare compute cycles (i.e., SETI) and emergent or self-organizational behaviour continues to fascinate me. However, I’ve never been completely comfortable with how viable this is taken at an human-intellectual level. A compute cycle is a known entity ? if it changes, it scales in size which affords you more work. Human intelligence is very much an unknown, in both scale and quality.

Wikipedia seems like a great example of how this might work. However, the anonymity behind Wikipedia makes it hard to ascertain how much of it is truly the hivemind at work, versus several experts or fans creating information that is subsequently updated as time moves forward (aside: he makes an interesting parallel between Wikipedia and the Bible). Much of Lanier’s arguments against Wikipedia seem aimed more at the cultural ? search engines increasingly point to Wikipedia as the first listing, taking relevance away from other peripheral sites. As Wikipedia aims to be encyclopedic in nature, human opinions, insights and extremism is (often) missing from entries. Rightly or wrongly, it is these thoughts and opinions that gives us our rich and diverse global cultures.

The concern is that “hivemind” projects combined with the cloud-computing Overlords’ search algorithms is leading the human race down a path that inherently limits rather than frees the information we have readily accessible to us. While I have a small issue with his choice of words, the sentiment is beautifully summarized as:

“We should not seek to make the pack mentality efficient. We should seek to inspire the phenomena of individual intelligence.” – Jaron Lanier

I disagree with the phrase “pack mentality” as I feel there is a subtle but distinct differences between mobs, crowd behaviour, pack mentality, hivemind and (as we shall see) groupthink but I’m probably picking the sentence apart more than I should. Most people hold the opinion that people do smart things, crowds do dumb things (ironically, the fact that people think that could just be crowd behaviour at work!). We only have to look at the recent London riots and stock market crashes to see how mob, crowd and pack mentality plays out or look at common phrases like “too many chefs in the kitchen” or “a camel is a horse designed by committee” to see conventional wisdom doesn’t seem to favour crowd thinking.

I find very interesting parallels between what Lanier talks about and the groupthink phenomena that Lee Smolin discusses in The Trouble with Physics (in fact, it was Smolin’s appraisal for the book on the cover that attracted me to You are not a Gadget!). Groupthink is effectively a dynamic within a group of people that occurs when the desire for group cohesion seems to displace the desire to evaluate alternative strategies, decisions and viewpoints. It has been attributed to everything from the US’ seeming dismissal of the Japanese threat before Pearl Harbour, the disastrous Bay of Pigs operation, the Enron scandal, Coca Cola’s rebranding and (so it doesn’t look like I’m picking on the States!) the attitudes of Northern Rock or Swissair prior to their spectacular collapses.

I read Smolin’s The Trouble With Physics after Brian Greene’s The Elegant Universe principally because I felt super-symmetric string theory was (while beautiful and indeed elegant) a house of cards. The LHC was coming online and showing no signs (still!) of the Higgs Boson or anything else needed to finally prove there was an inkling of truth to this theory that had been studied for so long. Smolin’s account of physics was fascinating but the most terrifying revelation for me was just how deeply engrained groupthink mentality was in the scientific community. Decades of almost mass indoctrination of the globe’s most intelligent people has led to probably hundreds of millions (if not more) of man hours and research dollars to be focused on a theory that is increasingly looking like it is wrong (or at least not without some major rework). Obviously, therein lies the beauty of research and academia ? it is as important to discover what’s wrong as what is right, but not at the detriment of alternative avenues of discovery.

While reading The Trouble with Physics, the technologist within me pondered how technology could be used to help avoid groupthink mentality. It nagged me that Tim Berners-Lee invented the Internet as we know it to help researchers share and update information between them! This nagging further resonated when reading Lanier’s lamenting of how technology increasingly seems to limit our access to information and promote the hivemind over individuality.

Collaborative or social media technologies will no doubt play an interesting part in this ? but how? I feel that people need to understand the difference between what Lanier calls first-order vs. derivative content. First-order content is original in nature, with derivative content being a mash-up (to use that horrible Web 2.0 terminology!) of multiple original or derived sources. In the academic world, most content is based upon the scientific work of others but is for the most part original in content. Corporate content is often split depending on the technology available. When I worked at Microsoft, virtually all content was derivative as we had the technology to find the content created by the other 90,000 people within the organization, as well as the necessary culture to share it in the first place! I saw many clients that had neither the culture nor the technology, which lead to a lot of first-order content creation (and therefore time-wasting).

Have a quick look on your Facebook feed for shared content. How much of that is content written by your friends…? Aside from photos, I’ll bet very little to none of it. How much of it is derivative content: a BBC news article or a Youtube video of Maru the Cat etc.

The challenge for technology in the coming years will be to balance how to facilitate first-order content creation while promoting the sharing of derivative content too. Currently, most technologies fall into one or the other ? traditional collaboration suites are superb at first-order creation with smaller startups exceling at the derivate content. While it is clear that future technology will need to support both platforms for managing content, how can communities (either academic or corporate) ensure that groupthink doesn’t seep into the culture of such pervasive sharing?

Assuming there’s a culture to support it (admittedly, a big assumption!), I believe that social media provides a great platform to promote many of the attributes needed to avoid groupthink ? a flattening structure, an active, vocal and distributed leadership and constructive criticiscm/devil’s advocacy to name a few. We may call it “commenting” or “social tagging” or “multi-party editing” or any of the plethora of marketing-speak we attribute to technology these days, but the net effect will see individuals working and sharing content and information to select communities or the wider world.

Does the hivemind have a place in this story? Maybe controversially, I think it does. Lanier has a great point that our culture may be averaged out, but indeed many large corporations and “entities” like stock exchanges may benefit from large collections of piecemeal intelligence. To this end, I have bought a book much maligned by Lanier ? Surowiecki’s The Wisdom of Crowds to further some of this thinking…but it has taken its place next to the mountains of other material I want to read!

Despite this, however convincing Surowiecki’s arguments may be, I feel I will always value the opinions and insights of the individuals I trust over the wisdom of the crowd. Looking at the UK Top 100 music chart is enough to convince me I’m right! (…or I’m getting old.)

To say this was a meandering post is an understatement, but it has brought together a few strains of thinking I’ve had so, if nothing else, it has been an exercise in intellectual therapy for me…! Thanks for reading and I’ll leave you with a final quote from Lanier’s highly recommended book:

“If the crowd is so wise, it should be directing each person optimally in choices related to house finance, the whitening of yellow teeth, and the search for a lover. All that paid persuasion [advertising] ought to be moot. Every penny Google earns suggests a failure of the crowd—and Google is earning a lot of pennies.”