Collective Intelligence

Looking to Wikipedia for answers

By Thomas Malone

Published: November 5 2008 14:43 Financial Times

To understand how large-scale work was organised during the past 100 years, the best models were traditional hierarchical organisations such as General Motors, IBM, and Wal-Mart.

But to understand how large-scale work will be organised in the future, we need to look at newer examples such as Wikipedia, eBay, and Google.

In Wikipedia, for instance, thousands of people from across the globe have collectively created a large and surprisingly high-quality intellectual product – the world’s largest encyclopaedia – and have done so with almost no centralised control. Anyone who wants to can change almost anything, and decisions about what changes are kept are made by a loose consensus of those who care.

Wikipedia is a remarkable organisational invention that illustrates how new forms of communication, such as the internet, are making it possible to organise work in new and innovative ways.

Of course, new ways of organising work are not desirable everywhere. In many cases, traditional hierarchies are still needed to capture economies of scale or to control risks. But in an increasing number of cases, we can have the economic benefits of large organisations without giving up the human benefits of small ones – freedom, flexibility, motivation and creativity.

These human benefits can provide decisive competitive advantages in knowledge-based and innovation-driven work. During the coming decades, we can expect to see such ideas in operation in more and more parts of the economy.

These new practices have various names: radical decentralisation, crowd-sourcing, peer production, and wikinomics. But the phrase I find most useful is “collective intelligence”.

In our work at the Massachusetts Institute of Technology Center for Collective Intelligence, this phrase has inspired us to ask the provocative question:

“How can people and computers be connected so that – collectively – they act more intelligently than any person, group, or computer has ever done before?”

What if we could have any number of people and computers connected to, for instance, care for patients in a hospital? Or design cars. Or sell retail products.

We might that find the best way to do a task that today is done by five full-time people would be to use one part-time employee and a host of freelance contractors each working for a few minutes a day.

One important type of collective intelligence is “crowd intelligence”, where anyone who wants to can contribute.

Sometimes, as in the case of Wikipedia or video sharing website YouTube, people contribute their work for free because they get other benefits such as enjoyment, recognition, or opportunities to socialise with others. In other cases, such as online retailer eBay, people get paid to do so.

Anyone can become an eBay seller and most of the key decisions about product mix, pricing, and advertising are made not by managers at eBay, but by the collective intelligence of the eBay sellers themselves.

A few years ago, eBay managers were surprised to find members successfully selling automobiles on the site, so they added additional support for this product line.

Sometimes, crowd intelligence can even operate inside the boundaries of a single company. Google, Microsoft, and Best Buy have all used internal “prediction markets” to tap the collective intelligence of people throughout their organisations. In these prediction markets, people buy and sell “shares” of predictions about future events such as revenue levels. If their predictions are correct, they are rewarded (either with real money or with points).

Microsoft has used prediction markets to estimate completion dates for internal products. When it launched one of the first of these markets, the share price for a product scheduled to be finished three months later declined within minutes to a price indicating only a 1.2 per cent probability it would be completed on time. The managers in charge of the project had thought it was on schedule, but when they saw these results they investigated further and found problems. The product was eventually released three months late.

Here was a case where knowledge about the project’s problems was available inside an organisation but it took a prediction market to bring it to the attention of people who could do something about it.

Another important type of collective intelligence is “cyber-human intelligence”, where computers do not just connect people to each other, they provide their own “intelligence” as well.

Google harvests the intelligence of millions of people who create web pages and link them to each other, but its sophisticated algorithms also rank the pages based on how many links exist to a given page.

Electronically connected forms of crowd intelligence and large-scale forms of cyber-human intelligence have never before existed. Yet these examples are just the beginning, and it is very likely that innovative organisations will discover more ways to radically change existing industries or create new ones.

These changes will not happen overnight, but the rate of change is accelerating and business people a hundred years from now may find the pervasive corporate hierarchies of today as quaint as we find the feudal farming system of an earlier era.

Thomas W. Malone is the Patrick J. McGovern Professor of Management at the MIT Sloan School of Management and the founding director of the MIT Center for Collective Intelligence

Copyright The Financial Times Limited 2008