The how and why of collective intelligence
What is collective intelligence, and how can technology help us to access it? Fresh from Mash-Up Mayhem, Tony Hirst set out the basics in a lightning talk, and demonstrated how clustering data can help universities plan their courses - and allowed Tesco to corner the British grocery market. Here’s a summary:
The why of collective intelligence
The techy bit of collective intelligence is that there’s lots of data and stuff and people around but they are hard to engage with. So, being better able to appreciate and understand and see some of the patterns in that data, making sense of that stuff will help raise the level of debate and how we can engage with it. There are examples of it in use all over the place, like Amazon recommendations, that’s about connecting people. Connecting people who share interests and getting them to learn from each other. It could relate to collective action. Or there could be business models for recruitment: identifying developers with interests and skill sets who can contribute to tasks.
The how of collective intelligence
The book Collective Intelligence (O’Reilly) – has the tools you need and you can download the code from O’Reilly website. Can look for clusters or patterns so eg on Amazon, the books that I have bought or put on my wishlist sort of defines me and does so for other people. The clustering tools looks at everybody’s wishlists and joins together people into clusters, clumps of interest groups so that they can make recommendations. It allows you to segment your data into people who are doing this or that. You don’t need many segments for it to be useful. Tesco used to use just five or six segments. Clustering pulls together all the things that are similar and then you can display it in treemaps etc, hierarchical visualisations.
Finding ways to visualise this data is important. Visualisation tools are getting increasingly powerful and easy to use.
Another way is looking at big data and partitioning it into sensible, meaningful units. Could do this with library search data.
Collective intelligence in action
At the OU we set up a Facebook group where students could declare the courses they had taken and we plotted a chart from the clustering from this data (about 6,000 students) and recreated a degree map. Traditionally, OU students chose their own courses and with that data we could see which degrees were emerging from the clusters. Could also have a recommendation engine for courses. We also asked for timelines so we could plot sequences of courses and that feeds into planning. OU are now looking at doing this themselves.
Widening collective intelligence
You want to build tools that encourage people to put in their data themselves because it is useful themselves as an individual but the more people who do it, the more useful it is to look across the data as a whole. Example of Tesco clubcard – there’s individual utility but Tesco have also ruthlessly used the data to inform the decisions they make. It’s looking for opportunities to extract value and data where the patterns might be. You have got to start playing with the tools.