The leading lights of the digital industry gathered in Soho last night for a Chinwag Live event tackling the contentious issue of Measuring Social Media. The expert panel included Alex Burmaster from Nielsen Online, Robin Grant of 1000heads, Ankur Shah Co-founder of Techlightenment, Will McInnes of Nixon McInnes with Jim Sterne from the Web Analytics Association in the chair. While the debate generated a little heat there was a disappointing if not unsurprising, lack of light. The discussion covered four key strands:

1. Can Social Media be measured?
2. If it can, what’s the best way to do it?
3. Do we need an industry standard set of metrics?
4. Would such metrics actually be of interest/any use to the CEO?

The debate resisted the urge to begin by defining Social Media, but many of the difficulties around measurement are rooted in its disparate nature. Counting hits on YouTube, the number of friends on MySpace or downloads of a Facebook application are at the ‘relatively straightforward’ end of the measurement spectrum with blog conversations, product reviews on forums, negative search results etc, at the other. We are all used to measuring clicks but conversations are another matter.

Will made the point early on that there is something counter-intuitive about measuring “conversations” and questioned whether automated monitoring services from the Nielsen’s of this world can attempt to do that effectively. When it comes to analysis of human interactions people are slow and computers are dumb he added. I have to agree, from personal experience in producing social media audits for FTSE 100 companies, the value of those reports are not contained in the statistics but in the commentary and analysis of the data and trends.

Numbers need to be put into context by professionals who understand the company, its products and customers and the industry it is operating in. Apparently we are going to have to wait until 2029 for machines to become as intelligent as man, so until then slow humans and dumb computers are going to need to work better together to generate meaningful insight into Social Media impact. There are no short-cuts or quick fixes, which for those wishing to sell their Social Media expertise is surely a good thing?

Whether the evaluation process would be helped by developing an industry agreed set of metrics was the next question. Some on the panel strongly advocated the sector working together to develop open standards for measurement and effectiveness. Some in the audience shared their own proprietary and largely fluffy approaches, while others argued that it would be an impossible task.

This part of the discussion for me showed the digital industry at its parochial and navel gazing worst. Given the pace at which the landscape is developing, by the point a set of metrics had been agreed (conservatively c.2011) they would have limited relevance to the current reality. My frustration was shared by others in the audience and highlighted by a questioner who hit the nail on the head in arguing that it’s the outcomes rather than the outputs of Social Media which really matter the most to organisations.

In my mind Social Media techniques need to be employed to address specific marketing and business challenges and to reach out to specific audiences. The objectives and desired outcomes of a campaign whether it’s to directly drive sales, mitigate a crisis, reshape a reputation or whatever are going to be so particular to the client concerned that developing a one-size-fits-all set of metrics really misses the point.

Ultimately, with Social Media as with so much else, it’s not measuring it that counts but what you do with it!

Further reviews of the night from Wendy McAuliffe, Stuart Bruce and Seb Mysko.

Check-out this link next week to hear the podcast of the event.