The leading lights of the digital industry gathered in Soho last night for a Chinwag Live event tackling the contentious issue of Measuring Social Media. The expert panel included Alex Burmaster from Nielsen Online, Robin Grant of 1000heads, Ankur Shah Co-founder of Techlightenment, Will McInnes of Nixon McInnes with Jim Sterne from the Web Analytics Association in the chair. While the debate generated a little heat there was a disappointing if not unsurprising, lack of light. The discussion covered four key strands:
1. Can Social Media be measured?
2. If it can, what’s the best way to do it?
3. Do we need an industry standard set of metrics?
4. Would such metrics actually be of interest/any use to the CEO?
The debate resisted the urge to begin by defining Social Media, but many of the difficulties around measurement are rooted in its disparate nature. Counting hits on YouTube, the number of friends on MySpace or downloads of a Facebook application are at the ‘relatively straightforward’ end of the measurement spectrum with blog conversations, product reviews on forums, negative search results etc, at the other. We are all used to measuring clicks but conversations are another matter.
Will made the point early on that there is something counter-intuitive about measuring “conversations” and questioned whether automated monitoring services from the Nielsen’s of this world can attempt to do that effectively. When it comes to analysis of human interactions people are slow and computers are dumb he added. I have to agree, from personal experience in producing social media audits for FTSE 100 companies, the value of those reports are not contained in the statistics but in the commentary and analysis of the data and trends.
Numbers need to be put into context by professionals who understand the company, its products and customers and the industry it is operating in. Apparently we are going to have to wait until 2029 for machines to become as intelligent as man, so until then slow humans and dumb computers are going to need to work better together to generate meaningful insight into Social Media impact. There are no short-cuts or quick fixes, which for those wishing to sell their Social Media expertise is surely a good thing?
Whether the evaluation process would be helped by developing an industry agreed set of metrics was the next question. Some on the panel strongly advocated the sector working together to develop open standards for measurement and effectiveness. Some in the audience shared their own proprietary and largely fluffy approaches, while others argued that it would be an impossible task.
This part of the discussion for me showed the digital industry at its parochial and navel gazing worst. Given the pace at which the landscape is developing, by the point a set of metrics had been agreed (conservatively c.2011) they would have limited relevance to the current reality. My frustration was shared by others in the audience and highlighted by a questioner who hit the nail on the head in arguing that it’s the outcomes rather than the outputs of Social Media which really matter the most to organisations.
In my mind Social Media techniques need to be employed to address specific marketing and business challenges and to reach out to specific audiences. The objectives and desired outcomes of a campaign whether it’s to directly drive sales, mitigate a crisis, reshape a reputation or whatever are going to be so particular to the client concerned that developing a one-size-fits-all set of metrics really misses the point.
Ultimately, with Social Media as with so much else, it’s not measuring it that counts but what you do with it!
Further reviews of the night from Wendy McAuliffe, Stuart Bruce and Seb Mysko.
Check-out this link next week to hear the podcast of the event.
February 19th, 2008 at 8:08 pm
Hey Daljit - just writing my thoughts now but wondered: do you think that developing some standards would be pointless? I can see the point you’re making, just mulling whether to plough into this as an initiative or not!
February 19th, 2008 at 9:23 pm
I think we have to go even further than interpreting stats. I think we have to dig deep into the language and discursive practices that make up the conversations to see how content relationships are being developed.
February 19th, 2008 at 9:34 pm
Hey Will,
My worry is that for the standards to be ‘universal’ they will end up being quite vague and woolly and there was a lot of that last night. I think that truly robust evaluation can only come by looking at the specific client and their objectives and the campaign you have put together for them.
That said, it’s always good to have a starting point and framework that you can build on if needed so I would say go for it because the journey may end up being more worthwhile than the destination.
I will throw my two-penneth into the mix!
February 19th, 2008 at 9:37 pm
Hi Paul,
Thanks for the comment, I agree the more time that’s spent listening to the conversations generated by Social Media the greater the upside for the client.
February 20th, 2008 at 9:11 am
Love the picture and your closing statement. Couldn’t agree more.
Having reflected on the event, there seems to be a dichotomy between a common set of measurement standards and which metrics are important to a specific organisation.
It’s telling that the person who suggested focusing on outcomes is a psychologist. Perhaps this indicates that the process of measuring social media needs more insight from that discipline as well as the number-crunchers?
February 20th, 2008 at 12:13 pm
Hi Sam, you are right, there needs to be more insight from across the digital spectrum and beyond.
To echo Wendy’s point it was disappointing the panel was broadly dominated by number crunchers with no dedicated representation from PR which has been in the understanding ‘human communication’ business for a fair few decades now.
Yes we need metrics, but they shouldn’t be seen as the desired end point for all the fantastic initiatives and debates that were triggered by Monday’s event.
To give just one example, in the PR space there has been a lot of debate and hard work done recently to understand the nature of ‘online influence’. As a sector we haven’t spent our time trying to develop a single-figure metric to replace the Technorati Authority score.
We know that life is more complicated than that, and if we are going to try to develop common methodologies for Social Media evaluation, we do need to involve as many disciplines as possible.