Don’t Get Sentimental About Tools When Measuring Attitude
Recently at the SES London Jim Sterne discussed the hot topic of his new book Social Media Metrics. One of the things that stood out from his keynote was Sentiment measurement, or rather the inaccuracy of it. Jim basically said it sucks, which I’m delighted about because someone needed to say it.
Sentiment or attitude analysis is about understanding the mood of the person making a comment or post about a topic.
It’s been my opinion for some time that you can’t measure sentiment with any real confidence in the results, at least when you’re relying on the results of tools. Jim brilliantly illustrated this by showing some sentiment analytics from Twitrratr.
As you can see from the red strikeouts over 50% of the comments here could be categorized as falling into a different category. Many of the negative comments about Obama here are actually positive but because they talk about “lose” or “won’t” (as in might lose or won’t win) they have been categorised as negative sentiment.
They clearly aren’t negative, they are actually positive “Tod sez Obama could still lose pa, ohio and fla and still get to 270, that’s a favourable playing field” is a positive slant saying Obama could win without those states. It was listed as negative. Twice!
The point here is that even humans can’t agree in many cases on what’s negative and what’s positive in terms of sentiment so how do we expect a machine to do it?
My advice is to treat “buzz” or “sentiment” analysis in a qualitative way. In other words manually analyze whether it’s good, bad or indifferent! Kwantic for instance developed its own in house tools to randomly select comments around key terms.
Our tools randomly select comments and then we use one of our market research professionals to score the context of this random sample. The tool collects enough comments to ensure statistical significance is there and our analysts do the rest.
To me this is the only way to analyze sentiment correctly because only a human can put it into context, get the language right, understand slang and give an accurate picture of what people are discussing.
Comments as always welcome!
Previous Articles
If you remember one thing from your visit...
Remember this. As a web analytics specialist you should not be in the data business. You should be in the business of providing information.
This site will hopefully provide you with some ideas to get information vital to grow your analytics competence that will in turn help you improve your online marketing. Of course, I would love to find out what you think as well, so make sure to comment. See you around!
Comments