We’ve certainly had lots of feedback to focus on for the next one (in September). The three topics that we “crowdsourced” at the end of the day might well feature in our thinking… Thanks to Richard Sedley (cScape) for coming up with that one!

We have the full video footage of the day – so will be cutting that up over the weekend and posting it up next week.  All the presentations are now live on Slideshare and linked below, along with some useful posts and downloads:

Most social media monitoring companies offer automated sentiment detection services. Automation saves time and money: so it’s instantly appealing. The customer can log in and simply watch graphs and tables appear, showing them data about their brand in a user-friendly desktop environment. What could be better? Such services also have the benefit of being able to capture, assess and display data in near real-time speed, which is really compelling for companies running marketing campaigns or events. You can just sit back and watch the results coming it – saving your energy for posting timely replies and responses to negative tweets, posts and comments.

These automated services go way beyond the simple “smiley” test of many Twitter apps. They have constructed complex algorithms that can identify different elements of sentences and assess the sentiment of and connections between these words or phrases. Scoutlabs calls this “speech tagging” and describes it as “parsing the underlying semantic structure of a sentence and determining which emotion words apply to the key word”. Clever stuff, no doubt.

That said, there are several obvious downsides to automated sentiment detection:

• Irony/sarcasm – It’s currently impossible to accurately assess posts that include ironic or sarcastic comments.
• Slang – While some services incorporate dictionary slang, they (like most humans) cannot keep up with developments in street slang.
• Languages – Many services only operate in English, ignoring the other 6999 languages (yes – that’s how many there are) spoken in the world.
• Geographical variations – Different countries and regions us different expressions and slang, even within the same language.
• Context – Negative and positive are not the same for everyone and brands may consider it negative to be associated with certain terms or people. “Mugabe Gets an Iphone”, for example, may raise eyebrows in the Apple marketing team.

As a result of these issues, automated sentiment detection services can only offer an accuracy rate of 75%-80%. The claim is that this compares well with human intervention ratings, which apparently show a difference of opinion 20% of the time. But I find that hard to believe. Given strict guidelines I think most intelligent adults would rate posts accurately as positive, neutral or negative 99% of the time. Wouldn’t they? What kind of an idiot would misread the meaning of a Twitter post??

Well, actually, to answer my own question – if the value (or damage) of a comment is gauged by its effect on people reading it, we have to assume readers are not particularly smart or well-versed in the nuances of irony or slang. Your average twitterer is probably quite likely to misunderstand someone else’s post and act accordingly. If for example you saw my post “Just signed up with TalkTalk. Customer services as good as ever!” I’m not sure everyone would pick up on the truly scathing and malign undertones of my Tweet.

Automated sentiment detection services generally offer an option to amend sentiment ratings, and therefore enhance the quality of the data, manually – and, in my view, this is one of the most beneficial activities anyone running a social media monitoring campaign can engage in. Reviewing the top level graphs and charts can be hugely gratifying, but it’s at street-level, by reading and responding to individual posts, that you get a true picture of what people really think.

I’ll be posting thoughts about  non-automated sentiment detection shortly, but in the meantime I’d welcome additional comments and suggestions on this topic. We’ll be discussing the topic further at our upcoming social media monitoring event: Monitoring Social Media 09, in London on 17th Nov.