Why You Should be [Moderately] Unhappy About Sentiment Analysis

We’ve been busy doing some work (positive) with text analytics (neutral) for a major retailer (positive). It’s been a thoroughly enjoyable experience (positive), and we believe we are very near the leading edge of practising text analytics for marketing (positively arrogant).


One of the things we’ve learned is that, short of the development of a far more powerful human artificial intelligence, accurate sentiment analysis is not possible (negative). And it probably won’t happen in our life time either.


Here’s why.


Sentiment in language is determined by two things: semantics (meaning) and pragmatics (context). Pragmatics can’t be analysed digitally.


The context of a dialogue (and, since people increasingly write kinda like they talk, the replies to a customer satisfaction survey) is like a game where the rules are quickly constructed by the partners to fit that particular game.  Each game has different rules, heavily dependent on circumstance.


And by game, we don’t mean tennis, but more something like a Motherwell bar brawl.


Pictured: Discourse

Pictured: Discourse


To see pragmatics in action, consider this sentence: “Brian killed the man with a gun.”


Did the speaker mean that Brian shot the individual in possession of the firearm, or that he terminated the existence of an individual by means of a firearm?


We don’t know, because we don’t know the other side of the conversation: the rules are constructed without telling anyone else. A linguistic structure devoid of its non-linguistic context loses its meaning.


The human brain can establish these rules intuitively. Since computers can’t, sentiment analysis simply doesn’t work.


However, there is something that bypasses the need for text analytics to be capable of sentiment analysis.


More on that in the next post.


If you’d like to know more about text analytics or have something (positive, negative, or neutral) to say to us, please leave a comment or give us a call.