The FT On Text Analytics: What We Knew Already

In Saturday’s FT Life, John Sunyer recounts his trip to the “literary lab” of Stanford University’s Franco Moretti. Moretti is a pioneer of “distant reading”. This is the application of text analytics to literature, using the same kind of software as Verbal Identity, but to explore entire fictional genres.


As an example, the lab took 3592 works from 1780-1900, and analysed them by criteria like vocabulary and theme to find the period’s most influential authors. Conventional wisdom says it’d be luminaries like Dickens and Twain. Conventional wisdom loses: It was Walter Scott and Jane Austen. Authors considered as trashy then as Tom Clancy and E. L. James are now.


What’s driving conversation isn’t always what you think, and text analysis can show this. We never tire of being proved right. But we never tire of giving our opinions, either.


The article highlights a concern many will share. This encroachment of Science into the humanities may kill the “individual” scholar, and that companies like Google and Amazon might kill the writer. They already use data to find points in a book where people stop reading. What’s to stop them using distant reading to automatically write novels catering to specific readers?


Our recent successful experience with applying text analytics to the marketing of a major UK retailer suggests quite the opposite. We’d say that far from being a step towards a replacement, distant reading makes writers more important. Even though it endeavours to contextualise literature, we must remember that the context of literature is culture.


Culture is a dynamic system that can only be understood by being in it, much like a conversation. Writers write to explore problems they have with the conversation. And though new cultural touchstones may inspire new conversations, this creative engagement can’t be done away with.


Just like text analytics for marketing can’t eliminate good copywriters.


What it is likely to do is make new novels sharper by pinpointing weaknesses. And encourage writers to become more creative to subvert the new status quo.


Just like text analytics for marketing helps to sharpen up copy.