(Even real-time) search is not enough

December 07, 2009 • Author: Catherine van Zuylen • 0 comments
Tags: Search, Text Analytics, Social Media

Recently, the internet has grown into a treasure trove of customer, product, competitive, research, and market information. Information about business trends, risks, and opportunities are waiting for the enterprise to unlock. With the rise of increased connectivity and social media, a large volume of near-real-time “first person intelligence” is available for companies to examine. More and more, people are turning to social media to communicate their experiences and issues with products and companies. These communications contain product suggestions, requests for help, frustrations and intents-to-leave, and other time-sensitive information.

THE RISE OF SEARCH

In response to the rising volumes of communications, search became more important to the enterprise. Companies like Google and Yahoo rose to help find information on the internet, while companies like Autonomy, Verity, and Endeca arose to help enterprises find their internal information. Search can be a very useful tool for locating a particular document of interest containing a search term. For example, you can tell a search engine to “find all emails containing the words ‘oil drilling’” or to “find documents about my company”.

However, search begins to be inadequate for revealing trends or performing complex operations. For example, if a company is interested in monitoring social media for information on the top 10 issues that customers have with their products, a company will generally create queries for the name of the companies, their products, and so forth. Then, personnel must manually read through the search results and tally up the issues by hand. What happens is that you can get hundreds, or even thousands of results that then need to be read and tallied to get a picture of what people think about your products and services.

Where using search becomes even more complicated is where it is difficult to define a small set of keywords that can get you the answers you seek.

For example, how would you ask a search engine to deliver to your inbox all tweets and forum postings where someone is seeking your help? Or have an engine automatically route to product management various product suggestions? Or have customer service be quickly alerted to messages indicating an intent-to-leave?

THE RISE OF TEXT MINING

A similar situation used to exist in the world of physical goods. Back when there was a limited variety of goods sold at the corner shop, it was not a big deal to be able to manually inventory, sort, and count goods. If you had cans of tomatoes, you could walk down your aisles every night, quickly count the 20 cans on the shelf, inventory those as “tomatoes”, and reorder more cans of tomatoes on a weekly basis.

But, as the volume and variety of goods increased, and as just-in-time delivery of those goods became more expected, the process needed to become more automated. So the barcode was invented and standardized as a standard way of expressing the “aboutness” of a product. With the invention of the barcode and its cousin, the RFID tag, new generations of products were able to be designed to inventory, track, and move goods based on this “aboutness”.

Likewise, with the rise of unstructured data volumes and the need for just-in-time delivery of information, a new system was needed that would go beyond keyword indexing. This new system was text analysis – a way of “barcoding text” that could express information about that text and allow it to be mined for information and moved to the appropriate place based on its aboutness.

In our next blog, we'll explore the rise of text mining, and why keyword-based systems are also not enough.

Comments

Leave your comment

All fields marked with * has to be filled.