Text analytics

A company wants all incoming emails to be automatically routed to the correct department. The company must find a way to process the text and extract the information for which it is looking. Text analytics is the most efficient and effective way to obtain information.

In this lesson you are going to learn what text analytics is and how it is used to help understand customer sentiments and needs.

Consider this message from a customer.

Humans have the ability to understand the implications of a piece of text. We can summarize it or elaborate on it or even re-phrase it and still preserve the original meaning. In fact, we do this very well. But can we do it on a large scale?

Today, digital sources such as news, blogs, social media and emails are generating more text than ever before.

It’s not possible for humans alone to analyze and extract valuable information from such a large volume of text. Consider a company that has tens of thousands of customer service records in text form.

Let’s say the company wants all of the complaints to be automatically routed to the right department. Then it wants a report that breaks down the complaints by department. The company also wants to use these records to understand how satisfied its customers are about various products and services. The company must find a way to process the text so that the information it’s looking for can be extracted. Text analytics is the most efficient and effective way to do this.

Text Analytics is the process of deriving high-value information from text.

It involves converting unstructured text into structured data that can be analyzed using statistical methods. There are three main steps involved in text analytics.

First, data is fetched from a source, such as social media, emails or customer service records. Second, natural language processing techniques are applied to extract certain attributes from the text and present them as structured data. And third, the extracted structured data is stored and analyzed.

Natural language processing is a field of computer science, artificial intelligence and computational linguistics concerned with the interactions between computers and human language.

In particular, it is concerned with programming computers to fruitfully process large volumes of natural language text. At a high level, the following analyses occur during natural language processing.

Syntactic analysis analyzes the structure of text and recognizes the parts of speech, such as nouns, verbs, adjectives, etc. Semantic analysis establishes the meaning of a piece of text. So the relationship between words and sentences is analyzed. In the linguistic analysis phase, the nuances of a language are considered to ascertain the exact meaning of the text.

Here are the outcomes of natural language processing.

The language of the text is detected. The sentiment is detected as positive, negative or neutral. Sentiment is the general attitude expressed towards a subject. The overall sentiment as well as the sentiment per sentence is reported.

The text is classified into one or more pre-defined categories, such as the business functions in a company. For each category detected, the sentiment of the corresponding sentence is also reported. Entities refer to the proper nouns found in the text such as names of people, places, dates and times, organizations, etc. This helps in establishing the subject of the text.

You have now reached the end of this lesson which provided an overview of text analytics and how it works.