Natural language processing (NLP)

Natural language processing (NLP) is an AI-powered technology that makes natural language (i.e., human communications) intelligible for machines. NLP can analyze speech or written text, categorize it, extract insights or generate natural-language responses.

Natural language processing is an evolving interdisciplinary area of research. It combines linguistics and machine learning (a branch of artificial intelligence) to identify patterns in natural language, including contextual nuances.

To fulfill these purposes, the NLP solution can be trained to complete the following tasks:

• Speech recognition — converting speech into machine-understandable text 
• Grammatical tagging — identifying the part of speech for a word 
• Word sense disambiguation — selecting the right meaning for a word 
• Co-reference resolution — determining when two words refer to the same entity 
• Sentiment analysis — pinpointing attitudes, emotions and subtext 
• Natural language generation — producing natural language text or speech 

A natural language processing solution relies on a neural network or another machine learning model. But to achieve its primary goals, algorithms should ingest training data in large quantities with expected outputs (i.e., tags). The algorithms then identify patterns leading to every tag and make predictions based on those rules.

Before running text or speech through the ML algorithms, NLP solutions need to preprocess it. This preprocessing involves:

• Tokenization — breaking speech down into smaller units (tokens) 
• Stemming — removing affixes from a word 
• Lemmatization — reducing a word to its base form (lemma) 
• Stop word removal — removing words that bear little to no meaning, such as articles and prepositions 

Some common use cases for natural language processing include the following:

• Virtual assistants. Typically used in smart speakers, virtual assistants process and respond to spoken requests. Examples: Siri, Alexa 
• Predictive text. NLP algorithms can predict the next word users will type in the search bar or message field. Examples: Gmail’s autocomplete, iOS’s typing suggestions 
• Brand sentiment analysis. In this case, NLP categorizes social media posts or reviews based on the opinions expressed in them (positive, negative, neutral and everything in between). Examples: Idiomatic, TalkWalker
• Customer feedback analysis. An NLP tool can sort customer feedback into relevant categories based on their subject. Then, it routes customer support tickets to appropriate teams. Examples: Retently, Zendesk
• Chatbots. These applications deliver self-service customer support 24/7. Chatbots easily handle routine queries while rerouting complex ones to the support team. Examples: Lemonade’s Maya, Domino’s Order with Dom