The war on Natural Language Understanding

At last year’s Mobile World Congress, the conference organizers agreed, "There was overwhelming acceptance at the event of the inevitable shift of focus for brands and corporates to chatbots … the true potential of chatbots will require further advances in AI and machine learning.”

The question isn’t whether or not chatbots would take off but who would dominate AI. NLP makes machines as intelligent as human beings in understanding language and getting the NLU right is the starting point to a successful conversational experience.

• NLP — Natural Language “Processing”
• NLU — Natural Language “Understanding”
• NLG — Natural Language “Generation”

In my opinion, IBM Watson, Microsoft LUIS, and Google Dialogflow are the heavyweights in the NLU landscape. These frameworks are more or less equally capable in the domain. All of them provide easy ways to define intents, as well as model training to improve the intent recognition accuracy. I spent some time on these frameworks to see how they stack up against each other.

IBM Watson

IBM Watson can analyze text in 10 different languages to extract intents, and interprets entities and dialogs from content. It is a highly scalable and versatile NLU platform but has a really steep learning curve. You can plug IBM Watson into chatbots, apps, bots, and IoT services. Watson performs its entity extraction via a predefined list of value you upload. You can enter synonyms for different entity values. You will likely spend a fair amount of time figuring out the platform and preparing assets for your domain from the ground up - entity/relation/dictionaries for training. Once you get past the initial set up, the UI is rather straightforward and actually quite pleasant to use. I think this level of heavy-lifting would only make sense for huge enterprises.

Google Dialogflow

Getting up and running in Dialogflow is pretty straightforward. Dialogflow is very flexible with single-click integrations with many other channels. But I’d say it’s primarily for chatbots. It is the easiest to get started among the three. It relies heavily on intents for prediction by linking user’s request with the corresponding action. The intent matching may be restricted by listing contexts that must be present for intent to match. When we create an intent, Dialogflow gives us the “Training phrases” section where we will put the entries that will activate the Intent for the user.

Microsoft LUIS

Microsoft LUIS stands for Language Understanding Service. It supports 10 languages and uses similar logic as Dialogflow. It serves the primary purpose of resolving user intents and extracting entities. It comes with pre-built domains for music, and calendar. Like IBM Watson, it’s enterprise focused and aim to service a wide range of platforms from chatbots, apps, bots, to IoT. LUIS also provides integration with Microsoft’s Bot Framework and can be used by other bot platforms.

We touched on the three biggest names in NLU above and it should give you a general understanding on these NLP frameworks and help you feel more equipped to utilizes these tools. In my opinion, all technologies have their pros and cons, so we should not ask which is better but what is more suitable for when.

Learn More About Allganize's Technology

• Alli AnswerBot
• Cognitive Search
• NER API
• Review Analysis
• API Sentiment API
• Text Classification API