How to achieve answer bot accuracy of over 90% in half a day
Alli's AI doesn't require manual data tagging and it enables customer to deploy the answer bot right after they set up the first campaign. Furthermore, you can fine-tune Alli's AI to achieve even better answer accuracy with some additional settings.
Alli's AI doesn't require manual data tagging, and it enables customers to deploy the answer bot right after they set up the first campaign. Furthermore, you can fine-tune Alli's AI to achieve even better answer accuracy with some additional settings.
There're three ways to achieve hight answer bot accuracy.
- Train the AI using Query Training to update the Adjusted Score of each Q&A. The Adjusted Score decides if the Q&A will be shown as a possible answer to the question.
- Register Synonyms and Antonyms using the Dictionary feature to teach the AI industry-exclusive terms, acronyms, etc.
- Optimize the Q&A setting, considering the number of customers and the number of questions, and fine-tune the result.
Let's check them out one by one.
1. Train the AI using Query Training to update Adjusted Score
Alli adjusts the 'Adjusted Score' value accordingly by customers' feedback. The Adjusted Score increases when a customer selects a suggested Q&A. When a customer selects 'None of the Above', the Adjusted Score of all the suggested Q&As decrease. Adjusted Score decides the order of suggested Q&As toward customer questions (check out Letting Customer Feedback Affects Q&A Search Result for more details).
You can make a similar adjustment by using Query Training from the dashboard. There are two ways to do this: using the preview of the Q&A search, and the other is using the Query Training menu under the Q&A menu.
Check out our Agent Feedback for Q&A Search Result user guide to see how!
2. Register Synonyms and Antonyms
Keywords play a significant role in deciding Confidence value upon the Q&A search. Alli's natural language understanding AI doesn't require the same keyword to search for a Q&A, but it doesn't quite understand industry-specific terms or acronyms.
Using the Dictionary feature, you can add Synonyms and Antonyms to teach Alli. Synonyms let Alli understand multiple words with the same meanings, and Antonyms to distinguish similar words with different meanings. For example, you can teach Alli that MRC is Machine Reading Comprehension using the Synonyms, or prevent Alli from showing answers for similar-but-different product names using the Antonyms.
Dictionary is under Q&A > Q&A. Click the DICTIONARY button at the top right and select the SYNONYM or ANTONYM tab. Check Alli User Guide - Synonyms & Antonyms Dictionary for more details. You don't need to retrain the AI model after adding synonyms or antonyms.
In the example below, 'machine reading comprehension' and 'MRC' are registered as Synonyms, so the top search result shows the right one.
Register as many Synonyms and Antonyms as possible using your existing customer question data and expected list. By the way, you can also add them after the launch of the answer bot whenever it's necessary.
Repeating methods 1 and 2 increases the answer bot accuracy dramatically. In most cases, you can reach 80% to 90% accuracy to include the right answer in the answer candidate in just a few hours.
3. Optimize Q&A Configuration
The last step is to optimize Q&A Configuration and decide how to show the Q&A candidates to your customers.
A general example of the initial configuration is as below. These settings allow some flexibility to show the Q&A candidates, minimize direct answers, and run additional fine-tuning by customers' feedback (check out Letting Customer Feedback Affects Q&A Search Result for more details).
- Answer Similarity Threshold: 0.5 to 0.6
- Direct Answer Threshold: 0.9 to 0.95
- Number of question candidate: 3 to 5
- Impact of user feedback to Effective Confidence: 0.1 to 0.5 (set this higher if you expect a lot of customer questions, lower if not)
Adjust the values considering the fine-tune you've done, expected number of questions, etc. When you adjust the values while seeing the Q&A search result preview, the result changes in real-time so you can check the effect of the change you're making. Click Submit when you're done to save the settings.
I know it was a bit long read! If you have any questions about this user guide, feel free to reach out to your Allganize counterpart.
Learn More About Allganize's Technology