[Updated] Alli User Guide - Agent Feedback for Q&A and Documents Search Result

1/21/2021 - Document Query Training guide is added. Click here to see the guide.

To fine-tune Alli's AI, agents can give positive or negative feedback to any Q&A or Documents search results. When an agent gives feedback to a search result, a certain amount of adjustment is made to the 'Adjusted Score' of the search result. The adjustment is combined with feedback from customers and other agents, decides the order of the search results for the search query and the similar queries. You can read Alli User Guide - Letting Customer Feedback Affects Q&A Search Result to understand customer feedback and the Adjusted Score better.

There are two ways for agents to give feedback to the Q&A search result and the Documents search result. One is using the search preview, the other is using the Query Training menu.

Using the Search Preview: Q&A

When you try the Q&A search preview in the dashboard under Knowledge Base > Q&A, you can see the thumb up/down icon at the bottom right side of each Q&A listed.

You can simply click up or down on each Q&A to give positive or negative feedback. After you give your feedback, try the same search query again. You'll see the Adjusted Score has changed and is affecting the order of the search results.

From the screenshot above, you can click on the up icon again to cancel your feedback or the down icon to change your feedback to negative. Please remember that other agents can also give feedback for the same query. The total number of feedbacks will be shown next to the thumb up or down icon.

Using this way, you can give feedback to the top Q&A search results quickly. By the way, if you want to give your feedback to Q&A items not listed in the preview result, or if you want to manage feedbacks for all search queries that have any feedback, you can use the Query Training Menu. Please check the Query Training Menu section in this user guide.

Using the Search Preview: Documents

Giving Agent feedback to the Documents search result is pretty much the same as giving feedback to Q&A search result from the search preview. Go to Knowledge Base > Documents and enter your search query to see the result. You can see the thumb up/down icon on the bottom right side of each Document search result.

You can click up or down on each search result to give positive or negative feedback. Please remember that other agents can also give feedback for the same query. The total number of feedbacks will be shown next to the thumb up or down icon.

After you give your feedback, try the same search query again. You'll see the Adjusted Score has changed and is affecting the order of the search results.

From the screenshot above, you can click on the up icon again to cancel your feedback or the down icon to change your feedback to negative.

For Documents search too, you can use the Query Training menu to manage feedbacks for search results. You can also edit the search result or even create a new answer using the menu. Check out the guide below for more details.

Query Training Menu

Using the Query Training menu, agents can easily check and manage the agent feedbacks given for the Q&As and the Documents, or train new queries from the scratch. Go to Knowledge Base > Query Training. You can see the queries you already trained are listed under the 'Trained Query List' with the total number of positive and negative feedbacks for each query.

The number of positive and negative feedbacks for trained queries includes feedbacks for both Q&A search results and the Documents search result. For example, If the 'why machine learning?' query has one positive feedback for Q&A and two for Documents, the number of positive feedback will be shown as three.

• To check and manage the feedback already given, click the query you want to check from the Trained query list.
• To train a new search query, type in the query in the top field and click the Train button.

Either way, you will see the below screen:

From here you can manage feedbacks for the query being trained. Select a tab between the Q&A and the Documents to start managing feedbacks (the Q&A tab is selected as default). Click the back icon to finish training for the query and get back to the previous screen.

Query Training Menu: Q&A

All the Q&As in your database will be listed under the Q&A tab here. The Q&A with positive feedbacks will be listed at the top, then the ones with negative feedbacks, then the ones with no feedback.

You can click the thumb up or down icon to give feedback to each Q&A item. If necessary, use the keyword search bar to find the Q&A you'd like to give feedback. Or you can also check all the Q&As one by one, depending on the occasion.

• Each agent can give one feedback per each Q&A item for the query. The total number of positive and negative feedbacks will appear next to the thumb up and down icon.
• The feedback you gave is shown as purple-colored icon. If you click the feedback you already selected, you can cancel your feedback. For example, from the screenshot above, you can click the thumb up icon to cancel the feedback, or click the down icon to change your feedback to negative.

Once you finish giving feedbacks, feel free to go back to the Q&A tab and try the search query you just trained. You'll see that the feedback you just gave is affecting the Adjusted Score and the search result.

Query Training Menu: Documents

After selecting a trained query or typing in a new query, click the Documents tab in the next screen. Due to the nature of the Documents search, how to give feedback to the Document search result is a lot different from the Q&A's case. Unlike Q&A, it is impossible to list every potential answer from every uploaded document. If you never worked on the query you are training, you'll see the same document search result you can see under the Knowledge Base > Documents menu for the search query you're working on.

It means the search result you're seeing here is still following the 'Configure Documents' settings, so the potential answer you're looking for to give feedback may not be shown here due to the Threshold and the Number of answers settings (please see this user guide for more details). Then how can you give feedback to a certain part of the document that you know is the right answer to the query? There are three ways.

1. Use keyword search.
   Using the keyword search bar, you can search for the specific Document search result you're looking for, even when that result doesn't have enough Adjusted Score to be shown in the list. It is because the keyword search here works like a filter on the whole search results, ignoring the Threshold settings.
2. Edit an answer.
   If the best answer you found from the search result is still not good enough, you can click the edit icon to edit the answer. Edited answers will have a higher Adjusted Score and will automatically get positive feedback from the agent who edited the answer.
3. Create an answer.
   If you cannot find a good answer you are looking for even after trying #1 and #2, you can compose an answer by yourself. Click the 'Create my own answer' button to start. The created answer will also have a higher Adjusted Score and will automatically get positive feedback from the agent who created the answer.

Here are a few more notes about Documents query training:

• If there is any, the created answers appear first in the list, then the edited ones. The original Documents search results appear after them.
• The created and the edited answers won't be counted in for the Number of answers in Configure Documents settings.
• When you click the preview icon on the edited answer, you'll see the general preview of the document where it's based, and the message stating that this answer is an edited one.
• Created answers don't have the preview icon since it's not based on any document.
• Created and edited answers have trash icon to remove it, the 'Created answer' tag, and the information about created/edited time & the last contributed agent.

Thank you for checking out this long user guide! I hope it worth your time. Now you can fine-tune your AI any time using this feature to achieve even higher AI accuracy. For example, you can check the Candidates tab regularly and see if any unanswered questions actually can be answered by the existing Q&A database, and train the AI for those questions. Or you can think of possible customer questions by yourself and use those questions for training to get Alli ready before opening to customers.

Learn More About Allganize's Technology

• Alli AnswerBot
• Cognitive Search
• NER API
• Review Analysis
• Sentiment Analysis API
• Text Classification API