When does a conversational analytics bot have AI? (Part 2)

Written by Dr. Anand Ranganathan

In the previous part of this article, we discussed some of the key goals for a conversational analytics chatbot to have AI. We framed the problem in terms of a Turing test, i.e. how well can a conversational analytics chatbot imitate a human data analyst. And we identified 6 kinds of tasks:

• Requirements Gathering and Clarification
• Data Retrieval and Analysis
• Result Presentation
• Iterative Refinement
• Continuous Learning
• Automated, Pro-active Insights

In the previous post, we had gone through the first two kinds of tasks. Here, we’ll proceed with the next four.

Result Presentation

Good data analysts know exactly how to present results to a business user to convey the result of an analysis. For example, they know the best ways to represent a daily trend (e.g., as a line chart), metrics associated with different locations (e.g., as a heatmap), a histogram showing the distribution of the number of customers by their age (e.g., as a bar chart) or comparing different metrics for a few entities (e.g., as a radar chart). They also know the appropriate scales, colors, and sizes of the visualizations that make sense for a given dataset. Finally, they know how to add commentary in natural language to explain their points.

The challenge, now, is for a bot to do these tasks on its own. Furthermore, it has to tailor the presentation to the preferences of different users, for example, using certain corporate brand colors. Generating the most visually pleasing and appropriate visualization for any dataset is no easy task. It requires either learning or encoding deep knowledge about the properties of each kind of widget. The task of generating natural language is challenging too. While there has been some notable progress by companies like Yseop and Narrative Science, there is still a long way to go before a bot might be able to imitate a data analyst in coming up with concise text for a given business user’s needs, especially in a conversational setting.

Iterative Refinement

Business users and data analysts often have a close collaborative relationship. This manifests itself best when the two of them iteratively drill down to find a hidden nugget of insight from the data. For example, a conversation between the two might go as below:

• • • • • Business User (BU): “Compare sales last month vs the previous month”
        • Data Analyst (DA): “Hold on…”

• • • • • (After a few minutes)

• • • • • DA: “Sales last month was $1.5 million, compared to $1.7 million in the previous month, representing a 13.3% drop”
        • BU: “I see. Why did that happen?”
        • DA: “Hold on…”

• • • • • (After a few hours)

• • • • • DA: “Sales in North America was down by 35%, and Europe was down by 20%. Also, sales of red baseball hats was down by 40%. And finally, sales by Ivana Slakoff was down by 70%”
        • BU: “Hmm… who were my best and worst salespeople in North America in this period?”
        • DA: “Hold on…”

• • • • • (After a few minutes)

• • • • • DA: “The best salesperson in the last 2 months was Will U Bight with sales of $180K, while the worst was Luke Warm with sales of $50k”

Now, if a bot needs to imitate a data analyst, it needs to be able to engage in a similar iterative refinement with the business user. It needs to maintain a conversation, where each subsequent question might build upon the previous question. This kind of questioning, also called funnel questioning, often starts with a broad question, and then drills down to a more specific point in each subsequent question. The bot needs to maintain the context of the conversation, and take action accordingly.

Continuous Learning

As a data analyst spends more time working with business users in an organization, she might learn more about the ways the business users might frame requirements, how they prefer to see the results presented, and in general, what kinds of insights they are looking for. For example, the first time a business user asks a data analyst “Which are my best-performing products in North America?”, the analyst might ask for clarifications to determine the exact meaning of “best performing”. For example, did the business user mean by quantity sold, or by sales or by profit or in comparison to previous time periods?  However, once the data analyst has seen a few examples of such requirements and gets clarifications, we might expect her to learn and understand the business user without asking for clarification.

We would expect a bot that imitates a data analyst to do something similar. The bot must learn how business users ask questions, what kinds of insights are interesting to them, what are their favorite ways of visualizing different kinds of statistics, etc.

Automated, Pro-active Insights

Finally, a data analyst doesn’t just respond to questions from business users; she might also explore the datasets to come up with new insights. To do this, she must have some basic understanding of what kinds of insights might make sense to a business user and also, determine if a specific insight is interesting from a mathematical as well as a business perspective. She might then proactively present these insights to a business user, along with some explanation of why they are interesting or relevant or useful.

This is where a bot might actually have an advantage over human data analysts, at least in one aspect. While a human can explore the dataset in a small number of ways to uncover insights, a bot can explore millions of possible combinations of variables in the dataset to uncover potential insights. The main challenge for a bot is determining if any given pattern, anomaly, trend, or other insight on some combination of variables is interesting or relevant from a business point of view. Bots have a fair way to go before they can emulate humans in this regard.

To summarize, we have laid down several characteristics of a conversational analytics bot, if it is meant to have “AI”. Each of these characteristics is challenging computer science research problems on their own. At Unscrambl, we hope to make some basic progress towards them. As Robert Frost would say: These AI woods are lovely, dark and deep, But I have promises to keep, And miles to go before I sleep, And miles to go before I sleep.

Experience the power of conversational analytics with Qbo. Signup for a free trial to get started.