When does a conversational analytics bot have AI? (Part 1)

Written by Dr. Anand Ranganathan

Artificial Intelligence or AI is one of the hottest buzzwords today, but also one whose meaning is most open to interpretation. Generally, AI is used to describe systems that have cognitive capabilities that are associated with humans, like reasoning, learning, planning, and problem-solving. In other words, these systems can perform tasks that humans would agree, need “intelligence”.

However, the tasks that can indicate “AI” change frequently. This is often known as the AI effect, i.e., as soon as AI successfully solves a problem, the problem is no longer a part of AI. For example, systems that play chess or do character recognition or fingerprint recognition are often no longer considered AI. Douglas Hofstadter has a quip on Tesler’s Theorem:  “AI is whatever hasn’t been done yet.” (Lieberson and Hofstadter 1980)

This brings us to the topic of when does a chatbot has AI. At Unscrambl, we have been building chatbots for several years. We actually focus on a very specific kind of chatbot – that answers analytical questions on data in databases. This is also known as a conversational analytics bot.

Our product, Qbo, allows business users to ask data and analytics-related questions about data in databases, and it gives back answers using a combination of charts and text. One of the common questions we get asked after people see or use our system is: “Where is the AI?” or “Is that AI?”. So, let’s examine this question in more detail.

Through most of recent history, there have been several attempts to determine if a machine has AI. One of the more influential and well-known methods is the Turing test. The essence of the Turing test is an “imitation game”, i.e. can a machine imitate a human well enough that a human evaluator cannot distinguish it from a real human.

While there are various limitations to the use of a Turing test to detect intelligence, it does provide a good framework for answering the question “Does a conversational analytics system have AI?”. In this case, we refine the definition of the test as follows: “Can a conversational analytics system imitate a data analyst well enough that a business user cannot distinguish it from a real human (i.e. a real data analyst)?“

To answer this question, let’s first examine the typical tasks of a data analyst. Data analysts might take questions or analysis requirements from business users. Usually, these requirements are expressed in natural language (e.g. via email or chat or speech) and very often, they are framed in a loose or ambiguous manner. The data analyst first tries to clarify these requirements, along with the business user, to make them more precise. She may then retrieve data from various sources, organize them, perform analysis, reach conclusions, and present back charts or reports. This process might happen iteratively until the business user’s needs are met. As the analyst keeps working with a business user or in an organization, she might get more and more familiar with the data and needs of the business user and might produce the analysis needed with fewer clarifications or iterations. Finally, the data analyst might proactively discover interesting patterns or anomalies in the data and report them to the business user.

Now, let’s look at what a conversational analytics system must do to “imitate” a data analyst. We’ll break down the tasks as follows:

• Requirements Gathering and Clarification
• Data Retrieval and Analysis
• Result Presentation
• Iterative Refinement
• Continuous Learning
• Automated, Pro-active Insights

Requirements Gathering and Clarification

Conversational Analytics systems allow gathering requirements from business users in natural language. Usually, these requirements are in the form of questions. The questions might be precise like “How many customers bought Apple devices in the Northeast US in the last 3 months?” or vague like “What’s going on with Apple sales?”.  They may require diagnostic analysis like “What are the top factors leading to decline in Apple sales?”, predictive analysis like “What is the forecast of Apple sales over the next 3 months?”, or prescriptive analysis like “How can I increase Apple sales?”.

So, one measure of how well a conversational analytics system does is how well it understands various kinds of business user requirements. Firstly, language is often associated with “intelligence”. Hence, a natural language or conversational system can be considered to have more “AI” than a form-based system. The additional intelligence, in this case, is needed to process natural language, understand the user’s intent, and come up with the right response.

A key aspect of requirements gathering is handling imprecise or ambiguous questions. This is what humans do very well, but bots often struggle with it. So, for example, when a user asks “What is the revenue in the last month?”, there is some ambiguity on how “last month” is interpreted. For example, if today is August 5, It could mean the last calendar month (July 1-31), or it could mean the period from July 6 to August 5. Hence, the bot needs to either clarify the interpretation from the end-user or make clear which interpretation it picks.

Another example is if the user asks “What is the revenue in the last month?”, and there happens to be several attributes or metrics defined in the data called “gross revenue”, “net revenue” and “revenue before interest and taxes”. Again, in this case, the bot needs to figure out the possible interpretations from the dataset, and then clarify from the user which interpretation of “revenue” it should consider.

Another common example we see is a user just asking a question like “How is India doing?”. The user implicitly wants to get information like sales or revenue in India, but this is not clear from the question at all. Here again, the bot needs to clarify from the end-user, and ideally, learn this particular phrasing for that user.

Data Retrieval and Analysis

Given a question from a business user, a data analyst might first need to identify which datasets might hold the required information. Then, she might need to compose queries in SQL or other languages or use different tools to extract the required data, understand it, analyze it, and produce the required results.

Now, for a bot to do these tasks, it needs to have a good understanding of the various datasets, including their schema and their semantics. It needs to know about the tables, attributes, and values; as well as the “semantics” of the data in the tables. For example, if there is an amount column in a sales table, it needs to know the currency associated with the amount, even if that currency information is not explicitly mentioned in the database. Or if there is a column called the city, it should understand that the values in the column probably correspond to city locations and could be associated with a latitude and longitude or other geographical information.

The bot will need to discover the semantics in a semi-automated manner. Some kinds of semantic information can be inferred like cities are locations. Others may need explicit hints from a data analyst, like the currency of an amount column.

Once the bot has a good understanding of all the available datasets, it can then attempt to convert a business user’s requirements into the appropriate queries on the appropriate datasets followed by appropriate additional analysis, as needed. It will need to build the queries by first identifying what raw datasets are available, and then what operations (like filters, aggregations, group-bys, sorting, trend analysis, etc.) would need to be performed on the raw data to produce the results.

I’ll end this blog post here. To summarize, so far, I have defined the question of “does a conversational analytics bot have AI” in terms of a Turing test. I have identified various characteristics that such a bot must follow for it to have AI. And I’ve gone through the first two points in detail. See you at the next post on a discussion of the remaining four points.

Experience the power of conversational analytics with Qbo. Sign up for a free trial to get started.