Mind Your Ands and Ors: The Logical Vs. The Colloquial

Written by Dr. Anand Ranganathan

At Unscrambl, we believe that conversational interfaces are the new frontier in how humans interact with computers. We have spent a considerable amount of time over the past several years in building and refining our conversational analytics product, called Qbo. During this time, we have encountered several technical and human challenges, learned a lot about how humans can converse with bots for the purpose of gaining analytical insights, and developed a few strategies to smoothen the interaction.

At its core, Qbo has a natural language processing pipeline for translating free-form natural language queries into a sequence of one or more SQL queries. This allows business users to ask analytical questions against their databases in natural language and quickly get back results, without having to depend on an IT team or SQL / BI developers. However, as part of this process, we needed to understand how users can have analytical conversations with bots, and also figure out how to translate vague, ambiguous natural language queries into precise, logical SQL queries.

In this sequence of blog posts, we shall explore some of our experiences in developing a conversational AI product. We’ll cover various topics around NLP, Analytics, Machine Learning, AI, UX, and Collaboration. We have encountered challenges and had to make advancements in all these areas in the course of building a usable conversational analytics product.

In this post, we shall focus on one of the key NLP challenges: the inherent ambiguity of human language. Something is ambiguous when it can be understood in two or more possible senses or ways. Ambiguity has been the source of much frustration, bemusement, and amusement for philosophers, lexicographers, linguists, cognitive scientists, literary theorists and critics, authors, poets, orators, and just about everyone who considers the interpretation(s) of linguistic signs (Sennet 2011). From an NLP perspective, two kinds of ambiguities are especially interesting: lexical ambiguity and structural ambiguity (“Language Ambiguity” n.d.)

Lexical Ambiguity relates to ambiguity in a single word (and in some cases, a phrase). For example, ‘duck’ can be a verb or an animal. ‘Note’ can be a musical tone or a piece of writing. It often arises from “homonymy”, i.e. the same word having multiple meanings.

Structural (or syntactic) ambiguity comes from being able to parse a sentence in different ways. For example, “John enjoys painting his models nude.” In this case, it is ambiguous who is nude: John or his models. Or “The chicken is ready to eat.” (who is going to do the eating?).

We encounter both these kinds of ambiguities all the time in our conversational analytics product. For example, a marketing user exploring data about her customers might ask a question like “Number of customers over 10 years”. Did she mean “Number of customers whose age is over 10 years old” or did she mean “Number of customers who have been with my company for more than 10 years”. This is an example of lexical ambiguity associated with the word “over”.

Or a wealth manager in an investment bank might ask a question like “Top 5 account number of transactions”. Did she mean “Top 5 accounts in order of the number of transactions that occurred in the account” or “Top 5 account numbers of the transactions in order of the account numbers”? This is an example of structural ambiguity where it’s not clear what metric is to be used to find out the top 5.

One of the more insidious examples of lexical ambiguity that we encountered was in the different interpretations of “and” and “or”. On the face of it, these humble conjunctions seem to be pretty unambiguous, especially for those with a computer science background who might have spent several courses playing around with logical or boolean “ands” and “ors”. However, we soon found out that there is much more ambiguity in these words than what is immediately apparent.

For example, let’s say there’s a marketing analyst who has a query in a natural language like: “List all my customers from Germany and France”. Let’s assume there’s a table called “Customer” in a database with the following information:

So, when a user asks the question: “List all my customers from Germany and France”, then she might reasonably expect an answer like “Hans, Emmanuel, Max”. As it turns out, the colloquial “and” in this case actually corresponds to the logical “or”. So, the correct SQL query to produce this result is:

Select CustomerName from Customer where Country = ‘Germany’ OR Country = ‘France’

Note the logical OR in this query. If we had used an AND instead in the SQL query, we would not get any results. Hence, the colloquial “List all my customers from Germany and France” is equivalent to the logical “List all my customers from Germany or France”.

In fact, the word AND can be interpreted as a logical XOR (Exclusive Or) in this particular case since each customer can have exactly one country.

Let’s take another query: “List all my customers from Germany and France and over 32 years old”. In this case, a common interpretation might be: “List all my customers from either Germany or France and who are over 32 years old”, leading to an SQL query:

Select CustomerName from Customer where (Country = ‘Germany’ OR Country = ‘France’) AND (AGE > 32)

So, the first “and” got interpreted as a logical OR, and second “and” as a logical AND.

Let’s take a different table called Purchases:

Let’s say a business user asks a question: “List all my customers who bought Strudels and Bratwursts”.

One common interpretation is:  “List all my customers who bought both Strudels and Bratwursts”. This is the logical AND interpretation but observe that now that the colloquial “and” is equivalent to the logical “AND”, whereas in the first example in the Customer table, the colloquial “and” was equivalent to the logical “OR”.

However, there’s also the chance that she meant “List all my customers who bought at least one of Strudels and Bratwursts”. In the first case, the answer may be “Hans, Max”, and in the second case, the answer is “Hans, Max, Emmanuel”.

Now consider the business user asking the question: “List all my customers who bought Strudels or Bratwursts”. Does she mean “List all my customers who bought either Strudels or Bratwursts (but not both)”, or does she mean “List all my customers who bought Strudels or Bratwursts or both”?

In our product Qbo, we needed to deal with these ambiguities. Our approach has been two-fold:
a) to see if context can help disambiguate, and
b) to embrace the ambiguity in colloquial language.

First, we see if the context of the query (i.e. the entities, attributes, relationships of the query, and the data model) helps us disambiguate. If a user’s query still has multiple possible interpretations, we would present various alternatives to the end-user. The important thing is that when we present the alternative interpretations back to the end-user, they must be expressed in as unambiguous a manner as possible. Our use of a controlled natural language (which is essentially a constrained subset of natural language) to express queries unambiguously has proven to be very useful in this context.

Experience the power of conversational analytics with Qbo. Sign up for a free trial to get started.