AI

How Virtual Assistants Manage Multiple AI Bots Efficiently & Successfully

As AI chatbots are rolled out across different departments, regions, or for different business use cases, the issue of coordinating and managing multiple AI models in a consistent and efficient manner becomes evident.

For enterprises that have been early adopters of conversational AI tools based on natural language processing (NLP) they know that building superior conversational experiences is not as easy as it seems and that there is a progressive path to improvement. As they add additional AI bots across their business or hit the intent limit on a particular NLP engine, the need for a multi-model NLP orchestration architecture emerges.

This blog introduces our powerful multi-model NLP orchestrator, a superhero in enabling the deployment of multiple – of the order of 100s – of NLP bot models in parallel, each with its own training data, intents, and utterances. This is a key solution for companies that are witnessing some of the common NLP failures as they roll out multiple conversational AI solutions.

The Growth Challenge for Conversational AI Solutions in the Enterprise

When organizations start their journey with natural language or AI bot solutions they usually look at their top business objectives and how NLP and machine learning technology could benefit them in generating some fast business outcomes. This might involve digitalizing their customer onboarding processes, automating frequent customer support interactions, or deploying a conversational AI bot to enhance online conversions.

The initial bot project differs depending on the pressing business needs and the potential for quick wins, resulting in a very task-oriented AI bot that is trained and tuned specifically to answer to and resolve highly targeted user issues. For example, an onboarding bot is built with the skills to gather proof documents and customer information in order to complete their onboarding journey more efficiently whereas a customer service bot is designed to respond to the most common service issues that the company experiences.

As a company reaps the benefits of their initial digital AI assistant project the appetite for additional bots grows and before you know it multiple bots are in production, carrying out different conversations and tasks across departments, regions, or expanding to other use cases. They can have varying NLP engines, from DialogFlow, Amazon Lex, to IBM Watson, not to mention being developed using different platforms, tools, and developers. 

Then the issues of consistency, efficiency, customer experience, and management come to light, often with serious consequences.

The Need for Orchestration of Multiple AI Bots

As the implementation of virtual assistants matures we believe that there is a need for a single manager or NLP orchestrator to help overcome the issues that emerge in a multi-model AI world.

Early forms of the task-oriented AI virtual assistant came with mixed levels of success but, despite the hype, many failed to live up to expectations. In some cases, this AI bot was overloaded with too much functionality (or, in NLP parlance, there were too many intents), which prevented it from focusing on any one task at a time and completing it properly. As a result, the chatbot eventually failed because it tried to do too much.

We recognized that human conversations can go in a variety of different directions, so this was a crucial obstacle as we tried to find a solution. Even when the topic of the conversation changes to another inquiry or activity, the NLP orchestrator must be able to keep track of the conversation and preserve context. It must expand in terms of scope, i.e., its capacity to carry out more tasks and lead more natural language discussions utilizing conversational AI, in order to accomplish this. Additionally, it must produce better commercial outcomes and make user-appropriate judgments.

Our experience shows that this comes down to, not only the key capabilities of the chatbot manager or orchestrator, but its underlying architecture and its interaction with the multiple task-defined AI bots.

So, we decided that in addition to having many virtual assistants that are highly mission-focused, we needed a bot orchestrator or manager that would oversee the conversation and assign tasks to the individual mission bots. This allowed us to solve the problem of adding lots of functionality. We could simply add more narrowly focused task-driven bots that could be successful in their own roles while centralizing tasks like sentiment analysis, small talk, escalation decisions, and language detection.

This also enables different departments in an organization to launch their own mission-focused bots in order to achieve their specific business goals while centralizing brand identity and maintaining a consistent experience to the customer across different digital channels.

Introducing Our Concept of NLP Orchestration

“Multi-model NLP Orchestration is the solution for orchestrating multiple task-oriented AI bots that are capable of individual units of automation.” (ServisBOT)

We started with the idea of “task-oriented AI bots capable of individual units of automation”. As we started building out autonomous solutions, we quickly learned that developing bots that solve a single purpose was both sensible and feasible. This led to our Army of Bots, which includes individual bots having very specific missions and automating the tasks that are specific to that mission. For example, an Onboarding Bot manages a multi-step customer onboarding journey and automates the capture and verification of documents, a Renewal Bot renews a subscription or membership, and an Account Bot informs customers of the status of their account, and so on.

Then we had to consider how we would manage or orchestrate these bots to provide a single experience to the customer.

• How do you manage and control a growing number of bots that engage across different customer journeys?
• How does a particular AI bot know when to engage?
• How should we avoid having every bot deal with managing sentiment, language, small talk, and other conversational capabilities while automating the tasks at hand?

The answer for us was to centralize certain common tasks to avoid the effort and costs of replicating these in every bot.

Building an intelligent agent, capable of determining the correct intent from a corpus of closely related concepts and orchestrating the appropriate task bots to fulfill this proved to be infinitely more challenging. But we love challenges!

At ServisBOT we think of the NLP orchestrator as the brain of the engagement, handling all the conversations with the customer, employee, partner, or whoever the human user may be. But what does it do with these conversations? It needs to be clever enough to understand the high-level intent and then route accordingly to a bot, or bots, that can execute on this. In other words, the bot orchestrator manages the conversations to and from the user but also orchestrates other bots that are skilled to carry out specific missions or tasks. Hence we consider this super-bot to be the superhero of conversational AI.

Why this is Relevant for the Enterprise?

Like many technologies that started in the consumer space, natural language as an interface has quickly taken root in the enterprise arena as businesses see the value of consumers being able to use natural language, via both voice and chat, to search for things, raise tickets, query bills, transact, and interact in more convenient ways. The opportunity for enterprise is that this technology can be the key to advance automation and self-service for customers and employees, lowering costs, and increasing efficiencies. The problem for the enterprise is that there are so many workflows, processes, and tasks that a single AI bot won’t be enough to handle them all. Hence the need for greater intelligence than is normally bestowed on a single chatbot that responds to more basic consumer questions relating to things like weather, news, social events, recipes, etc.

Many consumer-centric examples of “what’s the weather in Dublin” exist. But there isn’t an off-the-shelf intent which can “book instrument ED1847 for clinical use next Friday at 2 pm”. The language of an enterprise is special, where proprietary terms and complex descriptions specific to the domain of expertise abound.

Traditional intent detection mechanisms often do not deal well with large volumes of intents, which can lead to intent overlap and ambiguity. By sending users to the bot that can handle their intent the most accurately, a multi-bot orchestration strategy aids in overcoming the intent restrictions of the underlying NLP engine. Numerous NLP bot models can be deployed concurrently, each with their own training data, intents, as well as utterances, regardless if they are powered by various NLP engines. This orchestration enables the business to combine independently managed bots into a cohesive experience. This is a highly powerful way for organizations to architect more sophisticated conversational AI solutions and/or to ensure greater consistency and efficiency across multiple chatbot solutions deployed across different departments or use cases.

Some Characteristics of an Enterprise Bot Virtual Assistant

So what characteristics do we associate with this Super Bot that can intelligently take on the role of a brand and digital experience ambassador? Here are seven attributes that quickly came to mind. Let’s dig in!

1.  Visibility to the Outside World

One key consideration is how it exposes its capabilities to customers or other users. Think of the digital channels for your conversations.

• Are you building a purely voice-based bot solution, exposed via a smart speaker in your meeting rooms? This will require integration with the smart speaker platform deployed within your enterprise.
• How about text-only, embedded on the pages of your corporate intranet? Here, a chat user interface will be needed to expose the experience to your users.
• What about a virtual assistant available to your corporate chat? Here, look for engagement adapters to these common corporate chat platforms.

2. Language Detection & Routing

Language-specific routing is an extension of a dispatcher capability i.e. it routes to a specific language skill or resource. When native language bots are available, the orchestrator can detect the language of the incoming user, and route them to bots that are capable of handling the ask without the use of translation. This provides a first-class experience, without any of the pitfalls translation can bring.

3. Translation

Another localization capability, bi-directional translation, can enable bots to converse in the native language of a user, without undergoing expensive translation cycles. This is not without pitfalls – accuracy can be reduced, but sometimes this can be an acceptable trade-off by offering substantially lower design & configuration activities.

4. PHI & PII Detection

Occasionally, users share a little more than they should. When a user shares personally identifiable information (PII), or personal health information (PHI) into a chat, the responsibility to detect and act on this information lies with your business. The bot manager should have the ability to automate the redaction of any privileged information your customers may share on inappropriate and unauthenticated channels.

5. Expanding the Context

When a customer first reaches out to live chat, we may already know a whole host of information about this user. Prompting your authenticated & logged in users for information that is already known about them creates a frustrating experience. A capable orchestrator will include a conversational context store which can be populated with known information about the user as the conversation is initiated. As a user interacts with a bot, further information can be populated in this context store.

We may find out useful profile information which we’d like stored back in our system of record. This known context may prove useful in determining which intents are relevant to an end-user. We may also learn a user’s preferences along the way, and use this to speed up their future orders. All this is achieved by filling the context store as users complete their conversational journeys.

 

6. Managing Context Switching

Think of a customer starting a conversation and how the fluidity of chat can quickly change their direction or intent and bring the conversation in different directions.

Even as users begin one intent, they may temporarily switch to another on their path to fulfilling their intent.

Being able to detect this transient switch in intent, then return the user to their previous task is an important skill in reducing the frustration and friction while using an intelligent virtual agent.

 

7. Escalating to a Human

A Bot Manager also needs a path to human handover, as it cannot always replace a human agent. Customers or employees may specifically request to interact with a human agent. Or exceptions and unusual circumstances may arise, requiring the intervention of a human worker. The bot orchestrator needs to slot more naturally into how the organization is already doing business, just as human workers do. To enable this more natural handover, it needs to integrate with existing customer service technologies.

The Benefits of an Virtual Assistant

I hope you’ve been interested in our approach to Multi-model NLP Orchestration. There is much more to this than meets the eye but if done correctly the bot orchestrator can be a real superhero for your conversational AI projects, delivering some clear benefits, such as:

• Creating a Consistent Brand Identity: Since the orchestrator handles all the conversations to and from the user, it acts as a brand ambassador across all customer interactions, providing a consistent brand experience no matter what language, region, use case, or product an enterprise supports.
• Providing a Scalable and Efficient Multi-Bot Architecture: The orchestrator is at the forefront of customer conversations and routes to the appropriate bot or bots according to the intent. This layered architecture is highly scalable and efficient. It leverages the benefit of single-purpose bots that work well rather than a single over-stuffed bot.
• Getting Bots to Market Faster: With the central bot orchestrator having a large set of capabilities, it then becomes faster and easier for the business to build and deploy individual or task-oriented bots to meet business demand. 
• Making Bots Smarter: This architectural approach is also critical to bots training and gaining skills faster. Capabilities can be added to each bot over time to make them smarter and more successful.

Our Multi-Model NLP Architecture takes an interesting approach to solving many of the challenges of enterprise bot implementations especially around multi-bot models that require a collaborative orchestration approach. 

If you’d like to learn more, please check out some of our resources. We have also written a post about the differences between chatbots and virtual assistants.

To discover more about our AI Bot Platform and how it can help your organization build and orchestrate your chatbots more efficiently, please contact us.