AI

The Bot Orchestrator: The Super-Bot in Managing Multiple AI Bots Efficiently & Successfully

By 9 Minute Read

Bot orchestration

As AI chatbots are rolled out across different departments, regions, or for different business use cases, the issue of coordinating and managing multiple AI models in a consistent and efficient manner becomes evident.

For enterprises that have been early adopters of conversational AI solutions based on natural language processing (NLP) they know that building superior conversational experiences is not as easy as it seems and that there is a progressive path to improvement. As they add additional AI bots across their business or hit the intent limit on a particular NLP engine, the need for a multi-model NLP orchestration architecture emerges.

This blog introduces our powerful multi-model NLP orchestrator, a superhero in enabling the deployment of multiple – of the order of 100s – of NLP bot models in parallel, each with its own training data, intents, and utterances. This is a key solution for businesses that are witnessing some of the common NLP failures as they roll out multiple conversational AI solutions.

The Growth Challenge for Conversational AI Solutions in the Enterprise

When organizations start their journey with natural language or AI bot solutions they usually look at their top business objectives and how NLP and machine learning technology could benefit them in generating some fast business outcomes. This could mean deploying a conversational AI bot to drive increased online conversions, or it could be digitalizing their customer onboarding journeys, or automating common customer service interactions.

The initial bot project varies according to the pressing business needs and the potential for quick wins, resulting in a very task-oriented AI bot that is trained and tuned specifically to respond to and resolve highly targeted user issues. For example, an onboarding bot is built with the skills to gather proof documents and customer information in order to complete their onboarding journey more efficiently whereas a customer service bot is designed to respond to the most common service issues that the company experiences.

As a company reaps the benefits of their initial digital AI assistant project the appetite for additional bots grows and before you know it multiple bots are in production, carrying out different conversations and tasks across departments, regions, or expanding to other use cases. They can have varying NLP engines, from DialogFlow, Amazon Lex, to IBM Watson, not to mention being developed using different platforms, tools, and developers. 

Then the issues of consistency, efficiency, customer experience, and management come to light, often with serious consequences.

The Need for Orchestration of Multiple AI Bots

Challenges of multiple AI Bots

As the implementation of digital AI assistants matures we believe that there is a need for a single manager or NLP orchestrator to help overcome the issues that emerge in a multi-model AI world.

Early forms of the task-oriented AI assistant came with mixed levels of success but, despite the hype, many failed to live up to expectations. In some cases, this AI bot was overloaded with too much functionality (i.e. too many intents in NLP parlance), so rather than doing one thing and doing it well, the bot tried to handle too much, ultimately to the point of chatbot failure.

This was a key challenge as we looked to solve this problem because we knew that human conversations can take many different twists and turns. The NLP orchestrator needs to be able to keep track of the conversation and maintain context, even as the conversation shifts to a different question or task. To achieve this, it needs to grow in terms of scope, i.e. its ability to do more things and manage more natural language conversations using conversational AI. It also needs to deliver better business results and make appropriate decisions for the user.

Our experience shows that this comes down to, not only the key capabilities of the bot manager or orchestrator but its underlying architecture and how it interacts with the multiple task-defined AI bots.

So, we decided that in addition to having many AI assistants that are highly mission-focused, we needed a bot orchestrator or manager that would oversee the conversation and assign tasks to the individual mission bots. This allowed us to solve the problem of adding lots of functionality. We could simply add more narrowly focused task-driven bots that could be successful in their own roles while centralizing tasks like sentiment analysis, small talk, escalation decisions, and language detection.

This also enables different departments in an organization to launch their own mission-focused bots in order to achieve their specific business goals while centralizing brand identity and maintaining a consistent experience to the customer across different digital channels.

Introducing Our Concept of NLP Orchestration

“Multi-model NLP Orchestration is the solution for orchestrating multiple task-oriented AI bots that are capable of individual units of automation.” (ServisBOT)

We started with the idea of “task-oriented AI bots capable of individual units of automation”. As we started building out autonomous solutions, we quickly learned that developing bots that solve a single purpose was both sensible and feasible. This led to our Army of Bots, with individual bots having very specific missions and automating the tasks that are specific to that mission. For example, an Onboarding Bot manages a multi-step customer onboarding journey and automates the capture and verification of documents, a Renewal Bot renews a subscription or membership, and an Account Bot informs customers of the status of their account, and so on.

Then we had to consider how we would manage or orchestrate these bots to provide a single experience to the customer.

  • How do you manage and control a growing number of bots that engage across different customer journeys?
  • How does a particular AI bot know when to engage?
  • How should we avoid having every bot deal with managing sentiment, language, small talk, and other conversational capabilities while automating the tasks at hand?

The answer for us was to centralize certain common tasks to avoid the effort and costs of replicating these in every bot.

Building an intelligent agent, capable of determining the correct intent from a corpus of closely related concepts and orchestrating the appropriate task bots to fulfill this proved to be infinitely more challenging. But we love challenges!

At ServisBOT we think of the NLP orchestrator as the brain of the engagement, handling all the conversations with the customer, employee, partner, or whoever the human user may be. But what does it do with these conversations? It needs to be clever enough to understand the high-level intent and then route accordingly to a bot, or bots, that can execute on this. In other words, the bot orchestrator manages the conversations to and from the user but also orchestrates other bots that are skilled to carry out specific missions or tasks. Hence we consider this super-bot to be the superhero of conversational AI.

Why this is Relevant for the Enterprise?

Like many technologies that started in the consumer space, natural language as an interface has quickly taken root in the enterprise arena as businesses see the value of consumers being able to use natural language, via both voice and chat, to search for things, raise tickets, query bills, transact, and interact in more convenient ways. The opportunity for enterprise is that this technology can be the key to advance automation and self-service for customers and employees, lowering costs, and increasing efficiencies. The problem for the enterprise is that there are so many workflows, processes, and tasks that a single AI bot won’t be enough to handle them all. Hence the need for greater intelligence than is normally bestowed on a single bot that responds to more basic consumer questions relating to things like weather, news, social events, recipes, etc.

Many consumer-centric examples of “what’s the weather in Dublin” exist. But there isn’t an off-the-shelf intent which can “book instrument ED1847 for clinical use next Friday at 2 pm”. The language of an enterprise is special, where proprietary terms and complex descriptions specific to the domain of expertise abound.

Traditional intent detection mechanisms often do not deal well with large volumes of intents, which can lead to intent overlap and ambiguity. However, a multi-bot orchestration model helps overcome the intent limitations of the underlying NLP engine by routing users to the bot that can best handle the intent with a high degree of accuracy. This orchestration enables the business to blend independently managed bots into a unified experience, Hundreds of NLP bot models can be deployed in parallel, each with its own training data, intents, and utterances, even if they are running on different NLP engines. This is a highly powerful way for organizations to architect more sophisticated conversational AI solutions and/or to ensure greater consistency and efficiency across multiple chatbot solutions deployed across different departments or use cases.

Some Characteristics of an Enterprise Bot Orchestrator

So what characteristics do we associate with this Super Bot that can intelligently take on the role of a brand and digital experience ambassador? Here are seven attributes that quickly came to mind. Let’s dig in!

1.  Visibility to the Outside World

One key consideration is how it exposes its capabilities to customers or other users. Think of the digital channels for your conversations.

  • Are you building a purely voice-based bot solution, exposed via a smart speaker in your meeting rooms? This will require integration with the smart speaker platform deployed within your enterprise.
  • How about text-only, embedded on the pages of your corporate intranet? Here, a chat user interface will be needed to expose the experience to your users.
  • What about an assistant available to your corporate chat? Here, look for engagement adapters to these common corporate chat platforms.

2. Language Detection & Routing

Language-specific routing is an extension of a dispatcher capability i.e. it routes to a specific language skill or resource. When native language bots are available, the orchestrator can detect the language of the incoming user, and route them to bots that are capable of handling the ask without the use of translation. This provides a first-class experience, without any of the pitfalls translation can bring.

3. Translation

Another localization capability, bi-directional translation, can enable bots to converse in the native language of a user, without undergoing expensive translation cycles. This is not without pitfalls – accuracy can be reduced, but sometimes this can be an acceptable trade-off by offering substantially lower design & configuration activities.

4. PHI & PII Detection

Occasionally, users share a little more than they should. When a user shares personally identifiable information (PII), or personal health information (PHI) into a chat, the responsibility to detect and act on this information lies with your business. The bot manager should have the ability to automate the redaction of any privileged information your customers may share on inappropriate and unauthenticated channels.

PII detection

5. Expanding the Context

When a customer first reaches out to live chat, we may already know a whole host of information about this user. Prompting your authenticated & logged in users for information that is already known about them creates a frustrating experience. A capable orchestrator will include a conversational context store which can be populated with known information about the user as the conversation is initiated. As a user interacts with a bot, further information can be populated in this context store.

We may find out useful profile information which we’d like stored back in our system of record. This known context may prove useful in determining which intents are relevant to an end-user. We may also learn a user’s preferences along the way, and use this to speed up their future orders. All this is achieved by filling the context store as users complete their conversational journeys.

 

VA Context switching

6. Managing Context Switching

Think of a customer starting a conversation and how the fluidity of chat can quickly change their direction or intent and bring the conversation in different directions.

Even as users begin one intent, they may temporarily switch to another on their path to fulfilling their intent.

Being able to detect this transient switch in intent, then return the user to their previous task is an important skill in reducing the frustration and friction while using an assistant.

 

7. Escalating to a Human

A Bot Manager also needs a path to human handover, as it cannot always replace a human agent. Customers or employees may specifically request to interact with a human agent. Or exceptions and unusual circumstances may arise, requiring the intervention of a human worker. The bot orchestrator needs to slot more naturally into how the organization is already doing business, just as human workers do. To enable this more natural handover, it needs to integrate with existing customer service technologies.

The Benefits of an NLP Orchestrator

I hope you’ve been interested in our approach to Multi-model NLP Orchestration. There is much more to this than meets the eye but if done correctly the bot orchestrator can be a real superhero for your conversational AI projects, delivering some clear benefits, such as:

  • Creating a Consistent Brand Identity: Since the orchestrator handles all the conversations to and from the user, it acts as a brand ambassador across all customer interactions, providing a consistent brand experience no matter what language, region, use case, or product an enterprise supports.
  • Providing a Scalable and Efficient Multi-Bot Architecture: The orchestrator is at the forefront of customer conversations and routes to the appropriate bot or bots according to the intent. This layered architecture is highly scalable and efficient. It leverages the benefit of single-purpose bots that work well rather than a single over-stuffed bot.
  • Getting Bots to Market Faster: With the central bot orchestrator having a large set of capabilities, it then becomes faster and easier for the business to build and deploy individual or task-oriented bots to meet business demand. 
  • Making Bots Smarter: This architectural approach is also critical to bots training and gaining skills faster. Capabilities can be added to each bot over time to make them smarter and more successful.

Our Multi-Model NLP Architecture takes an interesting approach to solving many of the challenges of enterprise bot implementations especially around multi-bot models that require a collaborative orchestration approach. 

If you’d like to learn more, please check out some of our resources.

To discover more about our Bot Platform and how it can help your organization build and orchestrate your chatbots more efficiently, please contact us.

Close this Window