Bots
5 Use Cases for Multi-Bot Architecture: Driving Better Customer Service Outcomes
Alleviating the Pressure on Customer Service via Conversational AI
Conversational AI and chatbots continue to grow in popularity as companies continue to look for ways to deliver better service to customers, partners, and employees in the current disruptive environment. The opportunity to bring service levels back to normal and alleviate the pressure on customer service organizations by digitizing repeatable or common customer service requests is still largely untapped and customers frequently complain that chatbot experiences are underwhelming.
Why are companies unable to maximize the benefits of digitization and deliver better customer experiences?
Overcoming Chatbot Challenges with Multi-Bot Architectures
We believe that part of the problem lies in the uni-dimensional (or monolithic) architecture of many chatbot applications. Like a well-designed software application, chatbot architecture should evolve from a detailed understanding of the task’s purpose.
In a recent Medium article by AWS expert, Arte Merritt on The Future of Conversational AI multi-bot orchestration was highlighted as one of the newest and biggest trends of 2022 with Sherry Comes of Deloitte Consulting noting that “too many companies have everything in one stack, and those days are in the past. Chatbots need to be able to communicate across stacks, and with each other.”
In this blog, I will explain the drawbacks of monolithic bot architectures, the concept of multi-bot architecture, and their benefits. In doing so I will also outline 5 use cases where a multi-bot architecture is especially suitable and applicable, leading to better customer and business outcomes.
The Problem with Monolithic Chatbot Architectures
In a traditional chatbot solution, the developer builds a chatbot by training a single NLP inference engine with a set of intents (i.e. what the user wants) and utterances (how the user asks for it) that reflect the scope and purpose of the chatbot.
Fulfillment (i.e. the response to the user request) is tied to the intent, such that if a customer asks “what is the latest balance on my account”, the chatbot might interpret the intent as “balance inquiry” and look up the customer’s balance in some IT application and respond with “your current balance is 1234”. So far so good.
Well-known chatbots such as Bank of America’s Erica or Royal Bank of Scotland’s CORA are trained on multiple intents to try and automate as many customer queries as possible and at a large scale.
The problem with monolithic architectures, particularly in the AI world, is that you can hit limits as to how much functionality you can add to the chatbot before it becomes difficult and costly to maintain and/or expand.
In the NLP world, there are practical limits to the number of intents and utterances you can add to a chatbot before confusion and intent overlap begin to affect the efficacy of the bot, leading to incorrect results. For chatbots with large training data sets, adding a new intent or additional utterances to existing intents can often break the model, necessitating leading to a whole new optimization and training process.
Other problems with monolithic architectures include a lack of reuse, tightly coupled fulfillment actions, a lot of custom coding, and sometimes a lack of transparency on how AI decisions are actually made.
Here’s some related content on some reasons for chatbot failure and how to overcome it and build better experiences, faster.
What is Multi-Bot Architecture?
In a multi-bot architecture, the scope of work is distributed across a number of individual chatbots with a single orchestrator or dispatcher at the top, managing a single conversation with the customer.
We are in effect sharding the intents and utterances across a number of bots with each bot focusing on a subset of the tasks. Think of them as skills-based bots – one might be responsible for dealing with billing queries while another might be responsible for technical queries and yet another for sales questions.
Organizations do this today in contact centers already – skills-based call routing. And now we start to introduce re-use – building a set of features such as id&v or data access into a single reusable bot that can be called by other bots.
The key to this multi-bot architecture is the dispatcher which controls the conversation and must decide which bot to send the request to. The dispatcher’s job is not to answer the query but just to decide, which bot is most likely able to respond to the request. It has its own inference model and can use automatic machine learning to get better over time e.g. if the dispatcher passes a request to a bot that cannot answer the query, it moves to the next most probable bot (using the confidence score) and can build up a history of which bot is capable of answering which type of query quite quickly.
Other added features of this architecture include the ability to centralize certain functions such as human handover, small talk, knowledge base lookup, or fallback steps. Pipelines can be introduced to manage sentiment, language detection, PCI or PHI redaction, etc.
Okay. Now that we have this picture of what a multi-bot architecture is, let’s look at some use cases where it makes sense to apply it.
The Five Use Cases for Multi-Bot Architecture
1.Intent Distribution (Intent Sharding)
The first use case is the most common – we call it Intent Distribution. This is where we apply the multi-bot platform architecture to cases where there is a large number of intents, where there are multiple products or service lines, or where there is constant change requiring regular updates and training.
By sharding i.e. distributing the intents across a number of bots, each with its own inference engine, we can greatly improve accuracy and lower the costs and time needed to maintain.
Accuracy improves because we can cluster the intents at design time and measure how closely related they are – and therefore reduce confusion. The solution is easier to maintain because we don’t have to retrain every single model each time – you only have to retrain the model that is impacted – and with smaller models, training is faster. The dispatcher will only need to be re-trained if new bots are added or new intents.
2. Multiple NLP Platforms
The second use case – Multiple NLP Platforms – involves multiple bots that have been developed, often separately and by different teams, on different NLP platforms.
This can come about because business units chose their own tools, sometimes due to the availability of certain functionality on a particular NLP engine, or because of inherited technology as a result of acquisitions.
Whatever the reason, we see situations where Lex bots sit alongside IBM Watson bots and Google’s Dialogflow. Salesforce Einstein bots may have been chosen because of their close integration with their application namesake while open-source aficionados may have chosen RASA because of the community.
Whilst there is often a desire to standardize on a single technology, it may not be possible within a short time frame. Being able to orchestrate different bots on different NLP engines while providing a single contact point to the customer or employee is a good use case for multi-bot orchestration. We often see this use case in multinational corporations that encourage local country or business unit decision making.
The benefits of a multi-bot platform architecture include centralized governance and security, creation of standards, a single platform, reusable components, portability, and more It frees the business to choose whatever tool fits the requirement but allows IT to introduce standards and corporate governance.
3. Language Variations
A third use case involves Language Variations, in particular the ability to support multiple languages from a single access point.
The multi bot platform architecture makes it easy to support multi-language by routing the request to the correct language bot which has been trained with native utterances and intents.
There are a number of ways to handle language – including real-time detection and translation to a base language such as English or Spanish. But translating utterances and responses does not always provide a natural experience – not everything translates well to/from another language. So having the option of using native language bots for the main languages that need to be supported and using translation for minor languages is a great way to provide the best experience to the most number of users.
The benefits in this use case come from higher efficacy (by using native language bots rather than translation) and lower development & maintenance costs (compared to using Multilingual Neural Machine Translation to handle language variations)
4. Jurisdictional Requirements
The fourth use case relates to Jurisdictional Requirements and use cases that need to adjust their responses based on local city, state, or even country regulatory or legal frameworks.
These may include healthcare-based use cases, real estate use cases, or local government use cases. The core features may be centralized with individual jurisdiction requirements being split out depending on the location of the user. The core advantage here is the lower cost of development and maintenance as only those components that need to be adapted to local laws or regulations need to be split out into separately managed requirements.
5. Multi-team Diverse Content
The final use case – Multi-team Diverse Content – is based around the need to supply diverse content managed by different teams through a single interface to the user.
There are many scenarios that fall into this category of use cases. For example, a government providing access to Citizen Information through a conversational interface.
The content can come from any branch of Government, is generally managed by different teams but is served through a common access point across multiple channels (web, phone, social etc.). Each team is responsible for building their own content-based bot but the multi-bot orchestration ensures that the user can access it all from their chosen access point.
Another good example here is internal HR bots dealing with employee queries. This includes HR helpdesk and IT helpdesk use cases where the content is developed and curated by different teams -IT, HR, payroll, training, procurement etc, – but accessed via a common interface such as Slack or MS Teams or internal employee assistant.
Solving Today’s Chatbot Problems via Multi-Bot Architectures
Multi-bot orchestration is a key capability in building complex or large conversational AI solutions that can solve many of the issues chatbot developers face – at design time, during development and testing, and especially during ongoing support and enhancements.
The key is to develop an architecture that supports the business requirements from the start. The requirements and architecture should dictate the technology and not the other way around. The benefits of multi-bot architectures are clear to see and the ROI is measured in weeks and months rather than years.
We have more resources related to this topic so please take a look:
Blog article: How to Lower the Cost of Chatbot Development and Management
eBook: Lower the TCO of Enterprise Conversational AI Solutions
Blog Article: Accelerate the Development and Enhancement of Conversational AI solutions powered by Amazon Lex