Architecture Best Practices for Conversational AI

The enhanced customer support experience that started with Chatbots is now being slowly taken over by the more proficient and interactive conversational AI with its ability to communicate in natural language and with personalized flair. Traditional rule-based chatbots are still popular for customer support automation but AI-based data models brought a whole lot of new value propositions for them. Conversational AI in the context of automating customer support has enabled human-like natural language interactions between human users and computers.

To get the most out of conversational AI, developers and architects need to focus on including the key components of conversational AI and other elements of consideration to achieve a scalable architecture., starting with UI design, tech stack choices, designing the conversation, backend integrations, and analytics design.  

Here in this blog post, we are going to explain the intricacies and architecture best practices for conversational AI design. 

Define the  Key Components

As you start designing your conversational AI, the following aspects should be decided and detailed in advance to avoid any gaps and surprises later.

User Interface 

The UI/UX should be clearly defined for all possible flows and interactions. Like for any other product, it is important to have a view of the end product in the form of wireframes and mockups to showcase different possible scenarios, if applicable. For e.g. if your chatbot provides media responses in the form of images, document links, video links, etc., or redirects you to a different knowledge repository.

The Technology  

 The technology choice is also critical and all options should be weighed against before making a choice. From open source to paid conversational AI platforms like Rasa, DialogFlow which provide a structured conversation flow, to the use of language models like GPT2, GPT3, BERT, etc. which are more flexible to provide human-like conversation but come with some possibility of unexpected responses depending on how the conversation proceeds and the accuracy the model has been able to achieve with proper fine-tuning. Each solution has a way of defining and handling the conversation flow, which should be considered to decide on the same as applicable to the domain in question. Also proper fine-tuning of the language models with relevant data sets will ensure better accuracy and expected performance.

Conversation design

The most important aspect of the design is the conversation flow, which covers the different aspects which will be catered to by the conversation AI.  You should start small by identifying the limited defined scope for the conversation as part of your design and develop incrementally following an Iterative process of defining, Design, Train, Integrating, and Test. 

Have the big picture in mind in the architecture consideration to ensure an easily scalable solution. Be open to improvements and enhancements as this is an always growing and improving industry so  Always Review, Revalidate and Refactor!

3 Key Components of conversation design

3 key components of conversation design include :

• domain-specific intent matching, 
• entity extraction and 
• shaping the dialog flow that can be either linear or nonlinear.

 For better understanding, we have chosen the insurance domain to explain these 3 components of conversation design with relevant examples. 

Intent Matching

Intents or the user intentions behind a conversation are what drive the dialogue between the computer interface and the human. These intents need to match domain-specific user needs and expectations for a satisfactory conversational experience. The same AI may be handling different types of queries so the correct intent matching and segregation will result in the proper handling of the customer journey.

Below are some domain-specific intent-matching examples from the insurance sector.  

Insurance Eligibility — 

• Do I need to get admitted to the hospital to be eligible for raising a claim?
• Are daycare procedures covered?
• Does it cover the pre-hospitalization and post-hospitalization periods as well?

Insurance Process —

• How should I initiate an insurance claim?
• When should I raise the claim?
• Which documents do I need to keep ready?

Insurance claim follow-up —

• How should I initiate an insurance claim?
• Could you please let me know when my claim request will get addressed? 
• How far has my claim been processed?

It covers the different scenarios to which the AI will be trained to respond to. 

Entity Extraction

Entity extraction is about identifying people, places, objects, dates, times, and numerical values from user communication. For conversational AI to understand the entities users mention in their queries and to provide information accordingly, entity extraction is crucial. 

Here below we provide a domain-specific entity extraction example for the insurance sector.  

Customer Query—

I’m a 57-year-old male. Am I eligible for 10 years of insurance coverage? 

Extracted Entities—

Unit-time (57 years, 10 years) 

Gender (male) 

Insurance-metric (coverage)

The AI will be able to extract the entities and use them to cover the responses required to proceed with the flow of conversations.

Shaping Dialogue Flow 

 Lastly for conversation Design, you need to control the dialogue flow so that specific user queries and needs can be addressed. Now dialogue flow can be both linear and nonlinear. 

Linear Dialog

In linear dialogue, the flow of the conversation follows the pre-configured decision tree along with the need for certain elements based on which the flow of conversation is determined. Here the ‘if-this-then-that’ kind of rules work for addressing user queries. If certain required entities are missing in the intent, the bot will try to get those by putting back the appropriate questions to the user. 

Here’s an example. 

Customer: I will be raising a claim next week for a heart treatment that my doctor is going to start tomorrow.

Disease    – Yes

Time          – Yes

Hospitalization  – Not
State know

Since the hospitalization state is required info needed to proceed with the flow, which is not known through the current state of conversation, the bot will put forth the question to get that information.

Chatbot: Are you going to get admitted?

And based on the response, proceed with the defined linear flow of conversation.

Nonlinear Dialogue—

In nonlinear conversation, the flow based upon the trained data models adapts to different customer intents. For conversational AI the dialogue can start following a very linear path and it can get complicated quickly when the trained data models take the baton. 

Non-linear conversations provide a complete human touch of conversation and sound very natural. The conversational AI solutions can resolve customer queries without the need for any human intervention. The flow of conversation moves back and forth and does not follow a proper sequence and could cover multiple intents in the same conversation and is scalable to handle what may come. 

Backend integrations 

Also understanding the need for any third-party integrations to support the conversation should be detailed. If you are building an enterprise Chatbot you should be able to get the status of an open ticket from your ticketing solution or give your latest salary slip from your HRMS.

The consideration of the required applications and the availability of APIs for the integrations should be factored in and incorporated into the overall architecture.

Analytics design

As you design your conversational AI, you should consider a mechanism in place to measure its performance and also collect feedback on the same. As part of the complete customer engagement stack, analytics is a very essential component that should be considered as part of the Conversational AI solution design. Having a complete list of data including the bot technical metrics, the model performance, product analytics metrics, and user feedback. Also, consider the need to track the aggregated KPIs of the bot engagement and performance. There should be a defined process to periodically review the conversation transcripts to understand the more popular topics and intents, and analyze the overall conversational experience to measure its shortcomings, failed/mishandled/unhandled intents and apply the improvements.

Other Important Considerations for Conversational AI Architecture Best Practices

Now that we have a comprehensive understanding of conversation design and what are the key aspects involved in the development process, let us now give some attention to the other notable things that we should consider in the architecture design. 

Security First – Always!

Data security is an uncompromising aspect and we should adhere to best security practices for developing and deploying conversational AI across the web and mobile applications.  Having proper authentication, avoiding any data stored locally, and encryption of data in transit and at rest are some of the basic practices to be incorporated. Also if the bot is being used org-wide and there is a need for Role-based access control to the data, extra caution should be taken to handle the conversation smoothly by gently refusing access to data to which the user is not entitle.

Separation of Bot Logic from the Channels

The logic underlying the conversational AI should be separated from the implementation channels to ensure flexible modularity, and channel-specific concern handling, and for preventing unsolicited interceptions with the bot logic.   This could be specific to your business need if the bot is being used across multiple channels and should be handled accordingly.

Build with a multi-channel-context approach

The conversational AI architecture should also be developed with a focus to deploy the same across multiple channels such as web, mobile OS, and desktop platforms. This will ensure optimum user experience and scalability of the solutions across platforms. So if the user was chatting on the web and she is now in transit, she can pick up the same conversation using her mobile app.

Allow for Integration with existing applications

When developing conversational AI you also need to ensure easier integration with your existing applications. You need to build it as an integration-ready solution that just fits into your existing application. 

Over To You

Conversational AI is known for its ability to answer deep-probing and complex customer queries. But to make the most of conversational AI opportunities, it is important to embrace well-articulated architecture design following best practices. How you knit together the vital components of conversation design for a seamless and natural communication experience, remains the key to success. 

With a well-thought-out process to start small and extend with deliberation, slowly encompassing all your processes, Conversational AI  is a must-have for any industry or setup, to provide a good experience and improve engagement with your customers externally as well as your employees internally.