Introduction

In the realm of modern technology, the role of intelligent chatbots has become increasingly pivotal. These bots offer more than just automated responses, they provide businesses with a means to deliver efficient customer support, automate tasks, and optimize operations. Microsoft has recognized this need and offers a powerful synergy of tools and services to help developers and IT professionals harness the capabilities of intelligent chatbots. In this technical blog post, we’ll embark on a journey to unlock the potential of Azure OpenAI, Power Virtual Agent (PVA), and Power Automate, combining their strengths to create a highly intelligent chatbot.

By the end of this deep dive, you’ll possess the technical knowledge to develop your intelligent chatbot, revolutionizing the way your organization engages with customers and automates internal processes. Let’s roll up our sleeves, dive into the technical intricacies, and explore the universe of intelligent chatbots, empowered by Azure OpenAI and the robust Microsoft Power Platform!

What are you going to create?

The below diagram provides a comprehensive overview of the Chatbot (Copilot) implementation using Azure OpenAI. In a typical use-case scenario. Users can ask a query to the Copilot which is deployed in a Canvas App, Microsoft Teams, or a web application. This Copilot uses Power Virtual Agent (PVA) to create a conversational experience. PVA is a Software-as-a-Service (SaaS) offering from Microsoft that allows users to create AI-powered chatbots for a range of requests – from providing simple answers to common questions to resolve issues requiring complex conversations.

When you create a Chatbot with PVA, you author and edit topics. Topics are discrete conversation paths that, when used together with a single bot, allow users to have a conversation experience. There are two kinds of Topics in PVA, Custom and System. In this use case, we have used System Topics like Conversation Start and Fallback to get you started. Fallback Topic is used to integrate with Azure OpenAI.

When the query asked by the user arrives at PVA. Fallback Topic is triggered. It then sends the request to a Power Automate Flow which orchestrates multiple tasks. These tasks involve extracting context from various data sources, system prompts, and initiating an HTTP request to the Azure OpenAI endpoint. The response received from Azure OpenAI is then routed back to the user through Power Virtual Agent Actions.

Additionally, we will also explore a way to maintain the conversation history. Maintaining history is important in LLMs so that we can create a conversation loop to maintain contexts while a user asks subsequent questions. It’s important to note that pruning history is beyond the scope of this blog post.

Now, let’s dive into the implementation and unlock the potential of Power Virtual Agent and Power Automate.

Prerequisite

You have already created an Azure OpenAI resource in your subscription and deployed gpt-35-turbo model in Azure. Once the model is created, you can get the Endpoint and Key from Playground in Azure.

Also, here a Solution is created in Power Apps where various configurations are defined as Environment Variables like the Azure OpenAI Endpoint and Key.

1. Create a Bot in Power Virtual Agent

In the solution (shown in Figure 3), we can add a new Power Virtual Agent Bot by clicking on New > Chatbot. After creating the Chatbot. You will see the below window.

2. Power Virtual Agent Topics

We will use two System Topics of PVA here in this example.

• Conversation Start: This topic is triggered at the beginning of the conversation.
• Fallback: This topic is triggered when none of the phrases in System and Custom match. In this example, we are not aware of what query will be asked by the user. Therefore, we will use this topic to integrate Azure OpenAI. Queries from users will be sent to the Fallback Topic which will route them to Azure OpenAI.

2.1 Conversation Start Topic

Chatbot can be tested in Power Virtual Agent canvas as shown in Figure 4. A default message “Hi, how can I help you?” is displayed by the Chatbot. This message can be configured in the Conversation Start Topic as shown below.

2.2 Fallback Topic

We will use this system topic of PVA to integrate Azure OpenAI. The below image displays how you can create a trigger for unknown intent. The Unknown Intent has a condition that checks if the global variable used to save the chat history is initialized or not. If it is already initialized, it preserves its value and appends the user query to it.

Explanation Figure 6.2: PowerFx script is used to concatenate the user query given by System.Activity.Text with chat history. The user query is formatted into OpenAI format.

In this article, I have used a global variable to save the conversation history. Ideally, in a production environment, we will not be storing the history in it. We will have to store this information in a data store like Dataverse. You can also view, export, and download transcripts/conversations with your chatbot in Power Virtual Agent. You can refer to this link to learn more about how you can work with conversation transcripts.

2.3 Add an Action to send the user query to Power Virtual Agent Flow and receive the response

In this step, we will be adding an action to the Fallback topic which will send the user’s query along with the chat history to Power Virtual Agent Flow. This flow will get the context from data sources and initiate an HTTP call to Azure OpenAI to get the answer. Will elaborate on the Power Virtual Agent Flow shortly. The below image shows the action we need to add to the fallback topic. The response from Azure OpenAI is saved in the “openAiResponse” variable.

2.4 Displaying the response from Azure OpenAI

After the Power Automate Flow is executed successfully, we will get the response back as an output that is stored in a variable. Next, we will create a Message Node which displays the answer to the user. Here we add this answer to the conversation history variable. We can also use the Substitute method to replace any invalid characters in the sequence.

3. Power Automate Flow which pulls data from the knowledge repository and sends it to Azure OpenAI

The below diagram shows the Power Virtual Agent Flow which connects to Dataverse to fetch the context and formulates a request body which is sent to Azure OpenAI. User queries can be first embedded, and a vector search can be executed against the knowledge repositories. But this technique is out of the scope of this blog post.

4. Deployment

With Power Virtual Agents, you can publish bots to multiple platforms or channels. These include live websites, mobile apps, and messaging platforms like Microsoft Teams.

Summary

In this article, we discussed how we can create a Chatbot using Power Virtual Agent and Power Automate. We discussed the sequence of steps to connect to Dataverse, get the knowledge context, and then connect to Azure OpenAI to get the response. This article serves as a launchpad for seamlessly integrating Azure OpenAI into Power Virtual Agent. Moreover, this approach can be extended to integrate Embeddings and Vector Search, allowing you to fully incorporate the Retrieval Augmented Generation (RAG) Pattern. Embeddings and vector search are techniques that use numerical representations of data to measure similarity and relevance. Embeddings are vectors that capture the semantic meaning of text, images, audio, video, or other types of data. Vector search is a way of finding the most similar vectors to a given query vector, using distance metrics such as cosine similarity.