For the last year and a half there has been a consistent ask in my conversations with customers entrenched in Microsoft 365: How can we scope a Gen AI conversational experience to specific SharePoint content? Up until recently, there were three options:

  1. Use Microsoft 365 Copilot to chat with SharePoint content, but there is no definitive way to narrow in on a SharePoint content source. In the last couple of months, you can now say things like “Use content from x site,” but there is no way to get a true “search vertical” like experience.
  2. Copilot Studio allows you to specify specific SharePoint sites, but in my experience the quality out-of-the-box was lackluster, and using SharePoint content to work consistently was challenging.
  3. The final option is to utilize Azure OpenAI with a SharePoint source. This option is much more reliable, and the responses are vastly improved, but the complexity to spin this up and manage is significantly higher.

None of those three options are particularly viable when you want an easy, consistent, better-than-average prompt/response experience with SharePoint content. That has now changed with the addition of Agents in SharePoint.

Agents in SharePoint (you may also see “Copilot in SharePoint”) experiences are now in general availability for all tenants, and this enables two things: 1) a default chatbot experience is created automatically for every site to chat with all of the site’s content, and 2) an owner/member can now create custom chat experiences based on a subset of that SharePoint site’s content.

The general UI and experience of interacting with a SharePoint Agent

Right now, this experience is only available for Microsoft 365 Copilot licensed users, but if your tenant is set up for Copilot Studio consumptive billing, there will soon be a pay-as-you-go meter to enable the experience. There is information out there on how to set up and use these agents, so I want to focus on what I think is great and not so great about the current implementation.

What Works

The first thing that is really impressive about Agents in SharePoint is how easy they are to use and set up. Right now, it’s a completely no-code experience to set up a new chat experience.

One way to create a SharePoint Agent

After you click that option, you can specify several aspects of the experience including content sources, default prompts, general chatbot behavior, and other configuration. Once it’s published, it’s immediately available to use from the Microsoft 365 suite bar. Agents can also be created from the “New” button in a library as well as the chatting pane.

The best part of this capability is the ability to narrow in the content source. You can choose up to 20 documents and/or folders within a SharePoint site to narrow the experience (if you use the root connected full site experience in a Hub site, it will also query across the hub but with limited effectiveness in my early testing). In my experience, the results are between a Copilot Studio and Azure OpenAI implementation—it’s much better than the former but not quite as good or as flexible as the latter. While any Gen AI experience can be a bit volatile and inconsistent, I generally found it to be pretty reliable. You can not only ask questions across your scoped content, but you can also do things like select particular files in a library and create FAQs, do translations, comparisons, and more. It is surprisingly robust for a first release. Also and importantly, all permissions are respected for what a user can interact with and query from the chat experience. The agent can then be added to a Teams chat by copying and pasting a link, and any chat members can ask questions of the agent (while respecting permissions to source content).

The other bit plus to this model is how incredibly easy it is for a site owner to manage. Right now, these agents create a “.agent” file, which is just a JSON configuration file, in the library it was created. You can treat it as any other file: add metadata, label it, etc. To make it available to all users, a site owner can approve it in the chat dialogue experience, or they can move the file to a designated approved folder in the site assets library. There is no way to currently edit or disable the site-wide agent that is created by default.

What Doesn’t Work

There are a few things that I think do need to improve, and I’m certain they will as the capability progresses. The first and most important missing component is a real model to govern this capability. Simply saying it’s “just a file” and managing it as such is not sufficient. Tenant admins need the ability to disable/enable completely. You can disable it at a site level if you have access to Restricted Content Discovery in SharePoint Advanced Management. Also, there is no report that tells admins where these experiences have been created, and analytics about their usage don’t exist.

Being that the configuration is simply a file, while it does follow the well-known document management experience, it also makes deleting these and even editing them a bit more volatile than I would prefer. Yes, it can be recovered through the recycle bin, but this is a configuration file for SharePoint experiences not simply a document in reality.

While I haven’t tested, I assume Purview treats prompts similar to Microsoft 365 Copilot and can be watched for example in a communication compliance policy (prompts are still stored in a user’s Exchange folder it appears like Microsoft 365 Copilot). For now, that feels sufficient, but it would be nice to have granular Purview controls for prompts in Microsoft 365 Copilot compared to these agents.

The other thing I find very jarring is how Microsoft continues to roll out these capabilities. For example, when it was pushed to our tenant license Copilot users simply got a modal saying the agents are available with little control for an admin to manage this experience. Since it’s tied to the M365 Copilot license allocation, there is no way to differentiate enabling this experience. It’s very similar to how Copilot for Sales simply shows up for certain Dynamics Sales licensed users leading to confusion.

What Is Coming for Agents In SharePoint

There are a lot of things that are being promised but currently are not available. Several of these will really make or break how much an organization heavily invests in this capability going forward.

  • Currently, you cannot edit the agent in Copilot Studio despite a button in the create agent dialogue suggesting you can. If you click it there is a message saying it’s coming soon (I am still absolutely perplexed why Microsoft continues to add buttons designating capabilities, but they are non-functional at the moment).
  • You cannot currently edit the ready-made agents, but that is promised to come.
  • You cannot include OneDrive content or content from other sites. The former is promised as forthcoming.
  • A “.agent” file cannot have a label applied to it currently, and if you want to manage .agent files with DLP you have to use the extension as a condition.
  • You cannot set a certain agent as the default one for a site, but that is being promised by Microsoft.
  • Lists are not currently supported: it’s only pages and documents (and certain document types at that), but that is also apparently on the roadmap.
  • Language support is limited to the best results in English right now, but that will improve over time.

I have heard discussions that these agents can be available in the Microsoft 365 Copilot (Business) Chat experience. This would allow for scoped SharePoint content queries inside of Microsoft 365 Copilot and it would finally bring us a unified and flexible content conversational experience in Microsoft 365.

Conclusion and Resources

I honestly may be most excited for this capability to develop more than any other Gen AI capability from Microsoft. Granted, as an ECM person I’m biased, but so many customers are clamoring for this capability in their tenants. Since SharePoint is the lifeblood of most organization’s file management, this capability really offers a lot of promise to Microsoft 365 users.

Microsoft has announced that if your tenant has at least 50 Microsoft 365 Copilot licenses you can access a significant trial for 6 months to allow access to SharePoint agents for non-licensed users. Read more about the promotion and how to enable it in your tenant.