I recently completed a large document management system on SharePoint 2010 that used FAST Search and claims-based authentication. The client wanted to secure and limit access to customer-specific documents based on data coming from their CRM system.
We decided to implement a custom claim provider that would query the CRM system at login for customer claims based on the user ID. On upload (based on the customer that was assigned to the document), we used the content organizer to route the document to the correct site, library and folder based on the organization and security rules that we had. Each library had a claim for the customer assigned to it so only users with that claim could view the documents in the library. We would use search for the UI so that the users had a single place to find and view the documents. Sounds simple, right?
It should’ve been.
Unfortunately, the implementation was anything but simple. From the beginning, we hit the core limits of SharePoint 2010, FAST and Claims. Now that we’ve made it to the end, I want to talk about the limits we ran into and steps you can take in your design to avoid them.
The first limit to understand when working with SharePoint and Claims is the ACL size limit. From the Microsoft TechNet page on SharePoint Server 2010 Software Boundaries and Limits, Microsoft states “The size of the scope affects the data that is used for a security check calculation. This calculation occurs every time that the scope changes. There is no hard limit, but the bigger the scope, the longer the calculation takes.”
While SharePoint may support calculating security checks up to 5,000 unique security principles, the reality is that other systems hit their limitations far before this limit. The reason for this is that the Windows operating system has a limit of 64 KB for an ACL, and that affects the maximum size of the security scope that search can crawl. I could spend an entire post talking about what ACLs, security principles and the 64 KB limit, but luckily someone already did. Check out Bugra Postaci’s very informative post on this very topic.
So why is this important when we’re talking about Claims? With classic authentication, the way we normally get around the limits is by moving individual users to active directory groups, and assigning AD groups to lists and libraries to give large numbers of users access to a site collection. However with Claims, we cannot put a claim in an Active Directory group. We have to assign the claim to an object somewhere in SharePoint. I can also put a claim in a SharePoint group, but that still contributes to the size of the ACL calculation.
Even worse, when we add a unique claim anywhere in a site, the limited access permissions to all of the parent objects (lists, webs, etc.) get added all the way up to the root site. These limited access permissions allow the users to navigate down through the webs, lists and folders to get to the item. So each time you add a new claim to a site, this bubbles up to the top-level site collection and permissions list. (Another limitation is the site permissions page, which does not have pagination. So the more users, groups or claims you add, the longer this monster page takes to load.)
If you are following the guidance of Microsoft, you might think you can add up to 5,000 unique claims per site. However, following Bugra Postaci’s post, you are really limited to 2,000…or less! I’ve even come across instances where people have issues with fewer claims than that. In the end, the actual number of characters in the claim can affect this. So my guidance for all of you planning to use Claims is this: Keep the number of characters of the claims as small as possible. Don’t use long text strings and use numeric IDs from the database if you can.
[pullquote]Keep the number of characters of the claims as small as possible.[/pullquote]If you go above the 64 KB size of the ACL and search crawls the content, the crawl will take forever to run and eventually stop or timeout without having crawled any content. In the crawl log, you will see one of the following errors: “The server is unavailable and could not be accessed” or “The filter daemon did not respond within the timeout limit.”
You might turn to the Internet and come across this very informative post by Martin Hatch about using AddToCurrentScopeOnly method and think that it’s a way to get around this. However, that attempt will be in vain, as search requires the limited scope permissions up to the root site in order to crawl the content. So if you do this, the documents will not appear in search results.
In order to avoid this issue, group your documents and the claims in sites so that the claims are split out so there are 2,000 or less claims per site. This isn’t always easy to do, especially if you need to apply multiple claims to a single document.
The second limitation has to do with FAST and the maximum number of claims a user can have to actually execute a search. In our project, we had some users that had access to up to 10,000 customers and as a result, they had 10,000 claims. This number was fine with a custom claim provider (although I would be careful going over 5,000 or 6,000 in a situation when you are using claims and federation, like with ADFS). But when one of those users went to submit a search against FAST, we would get an error saying “The search service is unavailable” on the search page. As you can imagine, it took us FOREVER to figure out why users were getting this error. Some people didn’t get this error and others did and finally we put the pieces together and figured out it was the number of claims a user had that was causing this error.
So what do you think the maximum number of claims that a user can have when submitting a search to FAST? 2,000? 1,000? In your dreams…Try 175. Yes, 175 claims was the maximum number with our claims format (which was pretty long, so maybe you’ll get up to 200).
We actually submitted this to Microsoft as a support ticket and asked for a resolution. Their response was that we must modify the TCPBinding settings in the fastsearch\bin\microsoft.sharepoint.search.extended.security.workerservice.exe.config by a factor of 10. But they also noted that a service pack could update this file, so there was not guarantee that this was a permanent solution.
<netTcpBinding> <binding name="InterProcessBinding" maxConnections="100" transactionFlow="true" maxBufferSize="655360" maxBufferPoolSize="5242880" maxReceivedMessageSize="655360" closeTimeout="00:02:00" openTimeout="00:02:00" receiveTimeout="00:02:00" sendTimeout="00:02:00"> <reliableSession inactivityTimeout="00:30:00" enabled="true" /> <security mode="Message"> <message clientCredentialType="Windows" /> </security> <readerQuotas maxDepth="32" maxStringContentLength="655360" maxArrayLength="16384" maxBytesPerRead="4096" maxNameTableCharCount="16384" /> </binding> </netTcpBinding>
In the end, this would have allowed us to have users with a maximum of 5,000 claims submit a search to FAST. But if your SharePoint administrator is not willing to make this change, you are going to be completely out of luck. My advice is to set up a standard SharePoint Search service and use that, as it doesn’t have the same limitation.
In our situation, our client was not willing to use standard SharePoint Search or make the configuration change to FAST, so we ended up having to implement our own security trimming in a custom web service. Since our library and folder structure followed a consistent pattern, we were able to trim the search results based on the library and folder names that we knew the current user had access to.
