{"id":518,"date":"2025-09-20T08:17:27","date_gmt":"2025-09-20T08:17:27","guid":{"rendered":"https:\/\/codingwithramin.com\/?p=518"},"modified":"2026-09-01T08:14:08","modified_gmt":"2026-09-01T08:14:08","slug":"beyond-manual-indexing-building-intelligent-sharepoint-agents-part-1-the-foundation","status":"publish","type":"post","link":"https:\/\/codingwithramin.com\/?p=518","title":{"rendered":"Building Intelligent SharePoint Agents &#8211; Part 1: The Foundation"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">I spent weeks building a custom indexing pipeline for SharePoint content: incremental updates, change tracking, governance controls, the works. Then I found out Microsoft had already shipped a tool that does most of it automatically.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you&#8217;re maintaining a RAG pipeline against SharePoint right now, you&#8217;ll recognize the problem before I finish describing it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Companies store thousands of policy documents, procedures, and knowledge base articles across multiple SharePoint sites. Building an AI agent against that content used to mean extracting it manually, chunking it into a vector store like Azure AI Search, writing refresh logic to catch changes, and mapping SharePoint&#8217;s permission model on top of all that. I&#8217;ve built this pattern more than once. It works. It&#8217;s also slow to build and easy to get wrong.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">In September 2025, Microsoft shipped the SharePoint tool for Azure AI Foundry Agent Service, now called Microsoft Foundry Agent Service. It&#8217;s still labeled preview as of August 2026, not generally available, but it already removes most of the manual work described above.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This post covers what the tool replaces, how it works, and what it costs against rolling your own Azure AI Search pipeline. Part 2 covers the implementation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>The problem with manual SharePoint indexing<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Before this tool existed, building an AI agent that could search SharePoint meant working through eight separate problems:<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Content extraction: pull documents with SharePoint REST APIs or Microsoft Graph<\/li>\n\n\n\n<li>Document processing: parse Word, Excel, PDF, and PowerPoint files into text<\/li>\n\n\n\n<li>Chunking: split content into segments small enough for a vector store<\/li>\n\n\n\n<li>Vector indexing: upload the chunks to Azure AI Search with metadata<\/li>\n\n\n\n<li>Permission mapping: replicate SharePoint&#8217;s security boundaries in the index<\/li>\n\n\n\n<li>Change detection: poll SharePoint for updates, deletions, and new content<\/li>\n\n\n\n<li>Incremental updates: refresh only what changed, not the whole index<\/li>\n\n\n\n<li>Search implementation: query the index in a way that still respects who&#8217;s allowed to see what<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">I&#8217;ve implemented this pattern several times. Every client had a different SharePoint configuration, a different permission structure, a different mix of file types. None of it transferred cleanly to the next project.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Five things made it worse:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Permissions: SharePoint&#8217;s access model works at the site, library, folder, and document level. Replicating that correctly in a separate vector store, and keeping it in sync, was a constant source of bugs.<\/li>\n\n\n\n<li>Freshness: documents change constantly in a live SharePoint tenant. Detecting those changes without hammering the API required careful throttling and state tracking.<\/li>\n\n\n\n<li>Scale: bigger content libraries meant slower indexing. Reprocessing a large library could take hours.<\/li>\n\n\n\n<li>File formats: Word, Excel, PDF, PowerPoint. Each needs its own parser, and each parser breaks differently on complex formatting.<\/li>\n\n\n\n<li>Search quality: even with good chunking, getting relevant results out of the index took real tuning of embeddings and retrieval logic.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What the SharePoint tool does instead<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The SharePoint tool connects a Foundry agent directly to a SharePoint site or folder. Point it at a site, and it uses the Microsoft 365 Copilot Retrieval API to handle discovery, indexing, chunking, and permission checks. Under the hood, it&#8217;s the same retrieval stack that powers Microsoft 365 Copilot itself.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That matters for two reasons. You&#8217;re not building or maintaining an index. And responses are grounded in whatever is current in SharePoint right now, not in whatever your pipeline last managed to sync.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What it handles automatically:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Discovers and indexes supported document types in the sites or folders you configure<\/li>\n\n\n\n<li>Re-indexes content as it changes, without a polling job you have to write<\/li>\n\n\n\n<li>Applies identity passthrough (On-Behalf-Of authentication), so a user only sees answers grounded in documents they already have permission to read<\/li>\n\n\n\n<li>Uses the same indexing and ranking Microsoft 365 Copilot uses, so search quality doesn&#8217;t depend on how well you tuned your own embeddings<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Instead of the eight-step pipeline above, the agent talks to SharePoint through the built-in tool, which handles document discovery, chunking, permission verification, retrieval, and citation on its own.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>How it actually works<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When an agent using the SharePoint tool gets a query, this is the sequence:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"189\" src=\"https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-1024x189.png\" alt=\"\" class=\"wp-image-522\" srcset=\"https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-1024x189.png 1024w, https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-300x55.png 300w, https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-768x142.png 768w, https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-1536x284.png 1536w, https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-2048x378.png 2048w, https:\/\/codingwithramin.com\/wp-content\/uploads\/2025\/07\/image-10-769x142.png 769w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ol class=\"wp-block-list\">\n<li>The agent sends the user&#8217;s query to the SharePoint tool<\/li>\n\n\n\n<li>The Microsoft 365 Copilot Retrieval API checks the user&#8217;s license (or pay-as-you-go entitlement) and their document permissions, using managed identity<\/li>\n\n\n\n<li>It runs a semantic search over the indexed SharePoint content<\/li>\n\n\n\n<li>It returns document chunks the user is allowed to see<\/li>\n\n\n\n<li>The agent generates a response grounded in those chunks, with citations<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Identity passthrough is what makes this safe to point at real enterprise content. The agent authenticates as the end user, not as a service principal, so SharePoint&#8217;s own permission checks apply on every single request.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Limits worth knowing before you build on this<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A few limits shape what you can realistically do with the tool today.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>File size: Word, PowerPoint, and PDF files up to 512MB are supported. Other file types top out at 150MB.<\/li>\n\n\n\n<li>Formats for full semantic search: .doc, .docx, .pptx, .pdf, .aspx, and .one. Anything else falls back to keyword-only retrieval instead of semantic search. Excel workbooks aren&#8217;t on that list at all, which surprised me the first time I hit it.<\/li>\n\n\n\n<li>Indexing delay: new or changed content can take anywhere from minutes to hours to show up in results, depending on Microsoft Search&#8217;s sync cycle.<\/li>\n\n\n\n<li>Throttling: the Retrieval API caps requests at 200 per user per hour, query strings at 1,500 characters, and results at 25 per query.<\/li>\n\n\n\n<li>Tables and media: retrieval from text in tables only works for .doc, .docx, and .pptx files. Images and charts aren&#8217;t retrievable; only text content gets indexed.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Test against your actual document set early. A general-purpose search index this isn&#8217;t.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How Microsoft&#8217;s Built-in Semantic Indexing Works<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The SharePoint grounding tool leverages built-in indexing capabilities to enhance search and retrieval experience, including intelligent indexing, query processing, and content chunking. Here&#8217;s how Microsoft&#8217;s semantic indexing works:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Automatic Content Discovery<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Crawling<\/strong>: Continuously monitors SharePoint sites for new, modified, or deleted content<\/li>\n\n\n\n<li><strong>Format Recognition<\/strong>: Automatically identifies and processes different file types using specialized parsers<\/li>\n\n\n\n<li><strong>Metadata Extraction<\/strong>: Captures document properties, author information, modification dates, and SharePoint metadata<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Semantic Understanding<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Text Extraction<\/strong>: Extracts meaningful text content from various document formats<\/li>\n\n\n\n<li><strong>Chunking Strategy<\/strong>: Intelligently breaks documents into semantically meaningful segments<\/li>\n\n\n\n<li><strong>Vector Embeddings<\/strong>: Creates high-quality vector representations using enterprise-optimized models<\/li>\n\n\n\n<li><strong>Relationship Mapping<\/strong>: Understands document relationships, references, and organizational structure<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Query Processing<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Intent Recognition<\/strong>: Understands user query intent and context<\/li>\n\n\n\n<li><strong>Semantic Matching<\/strong>: Matches queries to relevant content using semantic similarity rather than just keyword matching<\/li>\n\n\n\n<li><strong>Permission Filtering<\/strong>: Automatically filters results based on user access rights<\/li>\n\n\n\n<li><strong>Relevance Ranking<\/strong>: Applies sophisticated ranking algorithms to surface the most relevant content<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The key advantage is that this entire indexing pipeline is managed by Microsoft, eliminating the need for custom implementation while providing enterprise-grade performance and security.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Cost and Licensing Considerations vs. Azure AI Search<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When deciding between SharePoint grounding and traditional Azure AI Search approaches, consider these cost and licensing factors:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Microsoft 365 Copilot License Requirements<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Per-User Cost<\/strong>: $30\/user\/month for Microsoft 365 Copilot license (as of 2025)<\/li>\n\n\n\n<li><strong>Mandatory for All Users<\/strong>: Every user who will interact with the SharePoint agent needs this license<\/li>\n\n\n\n<li><strong>Enterprise-Only<\/strong>: Currently available only for enterprise customers, not for smaller organizations<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Azure AI Search Alternative Costs<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Service Tiers<\/strong>: Basic tier starts at ~$250\/month, Standard at ~$1,000\/month<\/li>\n\n\n\n<li><strong>Per-Query Pricing<\/strong>: Some tiers offer per-query pricing models<\/li>\n\n\n\n<li><strong>Storage Costs<\/strong>: Additional costs for storing indexed content<\/li>\n\n\n\n<li><strong>Compute Costs<\/strong>: Processing costs for indexing and search operations<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Total Cost of Ownership Comparison<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>SharePoint Grounding Approach<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u2705<strong>Lower Development Costs<\/strong>: Minimal custom development required<\/li>\n\n\n\n<li>\u2705<strong>Zero Infrastructure Management<\/strong>: No indexing pipeline to maintain<\/li>\n\n\n\n<li>\u2705<strong>Built-in Security<\/strong>: Enterprise-grade permissions automatically handled<\/li>\n\n\n\n<li>\u274c<strong>High Per-User Licensing<\/strong>: $30\/user\/month can be expensive for large user bases<\/li>\n\n\n\n<li>\u274c<strong>Limited to Licensed Users<\/strong>: Cannot extend to external users or customers without additional licensing<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Traditional Azure AI Search Approach<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u2705 <strong>Flexible Licensing<\/strong>: Can serve unlimited users with fixed service costs<\/li>\n\n\n\n<li>\u2705 <strong>External User Support<\/strong>: Can serve customers and partners without additional per-user costs<\/li>\n\n\n\n<li>\u2705 <strong>Customization Control<\/strong>: Full control over indexing strategies and search algorithms<\/li>\n\n\n\n<li>\u274c <strong>High Development Costs<\/strong>: Significant custom development and maintenance required<\/li>\n\n\n\n<li>\u274c <strong>Infrastructure Complexity<\/strong>: Must manage indexing pipelines, security, and updates<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Choose <strong>SharePoint Grounding<\/strong> when:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>User base is relatively small (&lt; 100 active users)<\/li>\n\n\n\n<li>Users already have Microsoft 365 Copilot licenses<\/li>\n\n\n\n<li>Development speed is critical<\/li>\n\n\n\n<li>Internal enterprise use only<\/li>\n\n\n\n<li>Security and compliance are paramount<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Choose <strong>Azure AI Search<\/strong> when:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Large user base (&gt; 500 active users)<\/li>\n\n\n\n<li>External users need access (customers, partners)<\/li>\n\n\n\n<li>Custom indexing strategies are required<\/li>\n\n\n\n<li>Budget for development and maintenance is available<\/li>\n\n\n\n<li>Need maximum flexibility and control<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Hybrid Approach Considerations:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>SharePoint grounding for internal enterprise users with Copilot licenses<\/li>\n\n\n\n<li>Azure AI Search for external-facing applications and non-licensed users<\/li>\n\n\n\n<li>Different agents for different user types and use cases<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Part 1 Conclusion: The Foundation is Set<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">We&#8217;ve covered the fundamental shift from manual SharePoint indexing to Azure AI Foundry&#8217;s SharePoint grounding tool. You now understand:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>The Problem<\/strong>: Why traditional SharePoint indexing approaches are costly and complex<\/li>\n\n\n\n<li><strong>The Solution<\/strong>: How Microsoft 365 Copilot API powers intelligent content discovery<\/li>\n\n\n\n<li><strong>The Economics<\/strong>: When to choose SharePoint grounding vs. Azure AI Search<\/li>\n\n\n\n<li><strong>The Architecture<\/strong>: How semantic indexing and identity passthrough work behind the scenes<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">With this foundation in place, you&#8217;re ready to build your own SharePoint agents. <a href=\"https:\/\/codingwithramin.com\/?p=532\">Part 2<\/a> walks through the code: setting up the connection, wiring the tool into an agent, and testing permission-trimmed retrieval end to end.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I spent weeks building a custom indexing pipeline for SharePoint content: incremental updates, change tracking, governance controls, the works. Then I found out Microsoft had already shipped a tool that does most of it automatically. If you&#8217;re maintaining a RAG pipeline against SharePoint right now, you&#8217;ll recognize the problem before I finish describing it. Companies store thousands of policy documents, procedures, and knowledge base articles across multiple SharePoint sites. Building an AI agent against that&hellip;<\/p>\n","protected":false},"author":1,"featured_media":529,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55,5,3],"tags":[49,51,9,59,60,7],"class_list":["post-518","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-microsoft-azure","category-sharepoint-online","tag-ai","tag-ai-foundry","tag-azure","tag-azure-ai-agent","tag-m365-copilot-api","tag-sharepoint"],"_links":{"self":[{"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/posts\/518","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=518"}],"version-history":[{"count":8,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/posts\/518\/revisions"}],"predecessor-version":[{"id":818,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/posts\/518\/revisions\/818"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=\/wp\/v2\/media\/529"}],"wp:attachment":[{"href":"https:\/\/codingwithramin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=518"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=518"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codingwithramin.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=518"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}