Beyond Manual Indexing: Building Intelligent SharePoint Agents – Part 2: Implementation

In Part 1, we explored the fundamental shift from manual SharePoint indexing to Azure AI Foundry’s SharePoint grounding tool. We covered the challenges of traditional approaches, the power of Microsoft 365 Copilot API, and the economic considerations for enterprise adoption.

Now it’s time to get our hands dirty with the actual implementation. In this part, we’ll build a complete, production-ready SharePoint agent system.

Prerequisites and Setup

Before diving into implementation, ensure you have the following ready:

Azure Requirements

Azure Subscription with access to Azure AI Foundry
Microsoft 365 Copilot License ($30/user/month – required by the API)
Azure AI User RBAC Role for all developers and end users

SharePoint Access

READ permissions to target SharePoint sites
Site URLs for the SharePoint sites you want to connect
Admin access to configure connections (if required by your organization)

Development Environment

.NET 8.0 or later installed on your development machine
Visual Studio 2022 or VS Code for development

Step 1: Setup SharePoint connection

If you haven’t set up an Azure AI Foundry Project yet, please refer to steps 1 and 2 in my first blog post: “Migrating from Bing Search APIs to Azure OpenAI Agent with Grounding Bing Search“

Navigate to “Connected Resources” in your AI Foundry project
Click “Add Connection“
Select “SharePoint” as the connection type
Enter your target site URL and give it a Connection name
Select “Add connection“

Troubleshooting Tip: If connection fails, verify that your Azure AI service has the necessary permissions to access SharePoint through your organization’s Microsoft 365 tenant.

Step 2: .NET Project Setup

Now let’s update our .NET application to host our SharePoint agents.

2.1 Configuration Setup

Update appsettings.json and add SharePoint to the agents section:

JSON

{
  "AzureAI": {
    "ConnectionString": "https://yourproject.services.ai.azure.com/api/projects/yourproject"
  },
  "AzureOpenAI": {
    "Endpoint": "https://your-ai-foundry-resource.openai.azure.com",
    "ApiKey": "your-api-key-or-use-managed-identity",
    "DeploymentName": "gpt-4o-deployment"
  },
  "Agents": [
    {
      "Name": "SharePoint Knowledge Agent",
      "Description": "Enterprise knowledge assistant with SharePoint access",
      "Instructions": "You are a knowledgeable assistant with access to enterprise SharePoint content. When users ask questions, search through connected SharePoint sites to provide accurate, current information based on official company documents. Always cite specific documents and respect user permissions.",
      "Deployment": "gpt-4o-deployment",
      "Tools": [
        {
          "ToolType": "SharePointGrounding",
          "ConnectionName": "MainSharePointConnection"
        }
      ]
    },
    {
      "Name": "Web Search Agent",
      "Description": "External information research assistant",
      "Instructions": "You are a research assistant that searches the web for current information. Use Bing search to find relevant, up-to-date information and provide accurate responses with proper citations.",
      "Deployment": "gpt-4o-deployment",
      "Tools": [
        {
          "ToolType": "BingGroundingSearch",
          "ConnectionName": "BingSearchConnection"
        }
      ]
    },
    {
      "Name": "Hybrid Research Agent",
      "Description": "Combined internal and external research specialist",
      "Instructions": "You are a comprehensive research assistant with access to both internal SharePoint content and external web information. Always check internal sources first for company-specific information, then supplement with external research as needed. Clearly distinguish between internal and external sources.",
      "Deployment": "gpt-4o-deployment",
      "Tools": [
        {
          "ToolType": "SharePointGrounding",
          "ConnectionName": "MainSharePointConnection"
        },
        {
          "ToolType": "BingGroundingSearch",
          "ConnectionName": "BingSearchConnection"
        }
      ]
    }
  ]
}

{
  "AzureAI": {
    "ConnectionString": "https://yourproject.services.ai.azure.com/api/projects/yourproject"
  },
  "AzureOpenAI": {
    "Endpoint": "https://your-ai-foundry-resource.openai.azure.com",
    "ApiKey": "your-api-key-or-use-managed-identity",
    "DeploymentName": "gpt-4o-deployment"
  },
  "Agents": [
    {
      "Name": "SharePoint Knowledge Agent",
      "Description": "Enterprise knowledge assistant with SharePoint access",
      "Instructions": "You are a knowledgeable assistant with access to enterprise SharePoint content. When users ask questions, search through connected SharePoint sites to provide accurate, current information based on official company documents. Always cite specific documents and respect user permissions.",
      "Deployment": "gpt-4o-deployment",
      "Tools": [
        {
          "ToolType": "SharePointGrounding",
          "ConnectionName": "MainSharePointConnection"
        }
      ]
    },
    {
      "Name": "Web Search Agent",
      "Description": "External information research assistant",
      "Instructions": "You are a research assistant that searches the web for current information. Use Bing search to find relevant, up-to-date information and provide accurate responses with proper citations.",
      "Deployment": "gpt-4o-deployment",
      "Tools": [
        {
          "ToolType": "BingGroundingSearch",
          "ConnectionName": "BingSearchConnection"
        }
      ]
    },
    {
      "Name": "Hybrid Research Agent",
      "Description": "Combined internal and external research specialist",
      "Instructions": "You are a comprehensive research assistant with access to both internal SharePoint content and external web information. Always check internal sources first for company-specific information, then supplement with external research as needed. Clearly distinguish between internal and external sources.",
      "Deployment": "gpt-4o-deployment",
      "Tools": [
        {
          "ToolType": "SharePointGrounding",
          "ConnectionName": "MainSharePointConnection"
        },
        {
          "ToolType": "BingGroundingSearch",
          "ConnectionName": "BingSearchConnection"
        }
      ]
    }
  ]
}

2.2 Agent Service Implementation

The Agent Service acts as the bridge between your application and Azure AI Foundry. Here’s what the key updates:

private async Task<ToolDefinition?> CreateToolDefinitionAsync(ToolsOptions tool)
{
	try
	{
		return tool.ToolType switch
		{
			"SharePointGrounding" => await CreateSharePointToolAsync(tool),
			"BingGroundingSearch" => await CreateBingSearchToolAsync(tool),
			"CustomBingGroundingSearch" => await CreateCustomBingSearchToolAsync(tool),
			_ => throw new NotSupportedException($"Tool type '{tool.ToolType}' is not supported")
		};
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Failed to create tool definition for {ToolType} with connection {ConnectionName}",
			tool.ToolType, tool.ConnectionName);
		return null;
	}
}

private async Task<ToolDefinition> CreateSharePointToolAsync(ToolsOptions tool)
{
	var connection = await GetConnectionByNameAsync(tool.ConnectionName);
	if (connection?.Id == null)
	{
		throw new InvalidOperationException($"SharePoint connection '{tool.ConnectionName}' not found or invalid");
	}

	_logger.LogInformation("Creating SharePoint grounding tool with connection: {ConnectionName}",
		tool.ConnectionName);
	
	return new SharepointToolDefinition(
		new SharepointGroundingToolParameters(connection.Id)
	);
}

private async Task<ToolDefinition> CreateBingSearchToolAsync(ToolsOptions tool)
{
	var connection = await GetConnectionByNameAsync(tool.ConnectionName);
	if (connection?.Id == null)
	{
		throw new InvalidOperationException($"Bing Search connection '{tool.ConnectionName}' not found or invalid");
	}

	return new BingGroundingToolDefinition(
		new BingGroundingSearchToolParameters(
			[new BingGroundingSearchConfiguration(connection.Id)]
		)
	);
}

private async Task<ToolDefinition> CreateCustomBingSearchToolAsync(ToolsOptions tool)
{
	var connection = await GetConnectionByNameAsync(tool.ConnectionName);
	if (connection?.Id == null)
	{
		throw new InvalidOperationException($"Custom Bing Search connection '{tool.ConnectionName}' not found or invalid");
	}

	var configuration = new BingCustomSearchConfiguration(
		connection.Id,
		tool.ConfigurationName ?? "default")
	{
		Count = 5,
		SetLang = "en",
		Market = "en-us"
	};

	return new BingCustomSearchToolDefinition(
		new BingCustomSearchToolParameters([configuration])
	);
}

private async Task<ToolDefinition?> CreateToolDefinitionAsync(ToolsOptions tool)
{
	try
	{
		return tool.ToolType switch
		{
			"SharePointGrounding" => await CreateSharePointToolAsync(tool),
			"BingGroundingSearch" => await CreateBingSearchToolAsync(tool),
			"CustomBingGroundingSearch" => await CreateCustomBingSearchToolAsync(tool),
			_ => throw new NotSupportedException($"Tool type '{tool.ToolType}' is not supported")
		};
	}
	catch (Exception ex)
	{
		_logger.LogError(ex, "Failed to create tool definition for {ToolType} with connection {ConnectionName}",
			tool.ToolType, tool.ConnectionName);
		return null;
	}
}

private async Task<ToolDefinition> CreateSharePointToolAsync(ToolsOptions tool)
{
	var connection = await GetConnectionByNameAsync(tool.ConnectionName);
	if (connection?.Id == null)
	{
		throw new InvalidOperationException($"SharePoint connection '{tool.ConnectionName}' not found or invalid");
	}

	_logger.LogInformation("Creating SharePoint grounding tool with connection: {ConnectionName}",
		tool.ConnectionName);
	
	return new SharepointToolDefinition(
		new SharepointGroundingToolParameters(connection.Id)
	);
}

private async Task<ToolDefinition> CreateBingSearchToolAsync(ToolsOptions tool)
{
	var connection = await GetConnectionByNameAsync(tool.ConnectionName);
	if (connection?.Id == null)
	{
		throw new InvalidOperationException($"Bing Search connection '{tool.ConnectionName}' not found or invalid");
	}

	return new BingGroundingToolDefinition(
		new BingGroundingSearchToolParameters(
			[new BingGroundingSearchConfiguration(connection.Id)]
		)
	);
}

private async Task<ToolDefinition> CreateCustomBingSearchToolAsync(ToolsOptions tool)
{
	var connection = await GetConnectionByNameAsync(tool.ConnectionName);
	if (connection?.Id == null)
	{
		throw new InvalidOperationException($"Custom Bing Search connection '{tool.ConnectionName}' not found or invalid");
	}

	var configuration = new BingCustomSearchConfiguration(
		connection.Id,
		tool.ConfigurationName ?? "default")
	{
		Count = 5,
		SetLang = "en",
		Market = "en-us"
	};

	return new BingCustomSearchToolDefinition(
		new BingCustomSearchToolParameters([configuration])
	);
}

CreateToolDefinitionAsync Method: This is the factory method that creates the appropriate tool based on configuration. It uses pattern matching to determine whether to create SharePoint, Bing search, or custom search tools.

CreateSharePointToolAsync Method: Specifically handles SharePoint grounding tool creation. It retrieves the connection information and creates a SharepointToolDefinition that the AI service can use to access your SharePoint content.

Connection Validation: The service includes robust error handling to ensure all connections are valid before creating agents. This prevents runtime errors and provides clear feedback during setup.

Why this architecture matters: By abstracting tool creation into a service layer, you can easily add new tool types, modify connection logic, or implement custom validation rules without changing the core agent logic.

2.3 Agent Wrapper Class

The Agent Wrapper provides additional functionality around the core Azure AI agent. It handles agent initialization, clean-up, and resource management to ensure optimal performance and prevent memory leaks. I’ve added another method to this class:

public async Task<bool> ValidateConnectionsAsync()
        {
            try
            {
                if (_agentOptions.Tools == null || _agentOptions.Tools.Count == 0)
                {
                    _logger.LogInformation("Agent '{AgentName}' has no tools to validate", _agentOptions.Name);
                    return true;
                }

                var allValid = true;
                foreach (var tool in _agentOptions.Tools)
                {
                    var isValid = await _agentService.ValidateConnectionAsync(tool.ConnectionName);
                    if (!isValid)
                    {
                        _logger.LogWarning("Invalid connection '{ConnectionName}' for tool '{ToolType}' in agent '{AgentName}'", 
                            tool.ConnectionName, tool.ToolType, _agentOptions.Name);
                        allValid = false;
                    }
                }

                return allValid;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Failed to validate connections for agent '{AgentName}'", _agentOptions.Name);
                return false;
            }
        }

public async Task<bool> ValidateConnectionsAsync()
        {
            try
            {
                if (_agentOptions.Tools == null || _agentOptions.Tools.Count == 0)
                {
                    _logger.LogInformation("Agent '{AgentName}' has no tools to validate", _agentOptions.Name);
                    return true;
                }

                var allValid = true;
                foreach (var tool in _agentOptions.Tools)
                {
                    var isValid = await _agentService.ValidateConnectionAsync(tool.ConnectionName);
                    if (!isValid)
                    {
                        _logger.LogWarning("Invalid connection '{ConnectionName}' for tool '{ToolType}' in agent '{AgentName}'", 
                            tool.ConnectionName, tool.ToolType, _agentOptions.Name);
                        allValid = false;
                    }
                }

                return allValid;
            }
            catch (Exception ex)
            {
                _logger.LogError(ex, "Failed to validate connections for agent '{AgentName}'", _agentOptions.Name);
                return false;
            }
        }

Connection Validation: Before an agent becomes active, it validates that all required connections are working properly. This prevents agents from failing during user interactions.

Error Handling: The wrapper includes comprehensive error handling and logging, making it easier to diagnose issues in production environments.

2.4 Main Application

The main application brings everything together:

using Azure.AI.Agents.Persistent;
using BingGroundingAgent;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System.ComponentModel.DataAnnotations;

var builder = Host.CreateApplicationBuilder(args);

var configuration = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddEnvironmentVariables()
    .Build();

builder.Services.AddSingleton<IConfiguration>(configuration);
builder.Services.AddSingleton<AgentService>();

// Register agents from configuration
var agentsConfig = new AgentsConfiguration();
configuration.GetSection("Agents").Bind(agentsConfig.Agents);

if (agentsConfig.Agents.Count == 0)
{
	Console.WriteLine("No agents configured. Please check your appsettings.json file.");
	return;
}

// Validate agent configurations
var validAgents = new List<AgentOptions>();
foreach (var agentConfig in agentsConfig.Agents)
{
	var validationResults = new List<ValidationResult>();
	var validationContext = new ValidationContext(agentConfig);

	if (Validator.TryValidateObject(agentConfig, validationContext, validationResults, true))
	{
		validAgents.Add(agentConfig);
	}
	else
	{
		var errors = string.Join(", ", validationResults.Select(vr => vr.ErrorMessage));
		Console.WriteLine($"Invalid configuration for agent '{agentConfig.Name}': {errors}");
	}
}

if (validAgents.Count == 0)
{
	Console.WriteLine("No valid agents found. Please check your configuration.");
	return;
}

// Register each agent as a named service
foreach (var agentConfig in agentsConfig.Agents)
{
    var agentType = agentConfig.Name; // Fallback to name if type is not set
    builder.Services.AddKeyedSingleton<AzureAgent>(
        agentType,
        (provider, key) =>
            new AzureAgent(
                provider.GetRequiredService<ILogger<AzureAgent>>(),
                provider.GetRequiredService<AgentService>(),
                agentConfig
            )
    );
}

builder.Services.AddLogging(logging =>
{
    logging.AddConsole();
    logging.SetMinimumLevel(LogLevel.Information);
});

var host = builder.Build();
var logger = host.Services.GetRequiredService<ILogger<Program>>();

// Get available agents from configuration
var availableAgents = new AgentsConfiguration();
configuration.GetSection("Agents").Bind(availableAgents.Agents);

// Let user select which agent to use
Console.WriteLine("Available agents:");
for (int i = 0; i < availableAgents.Agents.Count; i++)
{
    var agent = availableAgents.Agents[i];
    Console.WriteLine($"{i + 1}. {agent.Name}");
}

Console.Write("Select an agent (1-{0}): ", availableAgents.Agents.Count);
var selection = Console.ReadLine();

if (
    !int.TryParse(selection, out int agentIndex)
    || agentIndex < 1
    || agentIndex > availableAgents.Agents.Count
)
{
    logger.LogError("Invalid agent selection");
    return;
}

var selectedAgentConfig = availableAgents.Agents[agentIndex - 1];
var agentKey = selectedAgentConfig.Name;
var selectedAgent = host.Services.GetRequiredKeyedService<AzureAgent>(agentKey);

try
{
    logger.LogInformation("Initializing {AgentName}...", selectedAgentConfig.Name);
    await selectedAgent.InitializeAsync();

    while (true)
    {
        Console.Write("\nYou: ");
        var input = Console.ReadLine();
        if (string.IsNullOrEmpty(input) || input.ToLower() == "exit")
            break;

        logger.LogInformation("Processing query: {Query}", input);

        Console.Write("Assistant: ");
        await foreach (var update in await selectedAgent.ProcessQueryAsync(input))
        {
            try
            {
                if (update.UpdateKind == StreamingUpdateReason.MessageUpdated)
                {
                    if (update is MessageContentUpdate messageContent)
                    {
                        // Filter out citation markers and special characters
                        var text = messageContent.Text;
                        if (!string.IsNullOrEmpty(text))
                        {
                            Console.Write(text);
                        }
                    }
                }
            }
            catch (Exception updateEx)
            {
                logger.LogWarning(updateEx, "Error processing streaming update");
            }
        }

        // Output references (citations)
        await selectedAgent.DisplayCitationsAsync();
    }
}
catch (Exception ex)
{
    logger.LogError(ex, "An error occurred: {Message}", ex.Message);
}

logger.LogInformation("Application completed");
await host.StopAsync();

using Azure.AI.Agents.Persistent;
using BingGroundingAgent;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System.ComponentModel.DataAnnotations;

var builder = Host.CreateApplicationBuilder(args);

var configuration = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddEnvironmentVariables()
    .Build();

builder.Services.AddSingleton<IConfiguration>(configuration);
builder.Services.AddSingleton<AgentService>();

// Register agents from configuration
var agentsConfig = new AgentsConfiguration();
configuration.GetSection("Agents").Bind(agentsConfig.Agents);

if (agentsConfig.Agents.Count == 0)
{
	Console.WriteLine("No agents configured. Please check your appsettings.json file.");
	return;
}

// Validate agent configurations
var validAgents = new List<AgentOptions>();
foreach (var agentConfig in agentsConfig.Agents)
{
	var validationResults = new List<ValidationResult>();
	var validationContext = new ValidationContext(agentConfig);

	if (Validator.TryValidateObject(agentConfig, validationContext, validationResults, true))
	{
		validAgents.Add(agentConfig);
	}
	else
	{
		var errors = string.Join(", ", validationResults.Select(vr => vr.ErrorMessage));
		Console.WriteLine($"Invalid configuration for agent '{agentConfig.Name}': {errors}");
	}
}

if (validAgents.Count == 0)
{
	Console.WriteLine("No valid agents found. Please check your configuration.");
	return;
}

// Register each agent as a named service
foreach (var agentConfig in agentsConfig.Agents)
{
    var agentType = agentConfig.Name; // Fallback to name if type is not set
    builder.Services.AddKeyedSingleton<AzureAgent>(
        agentType,
        (provider, key) =>
            new AzureAgent(
                provider.GetRequiredService<ILogger<AzureAgent>>(),
                provider.GetRequiredService<AgentService>(),
                agentConfig
            )
    );
}

builder.Services.AddLogging(logging =>
{
    logging.AddConsole();
    logging.SetMinimumLevel(LogLevel.Information);
});

var host = builder.Build();
var logger = host.Services.GetRequiredService<ILogger<Program>>();

// Get available agents from configuration
var availableAgents = new AgentsConfiguration();
configuration.GetSection("Agents").Bind(availableAgents.Agents);

// Let user select which agent to use
Console.WriteLine("Available agents:");
for (int i = 0; i < availableAgents.Agents.Count; i++)
{
    var agent = availableAgents.Agents[i];
    Console.WriteLine($"{i + 1}. {agent.Name}");
}

Console.Write("Select an agent (1-{0}): ", availableAgents.Agents.Count);
var selection = Console.ReadLine();

if (
    !int.TryParse(selection, out int agentIndex)
    || agentIndex < 1
    || agentIndex > availableAgents.Agents.Count
)
{
    logger.LogError("Invalid agent selection");
    return;
}

var selectedAgentConfig = availableAgents.Agents[agentIndex - 1];
var agentKey = selectedAgentConfig.Name;
var selectedAgent = host.Services.GetRequiredKeyedService<AzureAgent>(agentKey);

try
{
    logger.LogInformation("Initializing {AgentName}...", selectedAgentConfig.Name);
    await selectedAgent.InitializeAsync();

    while (true)
    {
        Console.Write("\nYou: ");
        var input = Console.ReadLine();
        if (string.IsNullOrEmpty(input) || input.ToLower() == "exit")
            break;

        logger.LogInformation("Processing query: {Query}", input);

        Console.Write("Assistant: ");
        await foreach (var update in await selectedAgent.ProcessQueryAsync(input))
        {
            try
            {
                if (update.UpdateKind == StreamingUpdateReason.MessageUpdated)
                {
                    if (update is MessageContentUpdate messageContent)
                    {
                        // Filter out citation markers and special characters
                        var text = messageContent.Text;
                        if (!string.IsNullOrEmpty(text))
                        {
                            Console.Write(text);
                        }
                    }
                }
            }
            catch (Exception updateEx)
            {
                logger.LogWarning(updateEx, "Error processing streaming update");
            }
        }

        // Output references (citations)
        await selectedAgent.DisplayCitationsAsync();
    }
}
catch (Exception ex)
{
    logger.LogError(ex, "An error occurred: {Message}", ex.Message);
}

logger.LogInformation("Application completed");
await host.StopAsync();

Configuration Binding: The application reads agent configurations from appsettings.json and validates them at startup, preventing invalid configurations from causing runtime errors.

Agent Selection: Users can choose which agent to interact with, making it easy to test different capabilities or switch between internal and external research modes.

Streaming Response Handling: The application processes streaming responses from the AI service, providing real-time feedback to users as the agent formulates its response.

Citation Display: After each response, the application shows citations and sources, making it easy for users to verify information and access original documents.

Test SharePoint agent

Run the application and select SharePoint agent:

Now you can test it with some SharePoint queries:

What documents are available in SharePoint?
Find information about [specific topic in your SharePoint]
Show me the latest [document type] from [SharePoint site]
List key points from document [document name]

Security Considerations

Information Flow Security: Your SharePoint agents handle sensitive corporate information. Understanding the security model is crucial:

Data in Transit: All communication between your application, Azure AI Foundry, and SharePoint uses TLS encryption
Permission Inheritance: Agents automatically respect SharePoint permissions – users only see content they’re authorized to access
No Data Storage: Azure AI Foundry doesn’t store your SharePoint content; it queries it in real-time

SharePoint Site Optimization:

Limit Scope: Connect only the SharePoint sites that contain relevant information
Organize Content: Well-structured SharePoint sites with clear metadata improve search accuracy
Regular Cleanup: Archive or delete outdated documents to improve relevance

Conclusion

You’ve successfully built a powerful SharePoint agent that transform how your organization accesses and utilizes knowledge. These agents eliminate the traditional barriers of manual indexing while maintaining security and permissions.

The impact on your organization:

Time Savings: Users find information in seconds instead of hours
Better Decisions: Access to comprehensive, current information
Knowledge Democratization: Everyone can access organizational knowledge easily
Reduced IT Overhead: No search infrastructure to maintain

Ready to scale? Your SharePoint agents are now ready for broader deployment. Start with a pilot group, gather feedback, and gradually expand access across your organization.

Source Code

Access the complete source code for this blog post on GitHub: GitHub Repository Link