Beyond Manual Indexing: Building Intelligent SharePoint Agents – Part 3: Delegated Permissions and Enterprise Security

by Posted on
0
(0)

In Part 1, we explored the fundamental shift from manual SharePoint indexing to Azure AI Foundry’s SharePoint grounding tool. In Part 2, we built a complete, production-ready SharePoint agent system with multiple agent types.

Now it’s time to tackle the most critical aspect of enterprise SharePoint AI agents: proper identity passthrough and delegated permissions. This is where many implementations fall short in real-world scenarios, and where the true power of Microsoft 365 Copilot API’s identity integration becomes apparent.

The Enterprise Permission Challenge

When I implemented my first SharePoint agent using the basic setup from Part 2, I quickly discovered a significant limitation during enterprise testing. The agent was accessing SharePoint using the application’s service principal identity, not the actual user’s identity. This meant:

  • Security Risk: Users could potentially access documents they shouldn’t have permission to see
  • Compliance Issues: Audit trails showed the service account accessing documents, not individual users
  • Limited Functionality: The agent couldn’t respect user-specific permissions, site access, or department-level restrictions
  • Enterprise Blocker: IT security teams rejected the solution due to permission bypass concerns

The solution? Delegated permissions with On-Behalf-Of (OBO) authentication – letting the SharePoint agent act as the authenticated user rather than as a service account.

Understanding Delegated Permissions vs Application Permissions

Before diving into implementation, let’s clarify the two permission models:

Application Permissions (What We Built in Part 2)

  • Agent uses its own identity to access SharePoint
  • Same access level for all users
  • Simpler to implement but less secure
  • Not suitable for enterprise environments with strict permission requirements

Delegated Permissions (Enterprise-Grade Approach)

  • Agent impersonates the authenticated user
  • Respects individual user permissions
  • Maintains proper audit trails
  • Requires more complex authentication flow

Here’s what happens behind the scenes:

  1. User Authentication: User authenticates with device code flow and gets an access token scoped for Azure AI Foundry
  2. Agent Creation: Azure AI Foundry creates the agent with the user’s context embedded in the token
  3. SharePoint Access: When the SharePoint grounding tool makes requests, Microsoft 365 Copilot API automatically respects the user’s SharePoint permissions

Key Insight: We don’t need to explicitly request SharePoint or Microsoft Graph permissions because the SharePoint grounding tool leverages Microsoft 365 Copilot API, which already has access to SharePoint and handles identity passthrough internally. The user’s existing SharePoint permissions are automatically respected.

Prerequisites for Delegated Permissions

Beyond the requirements from Parts 1 and 2, you’ll need:

  • Azure AD App Registration with delegated Azure AI Foundry permissions
  • Device Code Flow Support (enable public client flows)
  • Microsoft 365 Copilot License for each user (still required)
  • Client Secret for OBO token exchange

Step 1: Azure AD App Registration Setup

First, we need to configure our Azure AD application to support delegated permissions instead of application permissions.

1.1 Update App Registration Permissions

In your Azure AD App Registration:

  1. Navigate to API Permissions
  2. Remove any Application permissions for SharePoint
  3. Select Add a permission
  4. Select APIs my organization uses
  5. Search Azure Machine Learning Services and select it
  6. Add user_impersonation permission

Note: We don’t need SharePoint or Microsoft Graph permissions because the SharePoint grounding tool handles all SharePoint access through Microsoft 365 Copilot API, which respects the user’s existing permissions automatically.

1.2 Generate Client Secret

Even with delegated permissions, you’ll need a client secret for the OBO flow:

  1. Navigate to Certificates & secrets
  2. Create a new Client secret
  3. Copy the secret value (you won’t see it again!)

1.3 Enable Public Client Flows

For console applications with device code flow:

  1. Go to Authentication tab
  2. Under Advanced settings, enable Allow public client flows
  3. Add Mobile and desktop applications platform and then add redirect URI: http://localhost (required for device code flow)

Step 2: Add MSAL Dependencies

Add Microsoft.Authentication.WebAssembly.Msal package to your solution.

Step 3: Device Code Authentication Service

IMPORTANT: Through testing, the correct permission scope for Azure AI Foundry delegated authentication is https://ai.azure.com/user_impersonation. This is the most important configuration detail for success.

Create a new authentication service that works perfectly with console applications:

C#
using Microsoft.Authentication.WebAssembly.Msal;

public interface IUserAuthenticationService
{
    Task<string> GetUserAccessTokenAsync();
    Task<bool> IsUserAuthenticatedAsync();
    Task SignInAsync();
    Task SignOutAsync();
    Task<string> GetUserIdentityAsync();
}

public class DeviceCodeAuthenticationService : IUserAuthenticationService
{
    private readonly IPublicClientApplication _clientApp;
    private readonly ILogger<DeviceCodeAuthenticationService> _logger;
    private readonly string[] _scopes = 
    {
        "https://ai.azure.com/user_impersonation" // For Azure AI Foundry OBO
    };

    public DeviceCodeAuthenticationService(IConfiguration configuration, ILogger<DeviceCodeAuthenticationService> logger)
    {
        _logger = logger;
        
        var clientId = configuration["AzureAd:ClientId"] ?? throw new InvalidOperationException("AzureAd:ClientId not configured");
        var tenantId = configuration["AzureAd:TenantId"] ?? throw new InvalidOperationException("AzureAd:TenantId not configured");
        
        _clientApp = PublicClientApplicationBuilder
            .Create(clientId)
            .WithAuthority($"https://login.microsoftonline.com/{tenantId}")
            .WithDefaultRedirectUri() // Uses http://localhost for device code flow
            .Build();
    }

    public async Task<string> GetUserAccessTokenAsync()
    {
        try
        {
            var accounts = await _clientApp.GetAccountsAsync();
            var firstAccount = accounts.FirstOrDefault();

            AuthenticationResult result;
            
            if (firstAccount != null)
            {
                // Try to get token silently (from cache)
                try
                {
                    result = await _clientApp.AcquireTokenSilent(_scopes, firstAccount)
                        .ExecuteAsync();
                    _logger.LogInformation("Successfully acquired token from cache for user: {Username}", firstAccount.Username);
                    return result.AccessToken;
                }
                catch (MsalUiRequiredException)
                {
                    // Silent acquisition failed, fall through to device code flow
                    _logger.LogInformation("Silent token acquisition failed, using device code flow");
                }
            }

            // Use device code flow - perfect for console apps!
            result = await _clientApp.AcquireTokenWithDeviceCode(_scopes, deviceCodeResult =>
            {
                Console.WriteLine();
                Console.WriteLine("Authentication Required");
                Console.WriteLine("─────────────────────────────────────────────────────");
                Console.WriteLine($"Please open a web browser and navigate to:");
                Console.WriteLine($"{deviceCodeResult.VerificationUrl}");
                Console.WriteLine();
                Console.WriteLine($"Enter this code: {deviceCodeResult.UserCode}");
                Console.WriteLine("─────────────────────────────────────────────────────");
                Console.WriteLine("Waiting for you to complete authentication...");
                Console.WriteLine();
                
                return Task.FromResult(0);
            }).ExecuteAsync();

            _logger.LogInformation("Successfully authenticated user: {Username}", result.Account.Username);
            return result.AccessToken;
        }
        catch (MsalException ex)
        {
            _logger.LogError(ex, "Failed to acquire user access token");
            throw;
        }
    }

    public async Task<bool> IsUserAuthenticatedAsync()
    {
        var accounts = await _clientApp.GetAccountsAsync();
        return accounts.Any();
    }

    public async Task SignInAsync()
    {
        await GetUserAccessTokenAsync(); // Device code flow handles sign-in
    }

    public async Task SignOutAsync()
    {
        var accounts = await _clientApp.GetAccountsAsync();
        foreach (var account in accounts)
        {
            await _clientApp.RemoveAsync(account);
        }
        _logger.LogInformation("User signed out successfully");
    }

    public async Task<string> GetUserIdentityAsync()
    {
        var accounts = await _clientApp.GetAccountsAsync();
        return accounts.FirstOrDefault()?.Username ?? "Unknown User";
    }
}

Step 4: Enhanced Agent Service with Delegated Permissions

Create a new DelegatedAgentService that builds on your existing pattern but uses delegated credentials:

C#
using Azure.AI.Agents.Persistent;
using Azure.AI.Projects;
using Azure.Core;
using Azure.Identity;
using AzureAIAgents;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using System.ComponentModel.DataAnnotations;

public class DelegatedAgentService
{
    private readonly IConfiguration _configuration;
    private readonly ILogger<DelegatedAgentService> _logger;
    private readonly IUserAuthenticationService _authService;
    private readonly ITokenExchangeService _tokenExchange;
    private PersistentAgentsClient _persistentClient;
    private Connections _connections;
    private AIProjectClient _projectClient;

    public DelegatedAgentService(
        IConfiguration configuration,
        ILogger<DelegatedAgentService> logger,
        IUserAuthenticationService authService,
        ITokenExchangeService tokenExchange)
    {
        _configuration = configuration ?? throw new ArgumentNullException(nameof(configuration));
        _logger = logger ?? throw new ArgumentNullException(nameof(logger));
        _authService = authService;
        _tokenExchange = tokenExchange;
    }

    public async Task InitializeAsync()
    {
        try
        {
            var connectionString = _configuration["AzureAI:ConnectionString"]
                ?? throw new InvalidOperationException("Azure AI connection string is not configured");

            // Ensure user is authenticated
            if (!await _authService.IsUserAuthenticatedAsync())
            {
                await _authService.SignInAsync();
            }

            // Get user's access token and exchange for Azure AI token
            var userToken = await _authService.GetUserAccessTokenAsync();
            var azureAiToken = await _tokenExchange.ExchangeUserTokenAsync(userToken);
            var credential = new StaticTokenCredential(azureAiToken);

            // Create Azure AI Project client with delegated credentials
            _projectClient = new AIProjectClient(
                new Uri(connectionString),
                credential
            );
            _connections = _projectClient.GetConnectionsClient();

            // Create Azure AI persistent client with delegated credentials
            _persistentClient = new PersistentAgentsClient(
                connectionString,
                credential
            );

            _logger.LogInformation("Successfully initialized Azure AI clients with delegated permissions");
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to initialize Azure AI clients with delegated permissions");
            throw;
        }
    }

    // Implement the same methods as AgentService but with delegated credentials
    public async Task<PersistentAgent?> GetAgentAsync(string name)
    {
        try
        {
            ArgumentException.ThrowIfNullOrWhiteSpace(name, nameof(name));

            var agents = _persistentClient.Administration.GetAgentsAsync();
            var agent = await agents.Where(a => a.Name == name).FirstOrDefaultAsync();

            if (agent == null)
            {
                _logger.LogWarning("Agent '{AgentName}' not found", name);
            }
            else
            {
                _logger.LogDebug("Found existing agent '{AgentName}' with ID: {AgentId}", name, agent.Id);
            }

            return agent;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to get agent {AgentName}", name);
            return null;
        }
    }

    public async Task<Connection?> GetConnectionByNameAsync(string connectionName)
    {
        try
        {
            ArgumentException.ThrowIfNullOrWhiteSpace(connectionName, nameof(connectionName));

            await foreach (var connection in _connections.GetConnectionsAsync())
            {
                if (string.Equals(connection.Name, connectionName, StringComparison.OrdinalIgnoreCase))
                {
                    _logger.LogDebug("Found connection '{ConnectionName}' with ID: {ConnectionId}",
                        connection.Name, connection.Id);
                    return connection;
                }
            }

            _logger.LogWarning("Connection '{ConnectionName}' not found", connectionName);
            return null;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to get connection '{ConnectionName}'", connectionName);
            throw;
        }
    }

    public async Task<PersistentAgent> CreateAgentAsync(AgentOptions agentOptions)
    {
        try
        {
            ArgumentNullException.ThrowIfNull(agentOptions, nameof(agentOptions));

            // Check if agent already exists
            var existingAgent = await GetAgentAsync(agentOptions.Name);
            if (existingAgent != null)
            {
                _logger.LogInformation("Agent '{AgentName}' already exists with ID: {AgentId}",
                    agentOptions.Name, existingAgent.Id);
                return existingAgent;
            }

            // Create tools
            var toolDefinitions = new List<ToolDefinition>();

            if (agentOptions.Tools != null)
            {
                foreach (var tool in agentOptions.Tools)
                {
                    var toolDefinition = await CreateToolDefinitionAsync(tool);
                    if (toolDefinition != null)
                    {
                        toolDefinitions.Add(toolDefinition);
                    }
                }
            }

            // Create agent with delegated permissions
            var agent = await _persistentClient.Administration.CreateAgentAsync(
                model: agentOptions.Deployment,
                name: agentOptions.Name,
                description: agentOptions.Description,
                instructions: agentOptions.Instructions,
                tools: toolDefinitions,
                toolResources: new ToolResources()
            );

            _logger.LogInformation("Successfully created agent '{AgentName}' with ID: {AgentId} and {ToolCount} tools (delegated permissions)",
                agentOptions.Name, agent.Value.Id, toolDefinitions.Count);

            return agent;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to create agent '{AgentName}' with delegated permissions", agentOptions?.Name ?? "Unknown");
            throw;
        }
    }

    private async Task<ToolDefinition?> CreateToolDefinitionAsync(ToolsOptions tool)
    {
        try
        {
            return tool.ToolType switch
            {
                "SharePointGrounding" => await CreateSharePointToolAsync(tool),
                "BingGroundingSearch" => await CreateBingSearchToolAsync(tool),
                "CustomBingGroundingSearch" => await CreateCustomBingSearchToolAsync(tool),
                _ => throw new NotSupportedException($"Tool type '{tool.ToolType}' is not supported")
            };
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to create tool definition for {ToolType} with connection {ConnectionName}",
                tool.ToolType, tool.ConnectionName);
            return null;
        }
    }

    private async Task<ToolDefinition> CreateSharePointToolAsync(ToolsOptions tool)
    {
        var connection = await GetConnectionByNameAsync(tool.ConnectionName);
        if (connection?.Id == null)
        {
            throw new InvalidOperationException($"SharePoint connection '{tool.ConnectionName}' not found or invalid");
        }

        _logger.LogInformation("Creating SharePoint grounding tool with connection: {ConnectionName} (delegated permissions)",
            tool.ConnectionName);
        
        return new SharepointToolDefinition(
            new SharepointGroundingToolParameters(connection.Id)
        );
    }

    private async Task<ToolDefinition> CreateBingSearchToolAsync(ToolsOptions tool)
    {
        var connection = await GetConnectionByNameAsync(tool.ConnectionName);
        if (connection?.Id == null)
        {
            throw new InvalidOperationException($"Bing Search connection '{tool.ConnectionName}' not found or invalid");
        }

        return new BingGroundingToolDefinition(
            new BingGroundingSearchToolParameters(
                [new BingGroundingSearchConfiguration(connection.Id)]
            )
        );
    }

    private async Task<ToolDefinition> CreateCustomBingSearchToolAsync(ToolsOptions tool)
    {
        var connection = await GetConnectionByNameAsync(tool.ConnectionName);
        if (connection?.Id == null)
        {
            throw new InvalidOperationException($"Custom Bing Search connection '{tool.ConnectionName}' not found or invalid");
        }

        var configuration = new BingCustomSearchConfiguration(
            connection.Id,
            tool.ConfigurationName ?? "default")
        {
            Count = 5,
            SetLang = "en",
            Market = "en-us"
        };

        return new BingCustomSearchToolDefinition(
            new BingCustomSearchToolParameters([configuration])
        );
    }

    public async Task<PersistentAgentThread> CreateThreadAsync()
    {
        try
        {
            var thread = await _persistentClient.Threads.CreateThreadAsync();
            _logger.LogDebug("Created new thread with ID: {ThreadId}", thread.Value.Id);
            return thread;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to create thread");
            throw;
        }
    }

    public async Task<PersistentThreadMessage> CreateMessageAsync(string threadId, string query)
    {
        try
        {
            ArgumentException.ThrowIfNullOrWhiteSpace(threadId, nameof(threadId));
            ArgumentException.ThrowIfNullOrWhiteSpace(query, nameof(query));

            var message = await _persistentClient.Messages.CreateMessageAsync(
                threadId,
                MessageRole.User,
                query
            );

            _logger.LogDebug("Created message in thread {ThreadId}", threadId);
            return message;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to create message in thread {ThreadId}", threadId);
            throw;
        }
    }

    public AsyncCollectionResult<StreamingUpdate> CreateStreamingAsync(string threadId, string agentId)
    {
        try
        {
            ArgumentException.ThrowIfNullOrWhiteSpace(threadId, nameof(threadId));
            ArgumentException.ThrowIfNullOrWhiteSpace(agentId, nameof(agentId));

            _logger.LogDebug("Starting streaming for thread {ThreadId} with agent {AgentId} (delegated permissions)", threadId, agentId);
            return _persistentClient.Runs.CreateRunStreamingAsync(threadId, agentId);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to create streaming for thread {ThreadId}, agent {AgentId}",
                threadId, agentId);
            throw;
        }
    }

    public async Task<List<MessageTextUriCitationAnnotation>> GetCitationsAsync(string threadId)
    {
        var citations = new List<MessageTextUriCitationAnnotation>();

        try
        {
            ArgumentException.ThrowIfNullOrWhiteSpace(threadId, nameof(threadId));

            var messages = _persistentClient.Messages.GetMessagesAsync(threadId);
            var messagesList = await messages.ToListAsync();

            if (messagesList.Count > 0)
            {
                var lastMessage = messagesList[0]; // Messages are returned in reverse chronological order
                foreach (var contentItem in lastMessage.ContentItems)
                {
                    if (contentItem is MessageTextContent textItem)
                    {
                        foreach (var annotation in textItem.Annotations)
                        {
                            if (annotation is MessageTextUriCitationAnnotation uriCitation)
                            {
                                citations.Add(uriCitation);
                            }
                        }
                    }
                }
            }

            _logger.LogDebug("Retrieved {CitationCount} citations from thread {ThreadId}", citations.Count, threadId);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to get citations for thread {ThreadId}", threadId);
        }

        return citations;
    }

    public async Task<bool> ValidateConnectionAsync(string connectionName)
    {
        try
        {
            var connection = await GetConnectionByNameAsync(connectionName);
            var isValid = connection != null;

            _logger.LogInformation("Connection validation for '{ConnectionName}': {IsValid} (delegated permissions)",
                connectionName, isValid ? "Valid" : "Invalid");

            return isValid;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to validate connection: {ConnectionName}", connectionName);
            return false;
        }
    }

    public async Task<string> GetCurrentUserAsync()
    {
        return await _authService.GetUserIdentityAsync();
    }
}

// Helper class for static token credential  
public class StaticTokenCredential : TokenCredential
{
    private readonly string _token;
    private readonly DateTimeOffset _expiresOn;

    public StaticTokenCredential(string token, DateTimeOffset? expiresOn = null)
    {
        _token = token;
        _expiresOn = expiresOn ?? DateTimeOffset.UtcNow.AddHours(1);
    }

    public override AccessToken GetToken(TokenRequestContext requestContext, CancellationToken cancellationToken)
    {
        return new AccessToken(_token, _expiresOn);
    }

    public override ValueTask<AccessToken> GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken)
    {
        return new ValueTask<AccessToken>(GetToken(requestContext, cancellationToken));
    }
}

Step 5: Configuration Updates

Update your existing appsettings.json to support delegated authentication:

JSON
{
...,
"AzureAd": {
    "ClientId": "your-app-client-id",
    "ClientSecret": "your-app-client-secret", 
    "TenantId": "your-tenant-id"
  },
...
}

Step 6: Enhanced AzureAgent for Delegated Permissions

Create a new DelegatedAzureAgent.cs that works with your existing pattern:

C#
using System.ClientModel;
using Azure.AI.Agents.Persistent;
using Microsoft.Extensions.Logging;

namespace AzureAIAgents;

public sealed class DelegatedAzureAgent
{
    private readonly ILogger _logger;
    private readonly DelegatedAgentService _agentService;
    private readonly AgentOptions _agentOptions;
    private PersistentAgent? _agent;
    private PersistentAgentThread? _thread;

    public DelegatedAzureAgent(ILogger<DelegatedAzureAgent> logger, DelegatedAgentService agentService, AgentOptions agentOptions)
    {
        _logger = logger ?? throw new ArgumentNullException(nameof(logger));
        _agentService = agentService ?? throw new ArgumentNullException(nameof(agentService));
        _agentOptions = agentOptions ?? throw new ArgumentNullException(nameof(agentOptions));
    }

    public async Task InitializeAsync()
    {
        try
        {
            // Initialize the delegated agent service first
            await _agentService.InitializeAsync();
            
            var currentUser = await _agentService.GetCurrentUserAsync();
            Console.WriteLine($"Initializing agent with delegated permissions for: {currentUser}");
            
            // Define the agent using delegated credentials
            _agent = await _agentService.CreateAgentAsync(_agentOptions);
            
            Console.WriteLine($"Agent '{_agentOptions.Name}' ready with user-level permissions");
            
            _logger.LogInformation(
                "{AgentName} initialized successfully with delegated permissions. Agent ID: {AgentId}",
                _agentOptions.Name,
                _agent?.Id
            );
        }
        catch (Exception ex)
        {
            Console.WriteLine($"Failed to initialize delegated agent: {ex.Message}");
            _logger.LogError(ex, "Failed to initialize {AgentName} with delegated permissions", _agentOptions.Name);
            throw;
        }
    }

    public async Task<AsyncCollectionResult<StreamingUpdate>> ProcessQueryAsync(string userQuery)
    {
        ArgumentException.ThrowIfNullOrWhiteSpace(userQuery);
        if (_agent == null)
        {
            throw new InvalidOperationException(
                "Agent is not initialized. Call InitializeAsync first."
            );
        }

        try
        {
            var currentUser = await _agentService.GetCurrentUserAsync();
            Console.WriteLine($"Processing query as user: {currentUser}");
            
            _logger.LogInformation("Processing query: {Query}", userQuery);

            // Create a new thread for the conversation
            var thread = await _agentService.CreateThreadAsync();
            _thread = thread;

            // Add user message
            await _agentService.CreateMessageAsync(thread.Id, userQuery);

            // Get response from agent with delegated permissions
            return _agentService.CreateStreamingAsync(thread.Id, _agent.Id);
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Error processing query: {Query}", userQuery);
            throw;
        }
    }

    public async Task DisplayCitationsAsync()
    {
        if (_thread == null)
        {
            _logger.LogWarning("No active thread to get citations from");
            return;
        }

        try
        {
            var citations = await _agentService.GetCitationsAsync(_thread.Id);

            if (citations.Count > 0)
            {
                Console.WriteLine();
                Console.WriteLine("Sources:");
                Console.WriteLine(new string('-', 50));

                for (int i = 0; i < citations.Count; i++)
                {
                    var citation = citations[i];
                    Console.WriteLine($"{i + 1}. {citation.UriCitation.Uri}");
                }
                Console.WriteLine();
            }
            else
            {
                _logger.LogDebug("No citations found for current thread");
            }
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to display citations for agent '{AgentName}'", _agentOptions.Name);
        }
    }

    public async Task<bool> ValidateConnectionsAsync()
    {
        try
        {
            if (_agentOptions.Tools == null || _agentOptions.Tools.Count == 0)
            {
                _logger.LogInformation("Agent '{AgentName}' has no tools to validate", _agentOptions.Name);
                return true;
            }

            var allValid = true;
            foreach (var tool in _agentOptions.Tools)
            {
                var isValid = await _agentService.ValidateConnectionAsync(tool.ConnectionName);
                if (!isValid)
                {
                    _logger.LogWarning("Invalid connection '{ConnectionName}' for tool '{ToolType}' in agent '{AgentName}'",
                        tool.ConnectionName, tool.ToolType, _agentOptions.Name);
                    allValid = false;
                }
            }

            return allValid;
        }
        catch (Exception ex)
        {
            _logger.LogError(ex, "Failed to validate connections for agent '{AgentName}'", _agentOptions.Name);
            return false;
        }
    }

    // Expose agent options for easier access
    public AgentOptions AgentOptions => _agentOptions;
}

Step 7: Updated Program.cs with Authentication Choice

Modify your existing Program.cs to support both application and delegated permissions:

C#
using Azure.AI.Agents.Persistent;
using AzureAIAgents;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;
using System.ComponentModel.DataAnnotations;

var builder = Host.CreateApplicationBuilder(args);

var configuration = new ConfigurationBuilder()
    .SetBasePath(Directory.GetCurrentDirectory())
    .AddJsonFile("appsettings.json", optional: false, reloadOnChange: true)
    .AddEnvironmentVariables()
    .Build();

builder.Services.AddSingleton<IConfiguration>(configuration);

// Register authentication services
builder.Services.AddHttpClient();
builder.Services.AddSingleton<IUserAuthenticationService, DeviceCodeAuthenticationService>();
builder.Services.AddSingleton<ITokenExchangeService, TokenExchangeService>();

// Register both agent services
builder.Services.AddSingleton<AgentService>();
builder.Services.AddSingleton<DelegatedAgentService>();

// Register agents from configuration
var agentsConfig = new AgentsConfiguration();
configuration.GetSection("Agents").Bind(agentsConfig.Agents);

if (agentsConfig.Agents.Count == 0)
{
    Console.WriteLine("No agents configured. Please check your appsettings.json file.");
    return;
}

// Register agents with both application and delegated versions
foreach (var agentConfig in agentsConfig.Agents)
{
    // Regular agent with application permissions
    builder.Services.AddKeyedSingleton<AzureAgent>(
        $"{agentConfig.Name}:Application",
        (provider, key) => new AzureAgent(
            provider.GetRequiredService<ILogger<AzureAgent>>(),
            provider.GetRequiredService<AgentService>(),
            agentConfig
        )
    );

    // Delegated agent with user permissions
    builder.Services.AddKeyedSingleton<DelegatedAzureAgent>(
        $"{agentConfig.Name}:Delegated",
        (provider, key) => new DelegatedAzureAgent(
            provider.GetRequiredService<ILogger<DelegatedAzureAgent>>(),
            provider.GetRequiredService<DelegatedAgentService>(),
            agentConfig
        )
    );
}

builder.Services.AddLogging(logging =>
{
    logging.AddConsole();
    logging.SetMinimumLevel(LogLevel.Information);
});

var host = builder.Build();
var logger = host.Services.GetRequiredService<ILogger<Program>>();

try
{
    Console.WriteLine(" Azure AI Agent with Permission Options");
    Console.WriteLine("=========================================");

    // Let user choose permission type
    Console.WriteLine("\nSelect permission type:");
    Console.WriteLine("1. Application Permissions (Service Account - your current setup)");
    Console.WriteLine("2. Delegated Permissions (User Authentication)");
    Console.Write("Choice (1-2): ");
    
    var permissionChoice = Console.ReadLine();
    bool useDelegatedPermissions = permissionChoice == "2";
    
    if (useDelegatedPermissions)
    {
        Console.WriteLine("\n Delegated Permissions Selected");
        Console.WriteLine("You will be prompted to authenticate...");
    }
    else
    {
        Console.WriteLine("\n Application Permissions Selected");
        Console.WriteLine("Using service account authentication...");
    }

    // Get available agents from configuration
    var availableAgents = new AgentsConfiguration();
    configuration.GetSection("Agents").Bind(availableAgents.Agents);

    // Let user select which agent to use
    Console.WriteLine($"\nAvailable agents:");
    for (int i = 0; i < availableAgents.Agents.Count; i++)
    {
        var agent = availableAgents.Agents[i];
        Console.WriteLine($"{i + 1}. {agent.Name}");
    }

    Console.Write($"Select an agent (1-{availableAgents.Agents.Count}): ");
    var selection = Console.ReadLine();

    if (!int.TryParse(selection, out int agentIndex) ||
        agentIndex < 1 || agentIndex > availableAgents.Agents.Count)
    {
        logger.LogError("Invalid agent selection");
        return;
    }

    var selectedAgentConfig = availableAgents.Agents[agentIndex - 1];

    if (useDelegatedPermissions)
    {
        // Use delegated permissions
        var agentKey = $"{selectedAgentConfig.Name}:Delegated";
        var selectedAgent = host.Services.GetRequiredKeyedService<DelegatedAzureAgent>(agentKey);

        logger.LogInformation("Initializing {AgentName} with delegated permissions...", selectedAgentConfig.Name);
        await selectedAgent.InitializeAsync();

        await RunChatLoop(selectedAgent, logger);
    }
    else
    {
        // Use application permissions (your existing setup)
        var agentKey = $"{selectedAgentConfig.Name}:Application";
        var selectedAgent = host.Services.GetRequiredKeyedService<AzureAgent>(agentKey);

        logger.LogInformation("Initializing {AgentName} with application permissions...", selectedAgentConfig.Name);
        await selectedAgent.InitializeAsync();

        await RunChatLoop(selectedAgent, logger);
    }
}
catch (Exception ex)
{
    logger.LogError(ex, "An error occurred: {Message}", ex.Message);
    Console.WriteLine($"Error: {ex.Message}");
}

logger.LogInformation("Application completed");
await host.StopAsync();

// Helper method for chat loop (works with both agent types)
static async Task RunChatLoop<T>(T agent, ILogger logger) where T : class
{
    Console.WriteLine("\n Chat with your agent (type 'exit' to quit):");
    
    if (agent is DelegatedAzureAgent)
    {
        Console.WriteLine("Using delegated permissions - agent will only access content you have permission to view.");
    }
    else
    {
        Console.WriteLine("Using application permissions - agent has broad access to configured SharePoint sites.");
    }
    
    Console.WriteLine();

    while (true)
    {
        Console.Write("You: ");
        var input = Console.ReadLine();
        
        if (string.IsNullOrEmpty(input) || input.ToLower() == "exit")
            break;

        logger.LogInformation("Processing query: {Query}", input);

        Console.Write("Assistant: ");
        
        // Handle both AzureAgent and DelegatedAzureAgent
        AsyncCollectionResult<StreamingUpdate> updates;
        if (agent is DelegatedAzureAgent delegatedAgent)
        {
            updates = await delegatedAgent.ProcessQueryAsync(input);
        }
        else if (agent is AzureAgent regularAgent)
        {
            updates = await regularAgent.ProcessQueryAsync(input);
        }
        else
        {
            throw new InvalidOperationException("Unknown agent type");
        }

        await foreach (var update in updates)
        {
            try
            {
                if (update.UpdateKind == StreamingUpdateReason.MessageUpdated)
                {
                    if (update is MessageContentUpdate messageContent)
                    {
                        var text = messageContent.Text;
                        if (!string.IsNullOrEmpty(text))
                        {
                            Console.Write(text);
                        }

                        if (messageContent.TextAnnotation != null)
                        {
                            Console.Write(messageContent.TextAnnotation.Url);
                        }
                    }
                }
            }
            catch (Exception updateEx)
            {
                logger.LogWarning(updateEx, "Error processing streaming update");
            }
        }

        // Display citations for both agent types
        if (agent is DelegatedAzureAgent delegatedAgentForCitations)
        {
            await delegatedAgentForCitations.DisplayCitationsAsync();
        }
        else if (agent is AzureAgent regularAgentForCitations)
        {
            await regularAgentForCitations.DisplayCitationsAsync();
        }
        
        Console.WriteLine();
    }
}

Step 8: Testing Delegated Permissions

When you run your updated console application, you’ll see this flow:

To test with different users, simply run the application multiple times and authenticate with different accounts during the device code flow. Each session will respect that user’s specific SharePoint permissions.

Conclusion: Enterprise-Ready SharePoint Agents

We’ve successfully transformed our basic SharePoint agent into an enterprise-grade solution with proper delegated permissions using a remarkably simple approach. You now have:

  • User-Level Security: Agents respect individual SharePoint permissions automatically
  • Device Code Authentication: Perfect console app authentication flow
  • Identity Passthrough: Seamless OBO authentication with Azure AI Foundry
  • Audit Compliance: Full traceability through Microsoft 365 Copilot API
  • Simplified Architecture: No complex Graph API management – SharePoint grounding handles everything

The foundation you’ve built with delegated permissions opens up possibilities for sophisticated enterprise AI scenarios while maintaining the security and compliance standards your organization requires.

Critical Success Factor

The most important discovery: The correct delegated permission scope for Azure AI Foundry is:

JSON
https://ai.azure.com/user_impersonation

This specific scope is essential for the OBO token exchange to work properly. Using incorrect scopes (like cognitiveservices.azure.com) will result in authentication failures.

Source Code

Access the complete source code for this blog post on GitHub: GitHub Repository Link

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Published by Ramin Ahmadi