Agents that can look things up, take actions and work inside the tools employees already use are one of the most valuable applications of generative AI. On Microsoft’s platform there are several ways to build them. This guide explains the options and a reference architecture that works.
The building blocks
- Models — Azure OpenAI and other models from the Azure AI Foundry catalogue.
- Knowledge — Azure AI Search over SharePoint, files and databases for grounded answers.
- Actions — connectors and APIs: Microsoft Graph, Dynamics 365, ServiceNow, SAP, custom APIs via Azure API Management, and Model Context Protocol (MCP) tools.
- Orchestration — the agent service in Azure AI Foundry, open-source frameworks such as Semantic Kernel, or Copilot Studio for low-code agents.
- Channels — Microsoft Teams, Microsoft 365 Copilot, web chat and your own applications.
Choosing a build approach
| Copilot Studio | Azure AI Foundry | |
|---|---|---|
| Who builds | Makers and IT, low-code | Software engineers, pro-code |
| Best for | Internal agents in Teams and M365 | Custom and customer-facing apps |
| Control over models & retrieval | Managed, less granular | Full control |
| Speed to first version | Days | Weeks |
A reference architecture
- Users interact through Teams or a web app, authenticated with Entra ID.
- The agent receives the request and, based on instructions, decides whether to search knowledge, call a tool or ask for clarification.
- Azure AI Search returns relevant, permission-trimmed content.
- Tools are exposed through API Management with per-agent scopes and rate limits.
- High-impact actions (refunds, record changes, emails to customers) require a human approval step.
- Every step is traced and logged to Azure Monitor for debugging, audit and evaluation.
Security essentials
- Run agents with their own identities and least-privilege permissions.
- Use private endpoints and keep secrets in Azure Key Vault.
- Apply content safety and prompt-injection protection, especially for agents that read external content such as emails or web pages.
- Respect sensitivity labels and data-loss-prevention policies via Microsoft Purview.
From pilot to production
Start with one high-volume workflow — IT help, HR questions, sales preparation — and one team. Measure resolution rate, time saved and user satisfaction against a baseline. Expand tools and audiences only when evaluation shows the agent is reliable. Treat prompts, tools and knowledge sources as versioned assets deployed through CI/CD.
Our Azure AI specialists and agentic AI team build agents on both Copilot Studio and Azure AI Foundry.
Frequently asked questions
Can AI agents built on Azure work inside Microsoft Teams?
Yes. Agents built with Copilot Studio or Azure AI Foundry can be published to Microsoft Teams and other Microsoft 365 experiences, as well as web and custom apps.
How do Azure AI agents connect to business systems?
Through connectors and APIs — for example Microsoft Graph, Dynamics 365, ServiceNow or custom APIs exposed via Azure API Management — and increasingly through Model Context Protocol (MCP) tools.
How do I keep Azure AI agents secure?
Use dedicated identities with least privilege, private networking, Key Vault for secrets, content safety and prompt-injection protection, human approval for high-impact actions, and full tracing.