Executive summary
Microsoft Foundry is the enterprise‑grade platform that enables organizations to construct, deploy, and operate AI‑driven agents at scale. The platform is delivered through a web‑based portal where each Foundry project acts as a logical container for all assets related to a specific line of business or functional area. The portal offers a set of generally available (GA) capabilities that cover the most common production scenarios, such as model discovery, prompt‑agent creation, fine‑tuning, and built‑in guardrails for model usage. In addition, the platform includes a set of preview features—monitoring extensions, virtual‑network integration for certain workloads, and the Voice Live real‑time speech capability—that are not yet covered by a formal service‑level agreement and must be enabled only after a formal risk‑assessment and explicit approval.
This deep‑dive guide explains how to design a secure landing‑zone, define an identity hierarchy, standardize networking, and implement governance controls that allow an enterprise to run large‑language‑model (LLM) agents reliably and responsibly. The focus is on practical steps that a platform team can take to bring the platform from a proof‑of‑concept environment into a production‑ready service.
Introduction
Enterprises that wish to leverage LLM‑driven agents quickly encounter two overlapping challenges. First, the underlying models are powerful but opaque, and they need to be wrapped in a controlled runtime that enforces policy, tracks usage, and isolates workloads. Second, the regulatory and compliance landscape demands strict auditability, role‑based access, and network segregation for any data that traverses the system.
Microsoft Foundry addresses both challenges by providing a unified portal and a set of services that sit on top of Azure’s core security and observability stack. A Foundry project groups together models, agents, toolboxes, and deployment settings, allowing administrators to apply a consistent set of policies across all assets in the project. The portal clearly separates GA features—such as model catalog browsing, prompt‑agent definition, fine‑tuning pipelines, and model guardrails—from preview capabilities, which include instant‑access model previews, voice‑enabled agents (Voice Live), and advanced networking options. Understanding this separation is critical when planning production workloads, because only GA features are covered by the platform’s SLA and are safe for mission‑critical use.
Technical fundamentals
Foundry’s technical model revolves around three persistent agent types, each with distinct runtime requirements:
- Prompt agents – These agents consist solely of a prompt configuration (instructions, chosen model, and declared tool set). The runtime interprets the prompt at request time; there is no custom code container involved.
- Voice‑based prompt agents – An extension of the prompt agent that adds real‑time speech handling via the Voice Live preview service. Audio is streamed to Azure Speech services, transcribed, processed by the prompt, and then synthesized back to the user.
- Hosted agents – Fully custom agents packaged as a Docker container or zip file. Hosted agents run on the Microsoft Agent Framework or a compatible SDK, giving developers the freedom to embed proprietary business logic, state management, or specialized tool integrations.
Authentication follows two parallel models. Most API endpoints accept a static API key that grants blanket permission to the service. However, any operation that manipulates evaluations, datasets, Content Understanding, agents, or workflows requires Microsoft Entra ID authentication. Entra ID enables fine‑grained role‑based access control (RBAC) and integrates with the organization’s existing identity governance processes. For production workloads, the recommended approach is to enforce Entra ID authentication exclusively, reserving API keys for isolated test environments where rapid iteration is needed.
Architecture and components
The core Foundry service is composed of several interconnected components, each of which is documented in the public service overview. The most important pieces are:
- Agent Runtime – Executes prompt and hosted agents, orchestrates tool calls, maintains conversation state, and handles lifecycle events such as version promotion and deprecation.
- Toolboxes – Managed endpoints that aggregate common capabilities (web search, file search, code interpreter, and custom functions). Toolboxes provide a single, secure surface for agents to invoke external services.
- Model catalog – A curated list of GA models (including the latest GPT‑4‑style offering, Llama‑based models, and DeepSeek) together with preview models that are available for experimentation.
- Observability – Built‑in tracing, metrics, and evaluation pipelines that feed into Azure Monitor and Application Insights, giving operators end‑to‑end visibility into request latency, token usage, and error rates.
- Agent optimizer – A preview‑only service that analyzes agent definitions and suggests improvements to prompts, tool descriptions, and model selection.
- Identity & security – Entra ID integration, RBAC, content filters, and optional virtual‑network isolation to keep traffic within a customer‑controlled perimeter.
- Publishing – Stable endpoint versioning, sharing mechanisms for Teams, Microsoft Copilot, and registration in the Entra Agent Registry for enterprise‑wide discoverability.
The diagram below illustrates a typical production flow for a hosted agent, showing how the container image is stored, how the runtime interacts with the model catalog and toolboxes, and how observability data is collected.
flowchart TD
A[Developer] -->|Push container image| B[Azure Container Registry]
B --> C[Foundry Agent Service]
C --> D[Model Catalog]
C --> E[Toolbox Endpoint]
C --> F[Application Insights]
C --> G[Entra ID RBAC]
D --> H[Inference]
E --> I[Tool Execution]
H --> J[Agent Runtime]
I --> J
J --> K[Response to Client]
Detailed implementation approach
The following step‑by‑step plan translates the architectural concepts into concrete actions that a platform engineering team can execute:
- Create a landing zone – Allocate a dedicated Azure subscription or resource group that will host all Foundry projects for the organization. Apply Azure Policy definitions to enforce mandatory tagging, geographic location constraints, and, where required, virtual‑network isolation.
- Define identity hierarchy – In Entra ID, create groups that reflect the responsibilities within the AI lifecycle: Platform Admins (full control of all projects), Project Owners (manage a specific project’s assets), Agent Developers (create and update agents), and Security Auditors (read‑only access to audit logs). Assign built‑in roles like Foundry Project Admin and create custom RBAC actions for granular permissions such as “read audit log” or “update model‑catalog binding.”
- Standardize networking – For any project created after 25 June 2026, enable the GA virtual‑network integration option. This forces all inbound and outbound traffic for the Agent Runtime to traverse a customer‑controlled virtual network, preventing exposure to the public internet. For legacy projects, document the limitation that they will continue to use the public registry and plan a migration path.
- Configure toolboxes – Register a shared toolbox that contains only approved internal services, such as an enterprise search API, a code‑interpreter microservice, or a proprietary data‑access layer. Apply content‑filter policies at the toolbox level to block disallowed data categories.
- Establish CI/CD pipelines – Leverage Azure Developer CLI or the Foundry SDK to automate the promotion of agent definitions and container images from source control to the Foundry project endpoint. Incorporate compliance checks that validate required tags, run static‑code analysis on container images, and enforce that only GA models are referenced.
- Enable observability – Connect the Agent Runtime’s telemetry streams to Azure Monitor and Application Insights. Turn on GA tracing for prompt and hosted agents; defer tracing for workflow agents until they are fully retired. Configure alerts for anomalous latency spikes or sudden increases in token consumption.
- Govern deployments – Institute a policy that production agents may reference only GA models. Use the Model Catalog UI to verify that the selected model is available in the target region. Any temporary use of preview models must be recorded in a change‑request system and approved by the compliance office.
Architecture and data‑flow
Data enters the Foundry system through either the Responses API (programmatic calls) or via a hosted‑agent endpoint that receives HTTP requests from downstream applications. For a hosted agent, the flow proceeds as follows:
- Client request reaches the Agent Service endpoint.
- The runtime authenticates the caller with Entra ID and validates the caller’s RBAC permissions.
- The runtime resolves the agent’s configuration and selects the appropriate model from the Model Catalog.
- If the agent invokes a tool, the runtime forwards the request to the Toolbox endpoint and receives the tool’s result.
- All interactions—including model selections, tool calls, and response payloads—are streamed to Application Insights and, when enabled, to the immutable audit log.
- The final response is assembled and returned to the client. If the agent has been published to Microsoft Copilot, the response may be wrapped in the Copilot harness before delivery.
This data‑flow ensures that every request is auditable, that model usage is centrally governed, and that external tool calls are mediated through a controlled toolbox.
Security and governance
Foundry incorporates several GA security controls that align with enterprise governance requirements:
- RBAC – Role assignments are managed through Entra ID, allowing the organization to reuse existing groups and policies.
- Audit logs – Immutable records capture every change to model selection, agent configuration, and deployment state, supporting forensic analysis and regulatory reporting.
- Compliance views – A preview feature that maps model usage to regulatory frameworks; while useful for assessment, it is not yet GA and should not be relied upon for production compliance.
- Virtual‑network isolation – GA for hosted agents created after the cutoff date, ensuring that all traffic stays inside the customer’s private network.
Because API keys provide only a binary allow/deny capability, the platform team should enforce Entra ID authentication for any workload that is visible to end users or processes sensitive data. When preview networking features are used (for example, tracing over a virtual network), a documented risk‑assessment ticket must accompany the deployment, and the ticket should be reviewed by the security governance board before the feature is enabled in production.
Enterprise use cases
| Use case | Agent type | Key Foundry features leveraged | GA / Preview |
|---|
| Customer support bot | Prompt agent | Model flow, tool integration, RBAC enforcement | GA |
| Real‑time voice assistant | Voice‑based prompt agent | Voice Live streaming, Azure Speech integration, monitoring hooks | Preview |
| Internal knowledge retrieval | Hosted agent | Custom toolbox registration, fine‑tuning, full tracing | GA |
| Automated compliance reviewer | Prompt agent | Model guardrails, red‑team evaluation, audit logs | GA |
These scenarios illustrate how different agent types can be combined with specific Foundry capabilities to address a range of business needs, from simple query‑answering bots to complex, voice‑enabled assistants that operate on internal data.
Benefits and limitations
Benefits
- Unified model catalog – Centralizes version control and simplifies governance of model usage across the enterprise.
- Entra‑ID‑backed RBAC – Leverages the organization’s existing identity infrastructure, reducing administrative overhead.
- GA tracing for prompt and hosted agents – Provides end‑to‑end visibility into request latency, token consumption, and tool‑call latency.
- Publishing integration – Agents can be surfaced directly in Teams, Microsoft Copilot, or the Entra Agent Registry, accelerating adoption across business units.
Limitations
- Network isolation is not universal – Some preview features still rely on public endpoints, which may conflict with strict data‑egress policies.
- Workflow agents are being retired – The scheduled retirement on 1 December 2026 forces teams to migrate to the Microsoft Agent Framework, requiring code changes.
- Monitoring and alerting remain in preview – Lack of SLA guarantees means that production teams must implement their own fallback monitoring for critical workloads.
- Private container registry support is limited to newer projects – Projects created before the 25 June 2026 cutoff cannot use private Azure Container Registry integration without migration.
Operational considerations
- Lifecycle management – The Project Details view in the portal displays quota consumption, expiration dates, and usage trends. Operators should regularly review this view to prevent unexpected throttling.
- Versioning – Before an agent is shared beyond its development team, promote it to a stable version. Stable versions are immutable and can be referenced by downstream consumers without fear of accidental breakage.
- Incident response – Combine audit‑log entries with Application Insights traces to reconstruct the full request path when investigating anomalies. Set up alerts for repeated authentication failures or spikes in tool‑call latency.
- Regional compliance – Model availability varies by Azure region. Verify that the selected model is deployed in the same region as the consuming application to avoid cross‑region data movement.
Best practices
- Post‑June‑2026 projects – Keep all production agents in projects created after the virtual‑network cutoff to benefit from private registry support and GA isolation.
- Restrict API‑key usage – Limit API keys to sandbox or development environments; enforce Entra ID authentication for any workload that processes real customer data.
- Adopt the Microsoft Agent Framework – Use the framework for all new hosted agents to future‑proof against the upcoming workflow‑agent retirement.
- Regular RBAC hygiene – Conduct quarterly audits of Entra ID groups, removing stale members and tightening permissions where possible.
- Document preview usage – Whenever a preview feature is enabled, capture the justification, risk assessment, and approval in a change‑control system that is auditable by compliance.
Analysis
The available documentation makes it clear that the Foundry portal’s GA status is limited to core scenarios. Model discovery, prompt‑agent creation, fine‑tuning, and model guardrails are fully supported under the platform’s SLA. In contrast, many networking, monitoring, and voice capabilities remain in preview and therefore lack the same reliability guarantees. This separation forces organizations to adopt a disciplined split between production workloads (restricted to GA) and experimental workloads (allowed to use preview features only under explicit governance).
The imminent retirement of workflow agents on 1 December 2026 is another pivotal point. Teams that still rely on workflow‑based logic must allocate effort to migrate to the Microsoft Agent Framework, which offers a more robust, GA‑supported development model. The migration path includes re‑packaging existing logic as a hosted container, re‑defining tool calls within a toolbox, and updating deployment pipelines to target the new runtime.
Overall, the platform provides a solid foundation for scaling AI agents, but success hinges on rigorous identity management, careful handling of preview features, and proactive migration planning for deprecated components.
What the sources do not settle
- The precise performance SLA for the Agent Runtime when hosted agents are run at large scale remains unspecified in the GA documentation.
- Cost attribution details for private Azure Container Registry usage after the 25 June 2026 cutoff are not provided, leaving budgeting teams without clear guidance.
- The impact of the partial GA status for Foundry IQ knowledge—where the API is GA but portal access is still preview—on downstream applications is not quantified.
- Guidance on migrating classic‑portal projects that rely on a hub‑based architecture to the newer project model is missing, creating uncertainty for legacy migrations.
Conclusion
Microsoft Foundry delivers a mature, generally available platform for building enterprise AI agents. Its strengths lie in centralized model management, Entra‑ID‑driven RBAC, immutable audit logging, and native publishing pathways to Teams and Copilot. By establishing dedicated landing zones, enforcing Entra ID authentication, and limiting production workloads to GA features, organizations can scale agent deployments while satisfying compliance requirements.
Preview capabilities—such as Voice Live, advanced networking, and the Agent Optimizer—provide valuable innovation but must be gated behind explicit risk assessments and documented approvals. As workflow agents are retired, the Microsoft Agent Framework will become the sole supported path for custom logic, ensuring a consistent development experience going forward.
References
- What is Microsoft Foundry Agent Service? | Microsoft
- Microsoft Foundry portal general availability overview | Microsoft
These published CloudExchangers articles are further desk reading. They are not sources for the technical claims above.