Microsoft has introduced a new AI governance architecture designed to transition from static policy documentation to active runtime enforcement and continuous observability.
TL;DR
Microsoft's new architecture shifts AI governance from mere policy-making to real-time runtime enforcement and auditing. It utilizes tools like the Foundry AI Gateway to ensure that AI agents and models adhere to security and operational constraints during production.
Microsoft is re-architecting its approach to AI governance to address the complexities of deploying autonomous agents and large language models in production environments. Moving away from traditional, static policy documents, the new architecture focuses on a continuous operational loop consisting of four primary functions: policy definition, runtime control, observability, and audit evidence. This framework aims to ensure that governance requirements are not just documented but actively enforced and verifiable during system operation.
The architecture is organized into nine critical domains, including data governance, model governance, security, identity and access, and a specialized focus on agent governance. To implement these controls, Microsoft leverages its existing ecosystem, integrating Microsoft Foundry with services such as Microsoft Purvi, Entra ID, Defender, and Azure API Management. A central component of this strategy is the AI Gateway within Microsoft Foundry, which serves as a runtime boundary to manage authentication, token limits, quotas, and IP restrictions. This allows for centralized governance of Model Context Protocol (MCP) tools without requiring modifications to the underlying agent code or servers.
A significant portion of the framework is dedicated to 'Agent Governance,' which introduces specific controls for the identity, activity, and workflow checkpoints of autonomous agents. Through the Agent Governance Toolkit and the Agent Control Specification, Microsoft provides mechanisms for intercepting inputs, model calls, and tool executions, even allowing for human-in-the-loop approvals for high-impact actions. Ultimately, this architecture maps the vendor-neutral principles of the NIST AI Risk Management Framework into concrete, actionable platform controls and telemetry, transforming operational data into audit-ready evidence for compliance and incident investigation.