Why enterprise AI needs a cross-platform governance layer

Agentic workflows are coming, and they are inherently horizontal. They don't live neatly within the four walls of your CRM, ERP, or collaboration suite. An agent booking a complex sales trip might need to touch your CRM for context, your HR system for travel policy, your finance software for expensing, and a dozen external services for flights and hotels.

None of these systems of record are a natural home for agent interoperability and governance.

This presents a foundational problem. Right now, every major SaaS player is building impressive agentic capabilities: Salesforce with Agentforce, Microsoft with Copilot Studio, ServiceNow with their AI agents. These platforms are solving real problems and delivering genuine value. But as enterprises adopt multiple agent platforms simultaneously, a new challenge emerges: how do you maintain consistent governance, security, and observability across this diverse ecosystem?

The outcome I'm primarily concerned about is enterprises struggling with fragmented oversight. Great agents that excel within their domains, but limited visibility and control at the enterprise level.

Another thing I’m thinking about is ecosystem diffusion and what happens when the “enterprise people” show up. H/T Elad Gil for making this diagram back in January:

Instead of leading with a specific solution, let's engage in a thought exercise. If we were to design the ideal enterprise solution for this new agentic reality from first principles, using a vendor-agnostic and customer-centric lens, what would it look like? From experience as a core contributor to Kubernetes on multi-vendor/multi-cloud resource allocation and isolation and seeing first hand how the Kubernetes ecosystem facilitated solving similar orchestration and industry collaboration challenges (more on that below,) I believe there are five critical questions we need to answer.

A Thought Exercise: Enterprise Agent Architecture from "1st Principles"

If an enterprise is going to deploy not ten, but hundreds or thousands of agents, we need to answer some critical questions that existing SaaS platforms simply aren't architected to handle.

1. How do we govern what is essentially a new class of non-human identity?

Agents act on behalf of people, and their actions must ultimately be attributable to a person who is accountable for the result. This is a core value that I personally believe in, and one that I've seen echoed by execs in many large enterprises. This means agent identity needs to be governed as carefully as human identity. We need seamless integration and centralized, fine-grained authorization policies. And every action must be observable and auditable in real-time (subject to kill-switches.)

Without this foundation, the enterprise is left unable to answer basic questions like "which agent authorized that $50K purchase order?" or "why did our marketing agent suddenly start accessing HR data?"

2. How do we control the financial blast radius?

Governance isn't just about access; it's about cost. Agents consume resources—API calls, compute cycles, and third-party services that all have real-world price tags. How do we prevent a single autonomous agent from draining a team's quarterly budget overnight?

We need the ability to set and enforce budget caps on agent activities. More importantly, how do we attribute these costs? For this to scale, finance departments will demand clear chargeback mechanisms, ensuring that the cost of an agent's operations flows back to the business unit that benefits from its work — marketing, sales, or R&D.

3. How do we prevent a (fragmented) configuration state explosion?

Today's model is brittle: every client manually configured to talk to specific servers or tools. When a new capability comes online, how do clients discover it? When a tool is deprecated, how do you revoke access cleanly?

1:1 config relationships make centralized access governance impossible and creates a sprawling, unmanageable web of connections. And in a classic fortunately/unfortunately situation these connections quickly become load-bearing for enterprise operations. Requiring a full-time operations or even an internal development team just to wrangle agent deployments, agent health monitoring, and security patches isn't a scalable answer. It's a sure sign that the architecture has not yet been properly factored.

4. How do we build a multi-agent system that is secure by design?

For business-critical AI deployments, we need to guarantee a minimum compliance bar that allows enterprises to responsibly manage risk, regardless of the underlying agent frameworks they choose to deploy.

The current standard for deploying many agentic tools involves running scripts that feel uncomfortably like `curl | bash`. This is a non-starter in an enterprise environment. Furthermore, many agent protocols are still maturing. We need a way to enforce robust, modern security standards (think complex OAuth flows) as a service, even when underlying tools only support simple bearer tokens and innovation continues to progress at the interoperability layer.

More fundamentally, we need seamless integration with existing Identity Providers like Okta or Microsoft Entra to extend identity-aware access policies to agent actions. Administrators should be able to apply the same conditional access rules, risk-based authentication, and fine-grained permissions to agents that they already use for human users. The governance system itself should appear as an enterprise application within the IdP to bootstrap administrative access.

5. How do we manage stateful, multi-step workflows?

Simple request-response interactions are table stakes. True agentic workflows are stateful and complex. A task might be initiated by one agent, handed off to another, and require data from multiple tools over hours or days. This demands persistent, stateful orchestration that can manage tasks, capabilities, and collaboration patterns.

These agentic workflows can become extremely complex with compounding loss buckets across multiple systems and decision points, making full observability absolutely critical. A2A's stateful tasks are a step in the right direction, but in an ideal deployment there would be zero dark requests to MCPs or agents. This comprehensive visibility is required to assess what's working and what's not, what investments are panning out, and what opportunities exist for operational improvements across the business.

These five challenges circumscribe the need for a unified governance layer that doesn't exist today.

A Hypothetical Solution: The Agent Service Bus (ASB)

Now, let's imagine a system that could elegantly meet this challenge. I like TLAs (Three Letter Acronyms) just as much as anyone else working in enterprise AI today so let's call it the Agent Service Bus, or ASB for short.

What it isn't: the system I'm imagining isn't another agent. It not an alternative to A2A for agent interoperability (however the Jury is still out on whether A2A is “it”.) It's not an alternative to MCP for Agent::Resource interactions.

What it is: ASB is the missing infrastructure governance layer that sits in between these. It's an open-source implementation of the A2A API server and persistent state, and it's a secure reverse MCP proxy and identity gateway that handles authentication, authorization, and secure communication for all agent traffic.

It should implement good distributed systems patterns like circuit breakers, particularly for MCP server traffic. MCPs tend to be backed by APIs or internal resources that depend on specific config. APIs fail a lot. Misconfiguration is extremely common, in general. With a system that's changing frequently and comprised of hundreds to thousands of actors, the expected MTBF implies a need to operate in a partially degraded state basically all the time.

I think this system should externalize unified logging, traces, and metrics using open standards like OpenTelemetry and Prometheus, giving operators the observability, auditing, and reporting they need using the tools they already have in place. This system should also be a central hub for financial governance, where policies for agent budget caps and chargebacks could be defined and enforced. Most of these capabilities could be designed as "batteries-included but swappable" interfaces, creating opportunities for vendors to innovate and differentiate without creating chaos and paralysis for the teams deploying and operating the agent infrastructure.

Since I'm imagining the system I want to use, it's open source and it could be self-hosted or purchased from a cloud vendor as a fully managed solution. This is a two-way decision, since either deployment model conforms to a standard behavioral and API specification.

To me, the beauty of this modularity is that agents can peer with enterprise resources at any of the protocol layers, with significant benefits over the alternative at each layer.

Layer 1: Inference via an OpenAI API compatible proxy (similar to Salesforce's Models API)

Layer 2: Consume enterprise-managed MCP servers in a compliant and fully observable manner. I’d expect to see policy proliferate to lock down MCP access and centralize traffic through a trusted layer. It’s a good idea and making implementation easier will help guide more organizations to adopt this pattern.

Layer 3: Interact with other enterprise agents via A2A

Layer 4: Integrate directly with ASB, useful for governance agents that need access to metadata about resource consumption, usage patterns, outcomes, and global optimization across the enterprise agent population.

Enterprises get complete visibility and control over their production AI deployments. Complex authentication is handled centrally. Administrators can use familiar tools to manage access and financial policies. And you can mine the interaction patterns across the business for powerful insights about where to invest next.

Thanks for reading! Subscribe for free to receive new posts and support my work.

A Kubernetes Analogy

Why should a system like this be an open-source project with broad industry participation?

For the same reason Kubernetes became the standard for container orchestration. Before Kubernetes and the CNCF, every cloud provider had excellent but incompatible ways of managing compute resources. The breakthrough wasn't replacing these systems. It was creating a common governance layer that allowed enterprises to operate consistently across multiple platforms.

Kubernetes created a shared foundation that made it easier for both vendors and customers to innovate. It was subsequently embraced and enhanced by every major vendor, creating tremendous value for the entire ecosystem.

Another key benefit of the containerization wave also applies here. Containerization finally created a clean API and deployment abstraction that enabled platform teams and application development teams to move at their own pace, through self-serve lightweight platforms-as-a-service built on top of open source orchestrators. Right now there's a real problem in enterprise, where lines of business are ready to go ahead with deeper investments into the agents that deliver real ROI today, and centralized AI committees are left fielding requests from each business unit, put in the unenviable position of trying to set standards, stay compliant, allow innovation, uphold fiduciary responsibility, manage risk, prioritize the value of potential benefits, and deduplicate similar use cases across the business. Oh… and also understand the unique benefits of thousands of new vendors, and compare those against the capabilities and roadmap of their existing vendors. Mind you, this is often nobody's only job!

We are at a similar inflection point with agentic AI. Companies like Salesforce, Microsoft, and ServiceNow are building powerful agent platforms that solve real problems. Those solutions are co-deployed with agents and AI-enabled applications from other vendors, low-code agents built on Zapier or Replit, and custom agents built in-house. What's missing is the shared governance layer that allows enterprises to operate these platforms together seamlessly.

"A CNCF for agents" would enable the major platform vendors to focus on what they do best while providing enterprises with the unified oversight they need. It's a lot of work to coordinate agendas, ideas, and technical contributions from organizations that are partially-aligned at best, and sometimes in direct competition with each other. But this isn't about replacing existing agent platforms; it's about making them work better together, so that the pie gets larger faster. That’s great for vendors and consumers alike, and allows larger organizations to maintain a high pace of business transformation without abandoning their commitments to shareholders to do so in a principled and responsible way.