Governing Claude on Google Vertex AI with MuleSoft Model Proxy: an OpenAI-Compatible Adapter Behind Omni Gateway Deliverable: A thin translation service (a Mule app or a small container) that exposes POST /v1/chat/completions in OpenAI shape and calls Claude on Vertex AI in Anthropic-native shape, deployed behind a self-hosted Omni Gateway Model Proxy. Callers present a MuleSoft client credential, the gateway enforces it and injects the upstream key, the adapter reaches Vertex with no key file at all, and Claude on Vertex becomes a governed model like any other provider in the proxy. Objective: Enterprises that standardized on Google Cloud often have to consume Claude through Vertex AI, sometimes under a zero-data-retention agreement, and much of their agentic spend (coding harnesses especially) runs there. As of November 2026, Model Proxy's provider catalog speaks the OpenAI Chat Completions wire format, and covers Anthropic direct and Anthropic on Bedrock, but not Anthropic on Vertex. Vertex's own OpenAI-compatible endpoint serves Gemini only. The result is that the model carrying the most traffic is the one the AI gateway can't govern, and an AI gateway evaluation can fail on "one endpoint across providers" for that reason alone. This blueprint closes the gap as an engineered interim path, until native support arrives, without giving any caller a Google credential. What's Getting Built Model Proxy (Omni Gateway, self-hosted, Connected Mode) — the governed front door; OpenAI-typed provider, full LLM policy stack · POST {gateway}/{proxy-path}/chat/completions Vertex-Claude adapter (translation service, Mule app or container) — OpenAI ⇄ Anthropic Messages, Opus allowlist and remap · POST /v1/chat/completions (JSON or SSE), GET /v1/models, GET /healthz Vertex AI publisher endpoint (target) — POST /v1/projects/{project}/locations/global/publishers/anthropic/models/{model}:rawPredict and :streamRawPredict Prerequisites A Google Cloud project with Vertex AI enabled and the Claude models you will allow enabled in Model Garden. Check per-minute token quota before the first test: a new project can show a quota of zero in a region. A GKE cluster (Autopilot works) where the gateway and adapter can run side by side, with Workload Identity enabled. If org policy forces private nodes, add Cloud NAT so the gateway can reach the Anypoint control plane. An Anypoint environment where you can create a provider-typed Model Proxy (the Enhanced Experience in API Manager) and a gateway registration for a self-hosted Omni Gateway. The person creating it needs environment-level permissions; a 403 on Model Proxy APIs is more often a user permission than a missing org entitlement. A Google service account with roles/aiplatform.user, bound to the adapter's Kubernetes service account. No key file is ever created. Your data-governance position on each Claude model. Some Claude models on Vertex return 403 until Anthropic data sharing is turned on at the project level; others don't need it. If you have a zero-data-retention requirement, allowlist only the models that don't need the toggle. Choose your path Mule app adapter — Choose this when the MuleSoft team will own the adapter long term: the translation is two DataWeave transforms plus an HTTP request, it versions in Exchange, deploys to CloudHub 2.0 or Runtime Fabric, and uses the same CI/CD and monitoring as the rest of the estate. It stays keyless to Google only on Runtime Fabric on GKE (Workload Identity) or with Workload Identity Federation; on CloudHub 2.0 with a cached token, you also own token refresh. Streaming translation is harder in Mule; the version behind this blueprint returns full completions only. Container adapter (about 300 lines of Python, FastAPI) — Choose this for the fastest stand-up and streaming support from day one. It runs as two replicas next to the gateway and authenticates to Vertex with Workload Identity. The cost is an open-source runtime your platform team has to patch. Both expose the same contract, so swapping one for the other, or later for native provider support, is a change to the Model Proxy upstream that callers never see. Architecture 1. Pin the Vertex contract the adapter must speak Claude on Vertex is not OpenAI-shaped. The model goes in the URL path, not the body. The body is the Anthropic Messages schema with "anthropic_version": "vertex-2023-10-16" and a required max_tokens. Auth is an OAuth bearer token plus an x-goog-user-project header. Use location=global. Regional endpoints in the environment behind this blueprint returned 429 (per-minute token quota of zero on a fresh project) or 400 (model not servable in that region), while global served immediately. Prove the endpoint with one direct rawPredict call before writing any adapter code. If that call fails, nothing above it will work, and the error is clearer here than three hops later. 2. Build the translation, and decide what it refuses to do Request: lift every system message into Anthropic's top-level system field, keep only user and assistant turns, flatten list-style content to its text parts, and take max_tokens from max_tokens or max_completion_tokens (default 1024 when the caller sends neither). Map stop to stop_sequences. Strip temperature and top_p by default. OpenAI clients send them unasked, and current Claude 5.x models reject them as deprecated. Make forwarding a configuration flag for older models. Response: concatenate only text blocks, drop thinking blocks, map stop_reason (end_turn and stop_sequence to stop, max_tokens to length, tool_use to tool_calls), and return usage as prompt_tokens, completion_tokens and total_tokens. That usage block is what the gateway's metering policy reads. Streaming: send stream: true to :streamRawPredict, emit an initial assistant-role chunk, translate each content_block_delta text delta to a chat.completion.chunk, emit the mapped finish reason from message_delta, and end with data: [DONE]. Model allowlist: resolve the requested model against an allowlist and remap anything else to the default. An OpenAI client that asks for gpt-4o gets the approved Claude model, never a model outside your data-governance position. This is a remap, not a refusal: an unknown model never reaches Vertex, but the caller gets a different model than it asked for. Return the served model in the response (model field and an x-model-remapped: true header), and offer a strict mode that answers 400 for non-allowlisted models when callers must know. Upstream errors: return Vertex's status and body to the caller (non-streaming), or one error chunk followed by [DONE] (streaming). Never turn an upstream failure into a 200 with empty content. Out of scope, and documented as such: tool and function calling, images, and surfacing thinking. Say so in the API description so no one assumes them. 3. Run the adapter keyless Bind the adapter's Kubernetes service account to the Google service account through Workload Identity, and get tokens from Application Default Credentials. No Google key file exists anywhere, so there's nothing to leak or rotate. Inbound, the adapter accepts a bearer key read from a Kubernetes Secret at runtime. Only the gateway holds it. Expose the adapter as a ClusterIP Service only. The gateway reaches it over in-cluster DNS, and nothing about it is public. Run two replicas with readiness and liveness probes on /healthz, as a non-root user. 4. Put a provider-typed Model Proxy in front of it Deploy Omni Gateway on the same cluster in Connected Mode. It registers outbound to the control plane over a websocket, so you don't open any inbound ports. Create the Model Proxy in the Enhanced Experience as an OpenAI provider whose base URL is the adapter's in-cluster /v1 URL. A generic passthrough created from the CLI deploys and routes, but it gets no llm-proxy-core metering policy and reports a null provider, so token usage is never computed. The provider-typed proxy auto-applies the LLM policy stack. In the run behind this blueprint that was cors, dataweave-headers-transformation, client-id-enforcement, llm-proxy-core, a key-lookup policy, model-based-routing, a token rate limit and openai-transcoding-policy. Set the inbound listener (port and path) before deploying. If you don't, the deploy fails with an empty proxy URI. Create a client application and an approved contract for each consumer. client-id-enforcement checks credentials against the instance's contracts, so with no contracts every caller is refused, including correct ones. 5. Separate the three credentials Caller to gateway: a MuleSoft-issued client_id/client_secret per consumer. You can rotate or revoke it without touching anything else. Gateway to adapter: the adapter key, held by the gateway and injected upstream by a header-transformation policy that overwrites authorization. Callers never see it. Adapter to Vertex: no key at all, only Workload Identity. That split is the answer to "configure the provider without sharing its key", and to "no shared service key on any path". 6. Security and compliance Data residency and retention: the allowlist enforces your zero-data-retention position in code. Change it only after a data-governance review, not as a runtime switch. Transport: TLS from caller to gateway, in-cluster traffic from gateway to adapter, and TLS to Vertex. Neither the adapter nor the gateway is exposed outside the cluster except through the listener you choose. Logging: log the model, finish reason, token counts and correlation ID. Do not log prompts or completions at INFO; they can contain anything your users paste. Ownership: this is customer-owned infrastructure on an interim path. Name an owner for model-version drift (new Claude IDs have to be added to the allowlist) and for retiring the adapter when native provider support ships. Sample payload — governed completion through the gateway { "id": "chatcmpl-{uuid}", "object": "chat.completion", "created": 1790709707, "model": "claude-opus-5-5", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "OK" }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 29, "completion_tokens": 4, "total_tokens": 33 } } The caller asked for gpt-4o with a valid client credential. The allowlist remapped it to claude-opus-5-5, and the gateway tagged the response x-llm-proxy-llm-provider: openai and x-llm-proxy-llm-model: claude-opus-5-5. Deterministic checks No credential returns 401 from the gateway with WWW-Authenticate: Client-ID-Enforcement. A bad credential returns 401. A streaming request with no credential returns 401. An unknown path returns 404. A valid credential returns 200 with a real completion, finish_reason: stop, and a usage block whose total_tokens equals prompt plus completion tokens. A valid credential with stream: true returns 200 and SSE chat.completion.chunk events ending in data: [DONE]. A non-allowlisted model ID returns 200 from the allowlisted model, never from the one requested, and the response says it was remapped. In strict mode it returns 400. Direct to the adapter: no bearer key returns 401, a malformed body returns a non-2xx status, and a system message changes the persona of the reply. max_tokens: 8 returns finish_reason: length, which proves the mapping. Thinking tokens count against max_tokens. The response carries the gateway's provider and model headers, which proves the metering policy ran. MUnit for a Mule adapter: one test per transform (request lifting, response mapping) against a real captured Vertex response, plus a live profile that calls Vertex for real. Testing and validation Script the battery above and run it with the client credential passed through environment variables. A blank environment variable sends an empty client_id header and produces a misleading "Client ID is not present" 401. The edge case that breaks naive builds is max_tokens. Claude 5.x emits thinking blocks that count against it, and the adapter drops them, so a small budget returns an empty string with finish_reason: length. Use at least 256 in tests and default to 1024. Score the result against the evaluation's gate, not just "it works." For an AI gateway proof of value, the row this closes is "multiple providers through one endpoint, open standards, no proprietary SDK." Also report what isn't closed yet; the metering publish in Troubleshooting is the usual one. Phased rollout Phase 1 — Connect Prove one direct rawPredict call on global, then run the adapter locally against it, and confirm non-streaming and streaming replies in OpenAI shape. Phase 2 — Map and validate Deploy the adapter and gateway to GKE, create the provider-typed Model Proxy and one consumer contract, apply key injection, and run the full battery through the gateway. Point one real harness or agent at the proxy. Phase 3 — Production Move to the customer's own cluster and Anypoint environment, add a contract per consuming team, set token rate limits, and record the retirement trigger: when native Vertex support lands, repoint the proxy upstream and delete the adapter. Success metrics Share of Claude traffic that passes through the gateway instead of direct Vertex calls. Consumers holding a shared Google or model key (target zero). Tokens per consumer and model, visible in LLM Insights once publishing works. On a self-hosted Connected-mode gateway this can stay at zero (see the metering entry in Troubleshooting); until that's fixed, measure from the gateway response headers and adapter logs. Added latency on the governed path. In the run behind this blueprint, a governed 200 took about 1.2 s end to end. Troubleshooting Model Proxy APIs return 403 and the org shows the LLM proxy flag off — Likely cause: user permissions, or the classic UI, not a missing entitlement. Fix: grant environment admin to the person configuring, and switch to the Enhanced Experience before you raise an entitlement request. Every credential returns 401, including the right one — Likely cause: the instance has no approved contracts (they can be lost when the instance is edited or repointed), or a self-hosted gateway hasn't re-synced its client cache. Fix: recreate the contract, then restart the gateway pod. A redeploy alone doesn't always refresh client applications. Completions return an empty string with finish_reason: length — Likely cause: max_tokens too small to cover thinking. Fix: raise it to at least 256. Vertex returns 400 "temperature is deprecated for this model" — Likely cause: sampling parameters forwarded to a Claude 5.x model. Fix: strip them by default. 429 or 400 from a regional endpoint — Likely cause: regional quota of zero or the model isn't servable there. Fix: use location=global, or request a regional quota increase. One Claude model returns 403 while another works — Likely cause: that model requires Anthropic data sharing to be enabled on the project. Fix: keep it off the allowlist if you have a zero-data-retention requirement, or enable data sharing after review. The metering policy computes usage, but LLM Insights stays at zero — Likely cause: a self-hosted Connected-Mode gateway failing to authenticate to the analytics ingest (logs show performing login request … BadArgument. Skipping update. and Could not publish metrics - HTTP status code: 401). This is a gateway-to-control-plane analytics problem, not a Model Proxy configuration problem. Fix: open a support case for the ingest auth; meanwhile, report token usage from the gateway response headers and adapter logs. After a node scale-down, the proxy port returns 503 or an empty reply while the pod reports ready — Likely cause: the new gateway pod built its listener before the upstream cluster was warm and rejected it (unknown cluster in the log), and the listener isn't retried. Fix: restart the gateway deployment until the log is clean and the port answers 401 without a credential. For demos, avoid node churn. The upstream returns 401 after about an hour on CloudHub 2.0 — Likely cause: the Mule adapter's cached Google token expired. Fix: run on Runtime Fabric on GKE with Workload Identity, or use Workload Identity Federation with automatic refresh.