AI Routing Vendor Audit

An AI vendor risk assessment for routing tools is different from a general AI audit. Your routing layer picks which model handles each prompt. If you have not audited the pool, you may not know where your data goes.

Most AI routing tools like OpenRouter and LiteLLM include models hosted in China by default. That means a single API call can send regulated data to a jurisdiction your compliance team has not reviewed.

This guide walks you through a routing-specific AI compliance audit. You will inventory your routing stack, check every model in the pool, verify data residency, and close the gaps before a regulator or client finds them.

Why AI Routing Audits Are Different from General Vendor Audits

A general AI vendor risk assessment asks whether one vendor meets your security bar. A routing audit is harder because one vendor routes to many models, each with its own data handling.

When you call a routing tool, the tool decides which model runs your prompt. That model might be hosted in the US, the EU, or China. The routing tool often picks based on cost or latency, not compliance.

This matters because your data flows to the model provider, not just the routing vendor. A SOC 2 report from the routing layer does not cover the downstream model host.

• One routing vendor can expose you to dozens of model providers
• Cost-based routing may silently pick the cheapest model, which is often a Chinese-hosted one
• Your BAA or DPA covers the routing vendor, not necessarily the model host
• Each model provider has its own data retention, logging, and jurisdiction

Step 1: Inventory Your AI Routing Stack

Start by listing every routing tool, gateway, or proxy your organization uses. This includes official tools and any that developers set up on their own.

Check for OpenRouter, LiteLLM, Portkey, Cloudflare AI Gateway, Vercel AI, Unify, and custom routing code. Search your codebase for API keys and base URLs that point to these services.

Shadow IT is common here. A developer who set up a LiteLLM proxy for a side project may have left it running with a default model pool that includes Chinese-hosted models.

• Search code repos for routing-tool API keys and base URLs
• Check infrastructure logs for outbound calls to routing endpoints
• Interview engineering leads about any routing or gateway tools in use
• Include staging and development environments, not just production

Step 2: Map Every Model in the Routing Pool

For each routing tool you found, list every model it can route to. The default configuration often includes models you never chose.

OpenRouter lists its full model catalog on its website. LiteLLM supports over 100 providers by default. Unless you locked the pool down, any model on the list could receive your data.

Pay special attention to models from DeepSeek, Qwen (Alibaba), Yi (01.AI), and Moonshot AI. These are hosted in China and subject to Chinese data laws.

• Export the full model list from each routing tool's config or dashboard
• Flag every model hosted outside your approved jurisdictions
• Check whether fallback or retry logic can route to unapproved models
• Document which models are in the pool by default versus explicitly added

Step 3: Verify Data Residency for Each Model Provider

For every model in your routing pool, confirm where the provider hosts the inference endpoint and where it stores or logs prompts. This is the data residency question.

US-based providers like OpenAI and Anthropic process data in the US and typically offer a DPA or BAA. China-based providers like DeepSeek process data in China under the Personal Information Protection Law (PIPL) and the Cybersecurity Law.

The distinction matters for ITAR, HIPAA, SOC 2, and any contract that restricts data to specific countries. If your data hits a China-hosted endpoint, you may have a compliance violation even if the routing tool is US-based.

• Check each provider's privacy policy and terms for data processing location
• Confirm whether the provider stores prompts, outputs, or both
• Ask for a Data Processing Agreement (DPA) that names the jurisdiction
• Flag any provider that cannot confirm data residency in writing

Step 4: Check ITAR, HIPAA, and SOC 2 Exposure

Map each model provider against the regulations your organization must follow. The question is simple: does sending data to this provider violate any of your obligations?

ITAR forbids sharing controlled technical data with foreign nationals or foreign-hosted systems. If your routing tool sends an ITAR-controlled prompt to a DeepSeek endpoint in China, that is a potential export violation.

HIPAA requires a Business Associate Agreement with any entity that handles protected health information. No Chinese AI provider currently offers a HIPAA BAA. SOC 2 Type II reports are also uncommon from Chinese providers.

• ITAR: any Chinese-hosted model is a potential export-control violation for controlled data
• HIPAA: confirm a signed BAA exists with every model provider that touches PHI
• SOC 2: verify each provider holds a current SOC 2 Type II report
• Contractual: check client agreements for data-residency restrictions that routing may break

Step 5: Review Prompt Logging and Retention Policies

Check whether the routing tool and each model provider log your prompts and outputs. Logging is useful for debugging but dangerous if it stores sensitive data in an uncontrolled location.

Some routing tools log every request by default for analytics and cost tracking. Some model providers retain prompts for training unless you opt out. Both create data copies you may not have accounted for.

For regulated data, you need a clear answer: who stores what, where, for how long, and under what legal basis. If you cannot get that answer in writing, the model should come out of the pool.

• Check the routing tool's logging settings and disable what you do not need
• Confirm each model provider's data retention and training-data policies
• Ask whether prompts are stored in the same jurisdiction as inference
• Ensure logging complies with your data-minimization obligations

Step 6: Lock Down the Routing Pool

After your audit, restrict the routing pool to only the models your compliance team has approved. Most routing tools let you set an allowlist of models.

In LiteLLM, you can configure the model list in your proxy config file. In OpenRouter, you can specify the exact model in each API call rather than using auto-routing. In Portkey, you can set up gateway configs that restrict the model pool.

Document the approved model list and review it every quarter. New models appear on routing platforms regularly, and a default update could re-introduce a model you previously removed.

• Set an explicit allowlist of approved models in every routing tool
• Disable auto-routing or cost-based routing that can pick unapproved models
• Block fallback to unapproved models in retry and failover configs
• Review the approved list quarterly or whenever the routing tool updates its catalog

When to Bring in Outside Help

A routing audit is straightforward if you have one tool and a small model pool. It gets complex fast when you have multiple routing layers, shadow IT, or regulated data across several frameworks.

If your organization handles ITAR-controlled data, PHI, or client data with contractual residency requirements, an outside review adds a second set of eyes. The cost of missing a routing exposure is usually higher than the audit.

Layer3 Labs runs routing-specific AI audits for companies that use gateway and orchestration tools. We map your routing stack, flag every compliance gap, and hand you a prioritized fix list.

• Multiple routing tools or gateways across teams
• Regulated data (ITAR, HIPAA, SOC 2, client contracts)
• Shadow routing tools set up by individual developers
• Recent incidents or client questions about AI data handling