LLM API Gateways Compared: Portkey vs. LiteLLM vs. Kong AI Gateway vs. Cloudflare AI Gateway

The table reflects what each project's own documentation confirms, not marketing copy about performance or accuracy that cannot be independently verified.

That last column is the one worth reading twice. Three of the four gateways document native PII redaction or content guardrails. Cloudflare's does not, as of its current published documentation, which does not make it a worse product, but it does mean a team choosing it for guardrail coverage specifically would be relying on a feature that is not there yet rather than one that is.

All four sit in the same physical position, between your application and the model provider, but they are built around different mental models of what that position is for.

Portkey is architected as a managed observability and governance layer first. Every request routes through Portkey's gateway (SaaS by default, self-hosted or VPC-deployed for teams that need data to stay in their own environment), which normalizes calls across providers, logs them for cost and latency analysis, and applies configured guardrails and rate limits before or after the call. Its defining trait is that cost governance and evaluation tooling, not just proxying, are first-class citizens of the product.

LiteLLM is architected as a self-hosted proxy you run yourself, most commonly as a Docker container, a Kubernetes deployment, or via Terraform onto AWS or GCP. It exposes an OpenAI-compatible API surface in front of more than a hundred underlying providers, so application code written against the OpenAI SDK can point at your LiteLLM proxy and gain multi-provider routing without a rewrite. It can also be used as a Python library rather than a standalone service. Its core value is protocol normalization and full control, at the cost of you owning the infrastructure.

Kong AI Gateway is architected as an extension of Kong's existing API gateway, not a purpose-built new product. It runs as a set of AI-specific entities (AI Models, AI Model Providers, AI Auth Strategies, AI Consumer Groups) layered onto Kong Gateway's plugin architecture, either self-hosted or with a Konnect-managed control plane and data planes running in your own environment. Its defining trait is that it inherits Kong's existing API management capabilities, rather than being a single-purpose LLM tool, which matters directly to teams already running Kong for non-AI APIs.

Cloudflare AI Gateway is the lightest-weight of the four architecturally: a proxy layer built into Cloudflare's existing edge network, positioned for a one-line integration change in application code. Its documented feature set centers on analytics, caching, rate limiting, and automatic retry or fallback across providers, rather than deep content inspection or identity federation. It is architected as an operational convenience layer for teams already inside the Cloudflare ecosystem, not as a standalone AI security control plane.

Portkey offers the broadest deployment spread on paper: a SaaS control plane most teams start on, a self-hosted or open source deployment option for teams that want the gateway inside their own network, and an Enterprise private cloud or VPC-managed tier for teams with stricter data residency requirements. Which tier of guardrails and access control you get depends on which of those you choose, since RBAC and SSO are gated to the Production and Enterprise tiers respectively.

LiteLLM is self-hosted only, by design. You run the proxy yourself via Docker, Kubernetes and Helm, or the provided Terraform modules for AWS and GCP, and you own the uptime, scaling, and patching of that deployment. There is no LiteLLM-managed SaaS offering; the commercial Enterprise license adds features and support, not a hosted control plane.

Kong AI Gateway supports both models genuinely. Konnect gives you a managed control plane with data plane nodes running wherever you choose, self-hosted, cloud, or Kubernetes, with licensing handled automatically. Fully self-hosted Kong Gateway with AI Gateway plugins is also supported for teams that want no dependency on Kong's cloud control plane at all, managed through Kong's decK configuration tooling.

Cloudflare AI Gateway is SaaS only, and specifically only available as part of Cloudflare's own network. There is no self-hosted variant, which is a non-starter for any team with a requirement that LLM traffic never transit a third party's edge, but a non-issue for a team already routing traffic through Cloudflare for everything else.

All four gateways support the core promise of routing to more than one LLM provider, but the depth of that support differs.

Portkey integrates with the major hosted providers (OpenAI, Anthropic, and others) plus partner guardrail and evaluation tools at the Production tier and above, and adds data lake export and custom guardrail hooks at the Enterprise tier. Its integration surface leans toward observability and evaluation tooling rather than infrastructure breadth.

LiteLLM's headline integration claim is breadth: over a hundred provider and model integrations behind one OpenAI-compatible interface, which is the single biggest reason teams adopt it specifically to avoid rewriting application code every time they add or swap a provider. Multi-tenant cost tracking and spend management per project or user is built into the proxy itself, not an add-on.

Kong AI Gateway documents automatic failover across OpenAI, Anthropic, Azure, Amazon Bedrock, Google Gemini, and Vercel, with routing and retry handled through its AI Model Provider entities, plus semantic caching to reduce repeated calls to any of them. Because it extends Kong Gateway, it also inherits Kong's existing integrations with identity providers and its broader API management ecosystem, which is a real advantage for a team that already has Kong wired into its identity stack.

Cloudflare AI Gateway documents request retry and model fallback in case of a provider error as a native feature, along with caching to serve repeat requests without hitting the provider at all. Its integration story is narrower than the other three: it is built for teams already inside Cloudflare's network rather than for federating with external identity providers or third-party guardrail vendors.

Cloudflare AI Gateway is the lowest-operational-lift option of the four for a team already using Cloudflare, since there is no infrastructure to run and the documented integration is a one-line code change. The tradeoff is that you inherit whatever guardrail and identity gaps its current feature set has, rather than being able to configure your way around them.

Portkey's SaaS tier is similarly low-lift to start, since Portkey runs the control plane and you configure policy through its interface; the self-hosted and Enterprise VPC options trade that convenience for infrastructure you run yourself, in exchange for keeping request and prompt data inside your own environment.

Kong AI Gateway's operational effort depends heavily on which deployment model you pick. Konnect-managed control planes reduce the operational burden to running data plane nodes; fully self-hosted Kong Gateway asks more, and a team with no existing Kong footprint is taking on both the AI Gateway plugins and a full API gateway's operational surface at once, not just the AI-specific piece.

LiteLLM asks the most of your team by design, since it is self-hosted only. Running it well in production means operating the proxy's own uptime, scaling it under real traffic, and patching it yourself, including patching promptly when a security advisory lands, which is not a hypothetical for this specific project (see the security incidents below). That is a real, ongoing cost even though the core software carries no license fee.

For a gateway specifically, the security-relevant question is not just which boxes a features page checks, but whether the product's own operational history backs that up.

Portkey documents per-key budgets and rate limits at every paid tier, a PII Anonymizer, and partner guardrail integrations from the Production tier upward, with RBAC gated to Production and SSO to Enterprise. It also publishes SOC 2 Type 2, ISO 27001, GDPR, and HIPAA compliance claims and offers a BAA for regulated customers, which is meaningful specifically because it is a hosted SaaS product handling your prompts.

LiteLLM's virtual keys provide per-key rate limiting and budget enforcement natively, and per-project configuration extends to guardrails and caching, but SSO and full RBAC are reserved for the separate commercial Enterprise license, a distinction worth confirming against your actual compliance requirements before you assume the open source proxy alone covers them. LiteLLM also has a real, documented security track record worth factoring in precisely because it is self-hosted software you are responsible for patching: CVE-2026-42271, a command injection flaw in LiteLLM's MCP test endpoints with a CVSS score of 8.8, was confirmed exploited in the wild and added to CISA's Known Exploited Vulnerabilities catalog, and has been shown to chain into unauthenticated remote code execution. Separately, LiteLLM was the target of a March 2026 supply chain compromise in which threat actor TeamPCP breached a third-party scanning vendor and used stolen tokens to push malicious LiteLLM releases, exposing an estimated 434,000 CI/CD credentials downstream. Neither incident means LiteLLM is unsafe to run; it means a self-hosted gateway is also a self-hosted attack surface, and patch cadence and release verification need to be part of your operational plan, not an afterthought.

Kong AI Gateway documents the most content-inspection depth of the four: PII redaction before a prompt reaches the model, filtering for disallowed topics or keywords, and detection aimed at jailbreak and prompt-injection attempts phrased in natural language, plus integration with cloud vendor guardrails from AWS, Azure, and GCP. Its access control layer (OAuth2, consumer groups, token quotas) inherits from Kong's mature identity integrations. Worth noting plainly: none of the four gateways in this comparison, including Kong's, is a substitute for a dedicated runtime prompt injection defense when that is your primary threat model. If prompt injection detection accuracy against adversarial input is the deciding factor rather than a secondary feature bolted onto a routing layer, our runtime prompt injection firewall comparison evaluates purpose-built products for exactly that job.

Cloudflare AI Gateway's documented security-relevant capabilities, as of this writing, are logging and analytics, caching, rate limiting, and provider retry and fallback. Its public documentation does not describe a PII redaction feature or a prompt-injection guardrail, which is a meaningful gap if either is a requirement rather than a nice-to-have, and worth re-checking against the live docs before you commit, since gateway feature sets in this category are moving quickly.

Portkey publishes real, current pricing: a free Developer tier capped at 10,000 monthly logs and explicitly described as not suitable for production, a Production tier at $49 per month including 100,000 logs with $9 charged per additional 100,000, and custom Enterprise pricing for teams needing 10 million-plus logs, VPC deployment, SSO, and custom guardrail hooks, according to Portkey's own pricing page.

LiteLLM's core proxy has no license cost to state honestly, since it is released under the MIT license with no paid tier for the open source gateway itself. The separate commercial Enterprise license, covering SSO, RBAC, and dedicated support, is priced through a sales contact process rather than a published rate card, so budget for that conversation rather than assuming a number.

Kong AI Gateway has no explicit standalone pricing published either. Kong Gateway's open source tier carries no license fee, and Konnect-managed deployments handle licensing automatically as part of Kong's existing commercial relationship, which typically means AI Gateway is priced as an extension of whatever Kong or Konnect agreement a team already has rather than as a separate line item, per Kong's own documentation.

Cloudflare AI Gateway is stated to be available on all Cloudflare plans with no separate tier restriction and no additional AI Gateway charge documented beyond what you already pay Cloudflare, though your actual model provider costs (OpenAI, Anthropic, or whichever provider you route to) are entirely separate and unaffected by using the gateway.

Portkey fits teams that want cost governance, observability, and guardrails as first-class product features rather than an afterthought, with a real compliance posture (SOC 2, ISO 27001, HIPAA/GDPR readiness) for regulated data. Its limitation is that the meaningful security controls, RBAC and SSO specifically, sit behind paid tiers, and the free tier is explicitly not meant for production use.

LiteLLM fits teams that want a genuinely open, self-hosted proxy with the broadest provider coverage of the four and full control over where it runs. Its limitation is twofold: you own all the operational burden of running it well, and its documented security incidents in 2026 are a real reminder that self-hosting shifts patch responsibility onto you, with SSO and RBAC gated behind a separate commercial license if your compliance requirements need them.

Kong AI Gateway fits teams already running Kong for API management who want to extend an existing investment rather than add a new vendor, and it has the deepest documented content-guardrail feature set (PII redaction, prompt-injection filtering) of the four. Its limitation is that adopting it cold, with no existing Kong footprint, means taking on a full API gateway's operational surface, not a lightweight point solution, and pricing is not transparently published.

Cloudflare AI Gateway fits teams already standardized on Cloudflare who want caching, rate limiting, and provider failover with essentially no new infrastructure or cost. Its limitation is real: no self-hosted option, and no documented PII redaction or prompt-injection guardrail feature as of this writing, so it should not be selected as your primary content-safety control.

None of these four products is a universal answer. The right one depends on your existing infrastructure, your compliance requirements, and which specific gap, cost visibility, provider lock-in, content safety, or operational simplicity, you are actually trying to close.

If your organization has one or two services calling a single LLM provider, a small number of engineers who all know where the API key lives, and no regulatory requirement to prove per-user cost attribution or content filtering, adopting a dedicated LLM gateway today is probably premature. A well-configured API key with provider-side rate limits, a simple wrapper function that logs token usage, and a documented runbook for provider outages can cover the real risk at that scale without adopting a new piece of infrastructure to operate.

A dedicated gateway earns its cost once you have genuine LLM sprawl: multiple teams calling multiple providers with no central visibility into spend, a compliance requirement to redact PII before it reaches a third-party model, or enough request volume that a provider outage without automatic fallback becomes a real incident rather than an inconvenience. If your actual concern is adversarial prompt input specifically, jailbreaks, injection payloads embedded in retrieved content, rather than routing and cost, start with a purpose-built runtime prompt injection firewall instead of expecting a routing-focused gateway's bolt-on guardrail to carry that weight alone.

Whichever of these four you evaluate, or if you evaluate a different gateway entirely, run the same PoC checklist rather than trusting a vendor's feature page alone.