What Is an API Gateway? API Gateway vs. Load Balancer, Explained
A single entry point for routing, auth, rate limiting, and TLS across microservices — and why it isn't a substitute for a load balancer.

An API gateway is a single, managed entry point that sits in front of a group of backend services (often microservices) and handles the cross-cutting plumbing — routing requests to the right service, authenticating callers, enforcing rate limits, transforming requests and responses, terminating TLS, and logging everything — so individual services don't each have to build that logic themselves. It is not the same thing as a load balancer: a load balancer distributes network or HTTP traffic across a pool of identical servers to spread load, while an API gateway makes API-aware decisions about who is allowed to call what, how often, and in what shape. In most production systems, the two work together, with a load balancer sitting in front of (or behind) the gateway rather than replacing it.
Key facts
- An API gateway handles routing, authentication, rate limiting, request/response transformation, TLS termination and observability for a set of backend APIs or microservices.
- A load balancer distributes traffic across identical server instances at the network (L4) or HTTP (L7) layer; it has no concept of API keys, quotas or payload transformation.
- AWS API Gateway bills per million API calls ($1.00/million for HTTP APIs, $3.50/million for REST APIs, both for the first usage tier) plus a free tier of 1 million calls/month for 12 months.
- The default integration timeout on AWS API Gateway is 29 seconds (range 50 ms–29 s); exceeding it returns an HTTP 504.
- Common options span fully managed (AWS API Gateway, Google Cloud API Gateway, Azure API Management) and open-source/self-hosted (Kong, NGINX, Envoy/Istio ingress).
What is an API gateway?
An API gateway is a server (or managed cloud service) that acts as the single "front door" for a set of backend APIs. Instead of every client calling dozens of individual microservices directly, the client calls the gateway, and the gateway routes each request to the correct backend. Amazon describes its own service as something that lets developers "create, publish, maintain, monitor, and secure REST, HTTP, and WebSocket APIs at any scale," acting as a front door for applications to access data, business logic or functionality from backend services such as workloads on EC2, code running on AWS Lambda, or any web application.
That framing matters because it captures what an API gateway is for: not moving packets, but managing APIs as a product. Azure API Management describes its gateway component the same way — as a facade that accepts API calls and routes them to the appropriate backend, verifies credentials such as API keys, JWTs and certificates, enforces usage quotas and rate limits, optionally transforms requests and responses, and emits logs, metrics and traces.
What problem does an API gateway solve?
The problem shows up as soon as an application grows past a single monolith into several services: mobile apps, partner integrations and internal tools all need to call those services, and each service would otherwise have to reimplement the same concerns — TLS, auth, throttling, logging, versioning — on its own. An API gateway centralizes that work in one layer so individual services can stay focused on business logic. Google Cloud frames this directly: exposing services publicly requires authentication, secure transport, protection from attacks, scaling, monitoring and usage tracking for billing, and without a common interface, developers have to learn each service's protocol and update their apps whenever a backend changes. API Gateway lets you change or move the backend without breaking deployed client apps, because the public-facing API contract stays stable even as the implementation behind it changes.
In a microservices architecture specifically, this also solves a discovery and consistency problem: clients get one stable hostname and one consistent set of conventions (auth headers, error formats, rate-limit headers) no matter how many services sit behind it, or how often those services are split, merged or redeployed. If you're comparing how services talk to each other behind that gateway, it's worth also reading what gRPC is and how it differs from REST, since gateways frequently translate a public REST/HTTP contract into gRPC or other protocols on the backend.
API gateway vs. load balancer: what's the difference?
This is the comparison people search for most, and the short answer is: a load balancer distributes traffic across a pool of servers to keep any one instance from being overwhelmed; an API gateway makes decisions about a request — who sent it, whether they're allowed to, and what shape the data should be in — before or after it ever reaches a backend. NGINX's own documentation describes load balancing as distributing client requests across several application servers using methods like round robin (the default), least-connected (least_conn, which sends the next request to the server with the fewest active connections) and IP hash (ip_hash, which routes a given client to the same server based on a hash of its IP address), with optional per-server weights. None of that logic knows what an "API key" or "rate limit" is — it only knows about servers, connections and response times.
An API gateway operates one layer up. It still needs something to spread load across backend instances (and often delegates that to a load balancer or the cloud provider's own load-balancing layer underneath it), but its distinguishing job is policy enforcement on the request itself: validating a JWT or API key, checking a usage quota, rewriting a path or payload, caching a response, and recording exactly who called what. That's why Azure API Management and AWS API Gateway are usually deployed in front of an existing load-balanced backend rather than instead of one.
| Capability | Load balancer | API gateway |
|---|---|---|
| Primary job | Distribute traffic across identical server instances | Manage, secure and route API traffic to possibly many different backends |
| Layer | Network (L4) or HTTP (L7) | Application/API-aware (L7) |
| Routing logic | Round robin, least-connections, IP hash, weighted | Path/host/header-based routing to distinct services or API versions |
| Authentication | Not typically handled | API keys, JWTs, OAuth2/OIDC, mutual TLS, Lambda/custom authorizers |
| Rate limiting / quotas | No native concept of a "client" or quota | Per-key, per-plan or per-consumer rate limits and quotas |
| Payload transformation | None | Request/response rewriting, format conversion (e.g. XML to JSON) |
| TLS termination | Yes | Yes |
| Typical products | NGINX, HAProxy, AWS/Azure/Google Cloud load balancers | AWS API Gateway, Kong, Azure API Management, Google Cloud API Gateway |
How an API gateway works: routing, auth, rate limiting and transformation
Most API gateways do some version of the same core jobs, even though the exact feature names differ by product:
- Routing — incoming requests are matched to a backend based on path, hostname, HTTP method or header, and forwarded to the right service, Lambda function, container or VM.
- Authentication and authorization — the gateway checks API keys, JSON Web Tokens, OAuth2/OpenID Connect tokens or mutual TLS certificates before a request is allowed through. AWS API Gateway supports IAM policies, Lambda authorizer functions and Amazon Cognito user pools for this; Azure API Management checks API keys plus JWTs and certificates at the gateway layer.
- Rate limiting and quotas — the gateway caps how many requests a given API key, user or IP can make in a given window, protecting backends from being overwhelmed and allowing tiered pricing plans.
- Request/response transformation — the gateway can rewrite paths, headers or bodies, or convert between formats, so the public contract doesn't have to match the backend's native format.
- TLS termination — the gateway (or the proxy underneath it, such as Envoy) handles the HTTPS handshake so backend services don't each need their own certificates.
- Observability — access logs, execution logs, metrics and distributed traces are emitted centrally. AWS API Gateway integrates with CloudWatch and AWS X-Ray for this; Envoy exposes statistics through an admin endpoint and supports third-party distributed tracing.
Envoy's own documentation is a useful illustration of where the "gateway" and "proxy" worlds overlap: Envoy is described as an L7 proxy with a core L3/L4 proxy and a pluggable HTTP filter layer, offering TLS termination, HTTP/1.1, HTTP/2 and HTTP/3 support, attribute-based routing, global rate limiting via an external service, automatic retries, circuit breaking and outlier detection. That's effectively an API gateway's feature list implemented as infrastructure — which is exactly why Envoy is the data plane inside service meshes like Istio and is commonly used as a Kubernetes ingress/API gateway.
Rate limiting and timeout behavior, explained
Rate limiting is how a gateway protects backends from traffic spikes or abuse, and it's usually implemented with a token-bucket or sliding-window algorithm. AWS API Gateway's account-level default is a 10,000 requests-per-second throttle across all HTTP, REST and WebSocket APIs in a region, with burst capacity from a token-bucket algorithm using a maximum bucket of 5,000 requests (lower, 2,500 RPS/1,250 burst, in a handful of newer regions). Kong's Rate Limiting plugin caps HTTP requests over a configurable window — seconds, minutes, hours, days, months or years — and its paid Rate Limiting Advanced plugin adds sliding-window algorithms plus local, cluster or Redis-backed counting for consistent limits across a cluster of Kong nodes; once a caller exceeds the configured limit, Kong (like most gateways) returns an HTTP 429 Too Many Requests.
Timeouts work differently: they bound how long the gateway will wait for a backend integration to respond before giving up and returning an error to the client. On AWS API Gateway, the integration timeout defaults to 29 seconds and is configurable from 50 milliseconds up to 29 seconds for most integration types; exceeding it returns an HTTP 504. AWS announced in June 2024 that Regional and private REST APIs can request an increase beyond the 29-second ceiling, though edge-optimized APIs and the account-level throttle quota can still constrain how high that goes in practice. If your backend is a long-running job rather than a quick request/response call, the usual pattern is to invoke it asynchronously and poll or use a callback, rather than trying to hold the HTTP connection open through the gateway.
Popular API gateway options: AWS, Kong, Google Cloud, Azure, NGINX and Envoy
There is no single standard product; teams pick based on whether they want a managed cloud service or something they run themselves, and on which cloud (if any) they're already using:
- AWS API Gateway — a fully managed service for REST, HTTP and WebSocket APIs, built to pair with AWS Lambda and other AWS backends, with built-in CloudTrail/CloudWatch logging, canary deployments, custom domains and AWS WAF integration.
- Kong Gateway — an open-source gateway built on NGINX, extensible through plugins for auth, rate limiting, transformation and analytics; Kong also sells self-managed Kong Enterprise and a hosted Kong Konnect SaaS tier on top of the open-source core.
- Google Cloud API Gateway — a managed service where you define each API as an OpenAPI 2.0 spec (backend entry point, request/response formats, auth method), deploy it with the gcloud CLI, and front backends such as Cloud Run, Cloud Run functions or App Engine; auth is handled via API keys, Google ID tokens or other Google authentication methods.
- Azure API Management — a hybrid, multicloud API platform made up of a gateway (data plane), a management plane and a developer portal, offered in Classic tiers (Developer/Basic/Standard/Premium), newer V2 tiers (Basic v2/Standard v2/Premium v2) and a serverless Consumption tier that bills per execution; it also supports a self-hosted gateway for on-prem or hybrid deployments.
- NGINX — primarily a reverse proxy and load balancer rather than a full API gateway product, but widely used as the HTTP layer underneath gateways like Kong, or configured directly with routing and `upstream` blocks for simpler API-fronting needs.
- Envoy / Istio ingress — Envoy is a standalone L3/L4/L7 proxy that functions as an edge or API gateway (TLS termination, attribute-based routing, global rate limiting, retries, circuit breaking) and is also the data-plane proxy inside the Istio service mesh, where it's commonly deployed as Kubernetes ingress.
If you're also deciding what sits in front of your gateway globally, it's worth understanding how a CDN works and what it caches — CDNs and API gateways solve different problems but are frequently layered together, with the CDN caching static or cacheable API responses at the edge and the gateway handling everything dynamic behind it.
API gateway pricing models
Pricing generally follows one of two models: pay-per-request (common among cloud-managed gateways) or a fixed subscription/license fee (common for self-hosted or enterprise products).
- AWS API Gateway charges per million calls with no minimum fees: HTTP APIs cost $1.00 per million requests for the first 300 million per month (dropping to $0.90/million above that); REST APIs cost $3.50 per million requests for the first 333 million per month (dropping in tiers above that), plus $0.09/GB for data transfer out; WebSocket APIs charge $1.00 per million messages plus $0.25 per million connection-minutes. New AWS customers get a free tier of 1 million REST, 1 million HTTP and 1 million WebSocket calls per month for 12 months.
- Kong Gateway is free and open source to self-host; Kong's commercial Enterprise and Konnect tiers are licensed/subscription products on top of that core, with pricing provided on request rather than published per-request rates.
- Google Cloud API Gateway and Azure API Management both publish usage- and tier-based pricing on their respective pricing pages, with Azure additionally offering a Consumption tier that bills per execution for serverless-style workloads, alongside fixed-capacity tiers (Basic/Standard/Premium and their V2 equivalents) billed by the hour regardless of traffic volume.
- NGINX (open source) and Envoy (open source, CNCF project) have no licensing cost themselves; you pay only for the compute they run on, though NGINX Plus (a commercial product) adds paid features like active health checks.
When you don't need an API gateway
A gateway adds a network hop, a new piece of infrastructure to operate, and (for managed options) a per-request cost — so it isn't free, architecturally or financially. A small app with a single backend service, a handful of endpoints and no need for per-client rate limits, API keys or request transformation usually doesn't need one: a load balancer (or, for very low traffic, nothing more than the backend's own TLS-terminating web server) is enough. The signal that it's time to add a gateway is usually one of: multiple backend services that need a single public entry point, external or partner developers who need API keys and usage plans, a need to version or deprecate endpoints without breaking existing clients, or compliance/observability requirements that call for centralized logging and throttling. Teams managing API-to-API communication at scale internally (service-to-service rather than client-to-service) often reach for a service mesh data plane like Envoy/Istio instead of, or in addition to, a public-facing API gateway.
What's next
If you're evaluating this for a real system, the practical next steps are: map how many distinct backend services your public API actually needs to front (one service rarely needs a gateway at all); decide whether a managed option (AWS, Google Cloud, Azure) or a self-hosted one (Kong, NGINX, Envoy) fits your existing infrastructure and ops capacity; and read the specific pricing and quota pages for the option you're leaning toward, since throttle limits, timeout ceilings and per-request costs vary meaningfully between products and even between regions on the same provider. For the data layer many of these backends sit on top of, see our explainer on Redis vs. Valkey for in-memory caching, which is a common companion to gateway-level response caching.
Frequently asked questions
What is an API gateway in simple terms?
It's a single entry point that sits in front of one or more backend APIs or microservices and handles routing, authentication, rate limiting, request/response transformation, TLS termination and logging, so individual backend services don't each have to implement that logic themselves.
Is an API gateway the same as a load balancer?
No. A load balancer distributes traffic across identical server instances at the network or HTTP layer, with no concept of API keys or quotas. An API gateway makes API-aware decisions — authentication, rate limiting, transformation — and typically sits in front of (or relies on) a load balancer rather than replacing it.
How much does AWS API Gateway cost?
AWS API Gateway bills per million requests with no minimum fee: $1.00 per million for HTTP APIs and $3.50 per million for REST APIs (both for the first monthly usage tier, with lower per-million rates above that), plus $1.00 per million messages and $0.25 per million connection-minutes for WebSocket APIs. New AWS customers get 1 million free calls per API type per month for 12 months.
What happens when an API gateway request times out?
On AWS API Gateway, the integration timeout defaults to 29 seconds (configurable from 50 milliseconds up to 29 seconds for most integration types, or higher on request for Regional/private REST APIs since June 2024). If the backend doesn't respond in time, API Gateway returns an HTTP 504 error to the client.
Do I need an API gateway for a small app?
Usually not. If you have one backend service, a handful of endpoints, and no need for per-client API keys, rate limits or request transformation, a load balancer (or just your web server's own TLS termination) is enough. A gateway starts paying off once you have multiple backend services, external API consumers, or compliance/observability needs that call for centralized control.
Can I use NGINX or Envoy as an API gateway?
Yes, both are commonly used this way. NGINX is primarily a reverse proxy/load balancer but can be configured for routing and TLS termination for simpler API-fronting needs. Envoy is a full L3/L4/L7 proxy with TLS termination, attribute-based routing, global rate limiting, retries and circuit breaking, and is the data-plane proxy inside service meshes like Istio, where it's frequently deployed as an API gateway or Kubernetes ingress.
Sources
- AWS: Amazon API Gateway Developer Guide — What Is Amazon API Gateway?docs.aws.amazon.com
- AWS: Amazon API Gateway Pricingaws.amazon.com
- AWS: Amazon API Gateway Quotas (throttling and timeout limits)docs.aws.amazon.com
- Kong: Kong Gateway Product Overviewkonghq.com
- Microsoft Learn: Azure API Management — Overview and Key Conceptslearn.microsoft.com
- NGINX Docs: HTTP Load Balancingnginx.org
Felix Moreau writes Pandromeda's software coverage and how-to guides. He covers Windows, macOS and Linux updates, the apps people rely on, emulators and developer tools, and turns official documentation into clear, numbered steps that work on the current version.

