AI Gateway review

AI Gateway is a mostly free proxy for LLM API calls, with logging, analytics, caching, rate limiting, and provider fallbacks. Most teams building with LLMs end up making half of this themselves. Getting it from a base URL change is useful, as is seeing OpenAI, Anthropic, Workers AI, and other providers in one analytics view.

The proxy behaves, caching can pay for the integration effort on repeated prompts, and the rate limits and fallbacks do what they claim. The drawbacks are an extra hop in a latency-sensitive path, occasional delays in supporting new provider features, and logs that help with debugging but do not replace proper observability.

I would avoid Unified Billing. Cloudflare has a long history of billing problems, including surprise invoices, opaque line items, and slow corrections. I don’t want that system between me and an inference bill. Thankfully, it is optional. Bring your own provider keys and the rest of AI Gateway is easy to put in front of existing LLM usage.