FinOps infrastructure for AI
The LLM gateway that stops the bill
Hard budget enforcement before your API calls go out. Per-identity, team and tag spend tracking with nano-USD precision, forecasting and anomaly detection. All of it running in Rust at sub-millisecond median overhead, with every check switched on.
- Pre-auth budget circuit breaker stops overspend before the request leaves your network
- Integer nano-USD accounting, so there is no rounding drift over billions of requests
- Works with 100+ providers: OpenAI, Anthropic, Gemini, Bedrock, and more
Built for cost control and reliability
Everything you need to enforce budgets, attribute spend and keep an LLM platform dependable at scale.
Hard budget enforcement
A pre-auth circuit breaker rejects the request before it reaches your provider. Caps apply per key, per user, per team and per tag. A project that hits its limit is stopped for everyone on it.
Financial correctness
Costs are integers in nano-USD, never floats. A billion small charges still sum exactly, which is what chargeback and audit actually require.
Built in Rust
Compiled, async, memory-safe, no garbage collector, so there is no pause to explain away in your tail latency.
Cost intelligence
Forecasting and anomaly detection on your own spend history, so you find the runaway agent on day one instead of on the invoice.
100+ providers
OpenAI-compatible on the front, with native adapters for Anthropic, Gemini and AWS Bedrock, including their pricing quirks, tiers, cache reads and reasoning tokens.
Resilience built in
Automatic retry, weighted load balancing and provider failover, so one upstream outage is not your outage.
Observability that fits your stack
Structured JSON logs, Prometheus metrics and OpenTelemetry traces out of the box. Nothing proprietary to adopt.
Extensible without forking
Add providers, middleware and cost logic through Rust or Python plugins against stable traits.
Measured, not claimed
What the gateway actually costs you
Every number below came off one machine, in one run, against the same mocked provider. The rig, the configuration and the handicaps are all stated, so you can argue with them.
Latency OxiGate adds
The same request sent twice: once through OxiGate, once straight to the provider. The difference is what OxiGate costs.
The control row is the identical test with OxiGate removed, so its median is zero by construction. It is what this machine contributes on its own. Subtracting it is the only honest way to attribute latency to a proxy.
Against a stock Python gateway
Same test, same mock, same minute. Only the proxy in the middle differs. Both carry 500 concurrent users. Bars span p50 to p95.
The comparison is stacked against us on purpose. LiteLLM is only forwarding here: no auth, no budget check, nothing written down. OxiGate is doing all three on every request it is timed on, and still finishes the work in a fraction of the time the other spends doing none of it.
What happens when you turn the load up
Median latency hides this. Watch the p95 column instead. That is where load lands first.
| Concurrent users | OxiGate p50 | OxiGate p95 | LiteLLM p50 | LiteLLM p95 |
|---|---|---|---|---|
| 62 | 0.75 | 1.78 | 6.90 | 7.00 |
| 400 | 0.78 | 1.91 | 9.87 | 1120 |
| 500 | 0.78 | 3.51 | 1412 | 1696 |
All figures in milliseconds.
Eight times the load moved OxiGate's median by three hundredths of a millisecond. The detail that matters: at 400 users the box was 70.9% idle, load average 2.02, so the Python gateway's collapse is not contention. A single Python process cannot use the cores sitting idle next to it. Give it four workers and it holds to roughly 300 users before saturating, at about 250 MB of memory per worker.
How this was measured
- Test rig
- One commodity 16-thread server, running both gateways
- Date
- August 2026
- Upstream provider
- Local static mock, so no model latency is in any figure
- Instrumentation
- Each probe sends one identical request through the proxy and direct to the mock, and reports the difference
These are single-machine figures against a mocked upstream, not a claim about your production traffic. Real numbers depend on your provider, your network and your request shape. The harness is in the repository. Run it on your own hardware and tell us if it disagrees.
Reproduce it yourselfBuilt in Rust for speed. Built for control.
How it works
Point your existing OpenAI client at OxiGate. Every request is authorised, priced and recorded on the way through.
RustRunning it takes one command
One container, plus PostgreSQL and Redis. Migrations are compiled into the binary, so an empty database bootstraps itself on first start.
# 1. Get it git clone https://github.com/getoxigate/oxigate.git # 2. Add your provider keys cp .env.example .env # 3. Gateway, PostgreSQL and Redis together docker compose up -d --build
The gateway then listens on localhost:8080. Change your client's base URL to http://localhost:8080/v1 and nothing else in your code has to move.
The questions people actually ask
Including the awkward ones.
Where does my data go?
Nowhere. OxiGate is self-hosted: you run the container, and it talks to your Postgres and your Redis. Provider API keys, prompts and spend records never leave your infrastructure, and the gateway phones no home.
What happens if OxiGate goes down?
It sits in the request path, so plan for it like any other proxy: run more than one instance behind your load balancer. It is a single stateless binary with a health endpoint, so this is ordinary infrastructure work rather than a special case.
Is the free tier crippled?
No. The Community edition is the whole gateway (budget enforcement, all providers, unlimited traffic) under AGPL-3.0. Paid tiers add the analysis layer on top: dashboards, forecasting, anomaly detection, support. Nothing that stops you overspending is behind a paywall.
How hard is it to migrate?
It speaks the OpenAI API, so in the usual case you change a base URL and a key. Native adapters for Anthropic, Gemini and Bedrock are there when you want provider-specific behaviour rather than the compatibility layer.
How do I know the cost numbers are right?
Costs are computed as integers in nano-USD from provider-reported usage, not estimated from character counts. The gateway asks providers to include usage on streamed responses and parses the final chunk. Tiered pricing, cache reads and writes, and reasoning or thinking tokens are all priced separately, because they are billed separately.
Who is behind this?
A small, independent team, not a platform trying to become the place your data lives. The source is public, the benchmark harness ships with it, and you can .