AI Gateway in Rust
A Rust gateway for applications that use several AI providers through one API. The application sends a request in the OpenAI or Anthropic format; the gateway translates it for the target provider and selects an account with available quota.
The application keeps an OpenAI- or Anthropic-compatible interface instead of integrating every provider separately.
The requested model, account access, live quota, and assigned network route are checked together. An unavailable model is reported, not silently replaced.
Quota loss, network-route failure, and service-version change are handled separately. The session keeps its context, and an open stream finishes on the version that started it.
How a request crosses the gateway
No prompts, accounts, credentials, private traffic, or proprietary source are shown.
- Application APIThe application uses an OpenAI- or Anthropic-compatible request instead of a provider-specific integration.
- Provider adapterModels, messages, tools, and streaming events are mapped to the selected provider's protocol.
- Where it can runModel support, account access, quota, and the assigned network path are checked together.
- Session continuationSession context stays intact while quota, network, and service-version failures follow separate recovery paths.
One API instead of several integrations
The application uses it like an ordinary API: send a familiar request and receive the response through the same interface. Inside, the gateway chooses the provider and account, translates the request, and keeps the context when conditions change.
Provider-specific formats, account access, and live quota stay inside the Rust gateway instead of spreading across every application integration.
The path of one request
- One application entrypoint: The application sends a familiar OpenAI or Anthropic request without rebuilding its integration for every provider.
- Request translation: Messages, tools, thinking fields, and streaming events are mapped to the provider's actual protocol instead of pretending every API behaves the same way.
- Where the request can run: The requested model, account entitlement, live quota, and assigned network path are checked before work starts. An unavailable model is reported instead of silently replaced.
- The session continues: Session context stays attached across requests and restarts. Tool history stays with the same continuation instead of becoming a new conversation.
- Long response and version change: Elapsed time alone cannot end an active generation. A new service version must complete a real model response before new traffic moves to it, while existing connections finish where they started.