AI Gateway in Rust
A Rust gateway that lets an application use one familiar API while the system selects where the requested model can run. It keeps account, live quota, network path, and service version changes separate, so the application receives the requested model or a clear failure without losing session context.
One familiar OpenAI- or Anthropic-compatible entrypoint for the requested model, instead of a separate integration for every provider.
The gateway selects an account after checking live quota, network path, and service version. An unavailable model is reported, not silently replaced.
The application receives a clear failure. Session context stays separate from the failed condition, and an open stream finishes on the version that started it.
How a request crosses the gateway
No prompts, accounts, credentials, private traffic, or proprietary source are shown.
- Application APIThe application uses an OpenAI- or Anthropic-compatible request instead of a provider-specific integration.
- Provider adapterModels, messages, tools, and streaming events are mapped to the selected provider's protocol.
- Where it can runModel support, account access, quota, and the assigned network path are checked together.
- Session continuationSession context stays intact while quota, network, and service-version failures follow separate recovery paths.
The application gets one entrypoint
The application uses it like an ordinary API: send a familiar request and receive the requested model's response through the same interface. Inside, the gateway chooses the provider and account, translates the request, and keeps the context when conditions change.
Provider-specific formats, account access, and live quota stay inside the Rust gateway instead of spreading across every application integration.
The path of one request
- One application entrypoint: The application sends a familiar OpenAI or Anthropic request without rebuilding its integration for every provider.
- Request translation: Messages, tools, thinking fields, and streaming events are mapped to the provider's actual protocol instead of pretending every API behaves the same way.
- Where the request can run: The requested model, account entitlement, live quota, and assigned network path are checked before work starts. An unavailable model is reported instead of silently replaced.
- The session continues: Session context stays attached across requests and restarts. Tool history stays with the same continuation instead of becoming a new conversation.
- Long response and version change: Elapsed time alone cannot end an active generation. A new service version must complete a real model response before new traffic moves to it, while existing connections finish where they started.