Developer tools

Self-hosted API gateway for LLM CLIs

Tollgate Keys · Self-hosted key vault and spend caps for your team's coding CLIs

WatchingWorth validatingDraft scoreEmerging market · since 2023 · AI estimate

75PWAS score

Confidence

Based on 9 of 9 factors

Overview

AIThis description was written by a model from public signals. The facts behind it are listed under Evidence.

Problem: Teams and individual developers who use multiple LLM providers (OpenAI, Claude, Gemini, Codex) through command-line tools lack a centralized way to manage API access, monitor usage, control costs, and maintain audit logs across these disparate services.
Solution concept: A self-hosted gateway that accepts CLI requests to any LLM endpoint format and routes them transparently to the underlying provider, while providing centralized request logging, spend tracking, usage quotas, and multi-tenant isolation through a web console—enabling teams to audit usage, control costs, and manage access without vendor lock-in.
Who has it: Individual developers and technical teams using AI-powered coding CLIs who need control over API access and spend management.
How they handle it today: Today, teams integrate directly with each LLM provider's API, managing credentials, quotas, and logs separately for each service, or use cloud-hosted gateways without data sovereignty or customization control.
Why now: The proliferation of LLM APIs (OpenAI, Anthropic, Google) and their integration into developer tooling creates fragmentation; rising costs of API calls and enterprise demand for cost control and compliance create urgency; and open-source AI gateway projects show there is market awareness and demand for this control layer.
How it could charge: Subscription licensing based on usage volume (requests per month), number of tenants or users, or deployment size; or a freemium model with self-hosting free but premium features (advanced analytics, compliance reporting, support) paid.

Verdict

Computed from the factor scores below, never written by a model. It decides nothing on its own: only the gate makes an opportunity a build candidate.

Score breakdown

A model sorted each factor into a rubric bucket; code turned buckets into numbers. Unknown factors are left out of the score and lower confidence instead.

Market validation

Confidence 88 · 4 evidence · Multiple evidence items confirm clear, established demand: teams explicitly need centralized LLM API management (E1, E6), existing market for self-hosted gateways confirmed (E2), and the space shows substantial adoption evidenced by a major competitor (Portkey) managing >$500K daily AI spend across 24,000 enterprises (E10).

Revenue evidence

Confidence 88 · 3 evidence · Portkey, a direct competitor in the LLM API gateway space, has raised $15M+ Series A (E9) and manages >$500K in daily AI spend for 24,000 enterprises (E10), proving a proven subscription market; LiteLLM offers enterprise pricing tiers (E12).

Growth

Confidence 95 · 3 evidence · Two independent signals show breakout growth: CLIProxyAPI achieved 57.1k GitHub stars (E11), and another repository in the space gained 1033 stars in 226 days plus 50969 stars in 1084 days (E3, E7), indicating sustained, rapid adoption.

Problem severity

Confidence 70 · 1 evidence · Managing separate credentials, quotas, and audit logs across multiple LLM providers creates recurring friction that costs time and money; cost control and usage monitoring are identified as key needs (E4), pointing to ongoing financial impact rather than blocking work entirely.

Economic proximity

Confidence 30 · Judged from the description · The product sits directly on API spend management—controlling costs, tracking expenditure, and enforcing quotas—making it part of the financial transaction flow between teams and LLM providers.

Distribution

Confidence 30 · Judged from the description · A self-hosted gateway serving multiple team members and tenants requires collaboration to be useful and valuable; centralized access control and audit logs imply multi-user workflows where the tool's value compounds with adoption.

Buildability

Confidence 30 · Judged from the description · A self-hosted API gateway requires integrating multiple third-party LLM provider APIs, building request routing, logging, and quota enforcement—substantial but achievable with a standard backend stack plus well-documented provider SDKs and off-the-shelf components.

Differentiation

Confidence 70 · 2 evidence · Self-hosting with no vendor lock-in and unified multi-provider management addresses a clear gap that users name (E1, E6) and that cloud-hosted competitors struggle to match without compromising data sovereignty or customization—a credible structural gap.

Retention and timing

Confidence 30 · Judged from the description · API gateway usage is a recurring, embedded workflow—teams must route all LLM requests through it daily; timing is strong because the LLM API market is growing rapidly and cost control is becoming critical as spend scales across organizations.

Risk penalties

Depends on one platformThe product depends on third-party LLM provider APIs (OpenAI, Claude, Gemini, Codex) and their continued availability and terms of service; however, this is mitigated by supporting multiple providers, reducing exposure to any single vendor's decisions.−2
IP or trademark riskThe product must route requests to branded APIs (OpenAI, Claude, Anthropic, Google) and the concept leans on their branding and ecosystems; however, it is a neutral proxy service, not misrepresenting affiliation.−2

Score history

45 · confidence 41 · config v1First score · 11 Oct 2026
49 · confidence 47 · config v1Evidence changed · 29 Oct 2026
75 · confidence 59 · config v1Evidence changed · 31 Oct 2026

Evidence · 12 items

Market validation

Teams use multiple LLM providers and coding CLIs and need centralized management
Public claim
Self-hosted AI gateway for coding CLIs — one OpenAI/Claude/Gemini/Codex-compatible endpoint, with a multi-tenant web console, request logs, and spend quotas.
GitHubRetrieved 11 Oct 2026
Existing market for self-hosted API gateways and LLM proxies
Public claim
Topics: ai-gateway, anthropic, api-gateway, claude-code, gemini-cli, go, llm, llm-proxy, multi-tenant, openai, openai-codex, self-hosted
GitHubRetrieved 11 Oct 2026
Organizations need to manage APIs for multiple LLM providers with unified access control, usage tracking and cost accounting.
Public claim
Self-hosted LLM gateway with OpenAI-, Claude- and Gemini-compatible APIs, model routing, access control, usage tracking and cost accounting.
GitHubRetrieved 29 Oct 2026
The project has substantial adoption and community interest.
AI inference
The model stated this without a quote found in the source.
GitHubRetrieved 29 Oct 2026
This is an active market with ongoing development needs.
AI inference
The model stated this without a quote found in the source.
GitHubRetrieved 29 Oct 2026
CLIProxyAPI (open-source CLI proxy): 57.1k GitHub stars (2026-10)
Public claim
57.1k stars
github.comRetrieved 31 Oct 2026

Revenue evidence

Portkey: 24,000 enterprises, >$500K daily AI spend managed (2026-02)
Public claim
manages more than $500K in daily AI spending at 24,000 enterprises.
inc42.comRetrieved 31 Oct 2026
LiteLLM: Open source $0 self-hosted; Enterprise annual pricing by request capacity, not per token (2026-10)
Public claim
priced to your gateway's scale, never per token.
litellm.aiRetrieved 31 Oct 2026
Portkey: $15M Series A led by Elevation Capital; total above $18M (2026-02)
Public claim
has raised $15 Mn (nearly ₹136.6 Cr) in its Series A funding round led by Elevation Capital.
inc42.comRetrieved 31 Oct 2026

Growth

GitHub repository gained 1033 stars in 226 days
Verified
GitHubRetrieved 11 Oct 2026
GitHub repository gained 50969 stars in 1084 days
Verified
GitHubRetrieved 29 Oct 2026

Differentiation

Cost control and usage monitoring are key needs
Public claim
Self-hosted AI gateway for coding CLIs — one OpenAI/Claude/Gemini/Codex-compatible endpoint, with a multi-tenant web console, request logs, and spend quotas.
GitHubRetrieved 11 Oct 2026

Distribution

AIWritten by a model on 31 Oct 2026 from the description and the evidence. Estimates and proposals, not verified facts.

Search: Developers search for specific problems such as tracking Claude Code or Codex CLI spend across a team, or routing a coding CLI through one key. Setup guides for each CLI, with the exact environment variables and base-URL settings, match those searches.
Collaboration: One team lead deploys the gateway and gives each developer a personal virtual key. Each developer has to point their CLI at the gateway, so the whole team uses it, and its budget limits apply to everyone.
Embedded: It sits in the CLI's config as a base-URL override and in the team's infrastructure as a container or Helm chart. Once it is in a shared dev-environment template or dotfiles repo, new hires pick it up automatically.
Native: An open-source self-hosted core published on GitHub, where the evidence shows this category gathers stars, gives it visibility. Forks and issues from people deploying it then reach other developers who are evaluating gateways.

Build

AIWritten by a model on 31 Oct 2026 from the description and the evidence. Estimates and proposals, not verified facts.

Suggested first version: A single-binary or single-container gateway for a small engineering team (about 5 to 30 developers) using Claude Code, Codex CLI and Gemini CLI. It exposes endpoints compatible with OpenAI, Anthropic and Gemini formats and forwards requests to the provider using keys stored on the server. Each developer gets a virtual key. A small web console shows requests, token counts and estimated spend per user and model, and enforces monthly budget caps. It deliberately leaves out translating requests between providers, load balancing and failover, SSO, prompt caching, guardrails and compliance reports.
Core features: Pass-through proxy endpoints compatible with the OpenAI, Anthropic Messages and Gemini request formats, including streaming
Virtual per-user API keys mapped to provider keys stored only on the server
Request log with user, model, token counts, latency and status; storing prompt bodies is optional
Spend estimation from a configurable per-model price table
Per-user and per-team monthly budget caps and rate limits, enforced at the proxy
Web console for managing keys, viewing usage dashboards and exporting logs as CSV
Copy-paste setup snippets for each supported coding CLI
Docker image and docker-compose file with an embedded database option
Outside services it would depend on: OpenAI API
Anthropic Claude API
Google Gemini API
Provider pricing data, maintained by hand as a configurable table
Architecture sketch: CLI clients send requests to the gateway using a virtual key. The proxy layer authenticates the key, checks the budget and rate limit, swaps in the real provider key, and streams the request to the provider and the response back. A usage extractor reads token counts from the response or stream and writes a log row to a relational database. A small admin API and web console read from that database for dashboards and key management. Budget counters are kept in memory or in the database and checked on each request.
Estimate: 25 working days · complexity medium
Running cost: Users self-host it, so PWAS's own running cost is mostly a demo instance and a license server. For users, the main costs are log storage, which grows with request volume when prompt bodies are kept, and their own LLM spend, which passes through to the providers.

Differentiation and risks

AIWritten by a model on 31 Oct 2026 from the description and the evidence. Estimates and proposals, not verified facts.

How it could differ: Built specifically for coding CLIs: tested configurations for Claude Code, Codex CLI and Gemini CLI, plus attribution of spend per developer and per repository, rather than a general-purpose LLM proxy.
Provider keys never leave the team's own infrastructure, which addresses the complaint in the material that third-party gateways require sharing API keys.
Minimal operational footprint: a single container with an embedded database, aimed at teams too small to run a full gateway stack.
Pricing that is not tied to volume for small teams, in contrast to the lock-in and pricing constraints the material attributes to third-party gateways.
Logging that is privacy-aware by default: metadata only, with prompt capture switched on per tenant, so the audit need is met without storing source code sent to the models.
Risks: CommercialThe space is already crowded with free options. The material shows LiteLLM offering a $0 self-hosted open-source version, CLIProxyAPI at 57.1k stars, and Portkey reporting 24,000 enterprises, which leaves little room for a paid new entrant.

Self-hosted API gateway for LLM CLIs

Overview

Verdict

Build this business

Score breakdown

Risk penalties

Score history

Validation gate

Evidence · 12 items

Market validation

Revenue evidence

Growth

Differentiation

Market

Existing products

Distribution

Build

Differentiation and risks

Legal reconstruction check

Community