LLM Gateway

Coming soon. How the LLM Gateway will let your code call AI models with one key per workspace.

Coming soon

The LLM Gateway isn't available yet. In the Client Portal, Build → LLM Gateway shows Coming soon, and you can't create a workspace key there. This page describes how the gateway works so you can plan for it; we'll announce it in the changelog when it's ready.

The LLM Gateway will give your code access to AI models from several providers through one OpenAI-compatible API and one key per workspace. You choose a model by name; the gateway routes the request and records usage.

Models

Model name	Use it for	Runs on
`gf-chat-default`	Chat and text	OpenAI GPT-4o mini
`gf-chat-pro`	Harder chat and reasoning	Anthropic Claude Sonnet 4.5
`gf-code-default`	Code	Anthropic Claude Sonnet 4.5
`gf-embed-default`	Embeddings, for search	OpenAI text-embedding-3-small
`gf-vision-default`	Images and text	OpenAI GPT-4o

Once the gateway is available, the models you can use will be listed under Build → LLM Gateway, and the list can include other models. The names above are the ones Groundfloor plans to provide by default and may change before launch.

Get a key

Once the gateway is available:

Create the workspace key

Go to Build → LLM Gateway and mint the workspace key. You need the workspace Admin or Owner role, or Owner or Admin in the organization. The key is shown once. You can't mint one while the workspace is frozen or the organization is suspended.

Store it in Secrets

Save it straight away in Secrets as LITELLM_API_KEY.

Rotate or revoke it when you need to

Minting a new key does not revoke the old one: the old key keeps working until you revoke it. To rotate, mint the new key, switch your code to it, then revoke the old one. To revoke a leaked key, use DELETE /v1/workspaces/{workspace_id}/llm/virtual-key (LLM Gateway API); it needs the same roles as minting, and it still works while the workspace is frozen.

Call a model

This example shows the planned shape of a call. Call the gateway from server-side code, such as a Coderunner: the key must never reach a browser. Read the key from Secrets at runtime, then use any OpenAI-compatible client:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: `${process.env.LLM_GATEWAY_URL}/v1`,
  apiKey: await getSecret("LITELLM_API_KEY"), // see Configuration and secrets
});

const reply = await client.chat.completions.create({
  model: "gf-chat-default",
  messages: [{ role: "user", content: "Hello" }],
});

Ask the Groundfloor team for your gateway's address to use as LLM_GATEWAY_URL; it isn't listed in these docs yet.

getSecret is the helper in Configuration and secrets. A Shell app calls the gateway through a backend.

See your usage

The LLM Gateway page is planned to show the last 7 days of usage: compute units, tokens and the number of requests. The LLM Gateway API is planned to return up to 30 days.

Gateway usage is planned to feed the AI part of your organization's Billing card, which is an estimate. It does not count against a quota: the LLM compute units row on the Quota card isn't connected yet and shows 0.

Next steps

Secrets and backends — AI features in a Shell app.
LLM Gateway API — models, usage and keys from code.

LLM Gateway

On this page