Built-in models

One bearer token, concrete model ids, an OpenAI-compatible URL or llm.complete from @superjolt/sdk, server-side tool dispatch. Rates and billing live on the pricing page.

POST /v1/llm/complete is a Superjolt-managed proxy that routes every call through one managed gateway to Anthropic, OpenAI, and Google. Devs call it via @superjolt/sdk, and the agent never writes a tool-dispatch layer. Usage is metered per token against your project balance — rates and billing are on the pricing page; this page covers how the endpoint works.

Why managed over BYO

• One balance. LLM usage rolls into the same Superjolt balance as compute and email — no separate provider bill, and the same monthlyBudgetCents cap applies.
• No keys anywhere. Your environment token is the only credential. There are no per-provider API keys — the upstream credentials live on Superjolt’s side, never in your app or ours.
• One tool surface. Pass SDK function names ('billing.refund', 'email.reply') in tools; the gateway resolves them, presents JSON schemas to the model, dispatches the calls back into your project’s services, and returns the final text.

Call shape

import { llm } from '@superjolt/sdk';

const r = await llm.complete({
  model: 'gpt-4o-mini', // a concrete model id — see the menu below
  system: 'You are customer support for MyApp. Be concise.',
  messages: [{ role: 'user', content: incoming.body }],
  tools: ['billing.refund', 'billing.getCustomer', 'email.reply'],
});

// r.content        — final assistant text after all tool turns
// r.model          — the concrete model that served the request
// r.requestedModel — the model id you asked for
// r.stopReason     — 'end_turn' | 'max_tokens' | … ('max_tokens' = truncated)
// r.toolCalls      — dispatcher trace: { name, args, result, isError }[]
// r.usage          — { inputTokens, outputTokens, costMicroCents,
//                      billedMicroCents, allowanceFundedMicroCents,
//                      freeAllowanceRemainingMicroCents }

The SDK reads SUPERJOLT_API_URL + SUPERJOLT_ENV_TOKEN from env by default; pass baseUrl / token explicitly if you need to override.

Inference is never gated by environment. Every call runs real inference, not a canned stub, whichever environment the token belongs to. What the environment does decide is where any tool-dispatch side-effects land: a Development-environment token’s billing.refund / email.reply hit that environment’s services (its Stripe posture, its email sender), and a Production token hits Production’s.

OpenAI-compatible endpoint — use any AI library

Every VM also boots with SUPERJOLT_LLM_URL, an OpenAI-compatible base URL. Point a stock OpenAI client (or LangChain, LlamaIndex, anything that speaks the OpenAI chat-completions shape) at it with the injected environment token — no Superjolt-specific code:

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: process.env.SUPERJOLT_LLM_URL,        // injected at boot
  apiKey:  process.env.SUPERJOLT_ENV_TOKEN,  // injected at boot
});

const r = await client.chat.completions.create({
  model: 'gpt-4o-mini',
  messages: [{ role: 'user', content: 'summarise this' }],
});

Supported subset: chat completions (model, messages, max_tokens). Streaming and client-side function/tool calling are rejected with a clear 400, not silently ignored — for platform tools use the SDK’s llm.complete (above), which runs the server-side tool loop.

Model menu

Pass a concrete model id (there are no aliases). Cheap models stretch your balance furthest in dev; reach for a frontier model only where quality matters:

llm_list_models        # (agent)  — or  llm.models()  (SDK)  — or  GET /v1/llm/models

returns every model with its provider, per-MTok input/output cost (USD, already including Superjolt margin), context window, and whether it supports tool calling. The same menu drives providerForModel(model) so model→provider routing is data, not branching code. The catalog lives in packages/pricing/src/catalog.ts (PRICING.llm.models) — publishing or repricing a model is a straight PR.

Tools — names, not schemas

The tools field takes string names that map to public SDK functions. The gateway:

1. Resolves each name against a server-side registry (one descriptor per SDK function).
2. Builds the provider-shaped tool schema from the descriptor’s zod schema.
3. On a tool_use response, parses + validates args, executes the registered handler in-process under the same environment token (so in the token’s environment), and feeds the result back to the model.
4. Loops until the model returns final text (cap: 16 tool turns).

You only write the prompt and the list of tool names. There’s no JSON-Schema authoring, no tool-result echoing, no Anthropic/OpenAI/Google translation layer in your repo.

Audit: every llm.complete writes one audit_events row with model, tokens, cost (and the allowance-vs-billed split), prompt hash, and the list of tools invoked. Sensitive tool calls (anything money-moving or email-sending) audit a second row with the prompt hash + arguments — the post-hoc forensics trail when an LLM does something weird.

Budgets and budget events

The balance-funded portion of each call inserts a type='llm' billing event and runs the same budget-threshold check the compute meter does (allowance-funded usage never touches the cap). When billed spend crosses 50%, 80%, or 100% of monthlyBudgetCents it additionally emits a domain-specific event:

• llm.budget_warning — at 50% / 80%.
• llm.budget_exceeded — at 100%. With hardCapEnabled, every VM under the tenant is suspended; the next gateway call 402s as normal.

Subscribe to these like any other event — the SDK events feed surfaces llm.budget_warning / llm.budget_exceeded alongside every other project event, off the injected token, with no URL to register and no signing to plumb:

import { events } from '@superjolt/sdk';

events.listen(async (e) => {
  if (e.type === 'llm.budget_exceeded') {
    // pause work, alert the operator, …
  }
});

Per-project spend visibility

Project Detail → Built-in models in the dashboard shows the calendar-UTC-month spend and a top-models breakdown (per-model spend + call count, sorted by spend, split into allowance-funded vs balance-funded), scoped to the one project. The same numbers come back via MCP — get_project surfaces the headline figures and account_status shows the free-allowance remaining; the per-model breakdown is at GET /v1/projects/:id/llm/spend.

Per-project budget enforcement is a follow-up; today the tenant monthlyBudgetCents is the only ceiling, and the project view is visibility-only.

Out of scope at v1

• Streaming responses (revisit when there’s demand).
• Embeddings / fine-tune APIs.
• Direct BYO provider keys in your app — the upstream provider relationships are managed by Superjolt; BYOK is a later platform-side optimization, not an app concern.