Models — provider factory inside runs, money plane for your code

PlungeAI touches models in two distinct places. Keep them apart:

1. Inside platform runs — agents, workflows, and missions resolve models through the platform's provider factory: a fleet of per-provider Workers behind one selection layer. You steer it with model/provider fields on tasks and missions.
2. The money plane — an OpenAI-compatible inference API at https://api.plungeai.com (/v1/chat/completions, /v1/embeddings, /v1/models) for YOUR code, with routing intelligence on top.

Discovery-first applies doubly here: model catalogs churn weekly. GET /v1/models is the priced, live catalog — never hardcode a model list in generated code.

1. Models inside platform runs (provider factory)

Every LLM-capable agent resolves its model through the central provider layer — one Worker per provider (Anthropic, OpenAI, Gemini, Groq, xAI, OpenRouter, Cerebras, Cohere, Mistral, Together, Fireworks, Perplexity, Bedrock, Vertex, and more; the fleet grows — treat any list as illustrative). Each provider Worker:

• serves streaming (SSE) and non-streaming completions behind the same interface;
• detects model capabilities by pattern, not hardcoded lists — new models of a known family work without platform changes (e.g. a new gpt-5.x or o<n> routes as a reasoning model automatically);
• auto-continues on truncation: when a completion stops at the output-token limit, the provider re-issues with a continue turn (capped at 3 extra turns) so workflow steps do not silently end mid-sentence — but only when maxTokens is left unset. An explicit maxTokens is a hard cap: it disables auto-continuation and the answer stops at the budget with finish_reason: length;
• records token usage per task so every run is priced post-hoc — this is where the cost numbers in the plungeai-results-traces skill come from.

Steering model choice in YAML

# On a plain task — flat fields on the task, passed through to the agent
- type: task
  id: analyze
  agent: llm-agent
  prompt: "…"
  model: claude-sonnet-5
  maxTokens: 4096

# On a harness mission (mission-level override)
- type: harness
  goal: "…"
  mission: |
    You are a careful researcher.
  model: claude-sonnet-5
  provider: anthropic
  effort: standard

# Follow-up behavior of a saved workflow
followup:
  provider: anthropic        # openai | groq | anthropic | gemini
  model: claude-sonnet-4-5
  temperature: 0.7

Flat task fields are the platform spec (persona/model/maxTokens etc. pass through). A nested config: {model, maxTokens} block is tolerated by llm-agent only, which flattens it — do not rely on nesting for other agents.

Omit model and the agent/provider default applies — usually the right call. Override only when the job demands a specific capability tier (reasoning depth, speed, cost). Provider-specific extras (extended thinking, reasoning effort, search grounding) pass through the same config fields per the agent's card.

2. The money plane (OpenAI-compatible, with routing)

Auth: sk-ocean- keys (not ozk_). Any OpenAI-compatible client works by pointing its base URL at https://api.plungeai.com/v1.

curl -s https://api.plungeai.com/v1/chat/completions \
  -H "Authorization: Bearer sk-ocean-YOUR_KEY" -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [{"role": "user", "content": "Say hello"}]
  }'

Model slugs are provider/model (e.g. anthropic/claude-sonnet-5). The response is standard OpenAI shape — with one honest twist:

model in the response is the slug that ACTUALLY served the request. After a failover it may differ from what you asked for. Bill and log on the response value.

Routing intelligence (what you get beyond a raw proxy)

{
  "models": ["anthropic/claude-sonnet-5", "<second choice from /v1/models>", "<third choice from /v1/models>"],
  "sort": "latency",
  "messages": [{"role": "user", "content": "…"}]
}

(Fallback slugs must come from GET /v1/models — discover, don't assume.)

• models[] — ordered fallback candidates (alternative to model). One retry per candidate on 429/5xx/network, then failover to the next; a provider that just failed is excluded for 30 s.
• sort — price | latency | throughput reorders models[] from rolling provider stats before the first attempt (no history sorts last).
• @preset/<slug> as model — expands a stored model+routing+params bundle; request-explicit fields still override.
• Response cache (opt-in per org): exact-match, only for temperature: 0 non-streaming requests; x-cache: hit|miss header when eligible.
• Guardrails per org/key: spend caps, model allow-lists, content filtering, BYOK gates — enforced server-side on chat AND embeddings.
• stream: true → text/event-stream.

Embeddings and catalog

curl -s https://api.plungeai.com/v1/embeddings \
  -H "Authorization: Bearer sk-ocean-YOUR_KEY" -H "Content-Type: application/json" \
  -d '{"model": "openai/text-embedding-3-small", "input": ["hello", "world"]}'

curl -s https://api.plungeai.com/v1/models \
  -H "Authorization: Bearer sk-ocean-YOUR_KEY"
# → {object: "list", data: [{id, pricing, provider}, …]}  — the priced catalog
#   routing candidates are drawn from. Discover, don't assume.

Money-plane errors — each names its fix

Guardrail refusals (402/403/429-cap) are trust fences: surface them, never engineer around them.

Choosing between the two surfaces

Both surfaces echo x-request-id and accept x-trace-id for correlation — see the plungeai-results-traces skill.