Configure an LLM provider

Task

Point Ethos at one of the supported LLM providers — Anthropic, OpenAI, OpenAI Codex, OpenRouter, Azure OpenAI, or a local Ollama or vLLM endpoint — and verify the next chat turn routes through it.

Result

ethos chat reaches the chosen provider, streams tokens back, and ethos doctor reports the provider as valid.

Prereqs

• ethos on PATH (Node 24+). Run ethos --version to confirm.
• An API key for the provider you're configuring, or a local Ollama install if you go that route.
• Write access to ~/.ethos/config.yaml.

Steps

The wizard handles the common path. Drop into hand-edit only when you need a non-default base URL or you're scripting the install.

Option A — Use the wizard

ethos setup

The wizard writes ~/.ethos/config.yaml and prompts for:

• Provider — one of anthropic, openai, codex, openrouter, azure, bedrock, xai, ollama, vllm.
• Model — the model id for that provider (see the table below). For azure, this is the deployment name. For ollama and vllm, the wizard fetches the served model list from the endpoint's GET /v1/models and lets you pick.
• API key — stored locally in ~/.ethos/config.yaml. The local providers (ollama, vllm) and bedrock skip this prompt.
• Default personality — pick one of the built-ins.

To re-run only the provider step on an existing config:

ethos setup auth
ethos setup model

Option B — Hand-edit the config

Open ~/.ethos/config.yaml and set four keys. The shape is plain key: value — no nested YAML.

provider: anthropic
model: claude-sonnet-5
apiKey: sk-ant-XXXXXXXXXXXX
personality: researcher

For OpenAI-compatible providers (openai, openrouter, xai, ollama, vllm), add baseUrl if you want a non-default endpoint:

provider: openrouter
model: anthropic/claude-sonnet-5
apiKey: sk-or-XXXXXXXXXXXX
baseUrl: https://openrouter.ai/api/v1
personality: researcher

For azure, baseUrl is your resource endpoint, model is the deployment name, and apiVersion is required:

provider: azure
model: <your-deployment-name>
apiKey: XXXXXXXXXXXX
baseUrl: https://<your-resource>.openai.azure.com
apiVersion: 2024-10-21
personality: researcher

A reasoning model (the o-series or gpt-5.x) needs max_completion_tokens instead of max_tokens. Ethos picks the parameter from the deployment name, so a deployment named after its model (o4-mini, the portal's default) works as is. If you named the deployment something else, set the parameter yourself on entry 0:

providers.0.provider: azure
providers.0.outputCapParam: max_completion_tokens

Without it, Azure refuses the request because it was sent max_tokens.

Provider matrix

provider	Default base URL	Where to get a key	Notes
anthropic	n/a (SDK default)	console.anthropic.com	Best fit for claude-* models; supports key rotation via ethos keys.
openai	https://api.openai.com/v1	platform.openai.com	Use for gpt-6-sol, gpt-6-astra, gpt-6-luna, etc.
codex	n/a — device auth	openai.com (ChatGPT account)	Experimental; authenticates via device code, no API key. See Use a ChatGPT subscription for coding work.
openrouter	https://openrouter.ai/api/v1	openrouter.ai/keys	One key for Claude, GPT, Gemini, Llama, and 200+ more.
azure	https://<your-resource>.openai.azure.com	portal.azure.com	model: is the deployment name; apiVersion: required (default 2024-10-21).
bedrock	n/a -- derived from region:	n/a -- AWS SigV4	No API key; credentials come from an IAM role, SSO session, or the Ethos secret store. region: defaults to us-east-1. See Run Ethos on AWS Bedrock.
xai	https://api.x.ai/v1	console.x.ai	Grok models direct from xAI. Paid only — no free tier. The key is stored at providers/xai/apiKey.
ollama	http://localhost:11434/v1	n/a — local	No API key; the wizard offers the served model list.
vllm	http://localhost:8000/v1	n/a — local	No API key; the wizard offers the served model list.

Provider strings are validated against packages/wiring/src/provider-catalog.ts. Anything else is rejected by ethos doctor.

bedrock is the one entry with no key step: the wizard skips the API-key prompt and asks for the model id as free text, because Bedrock signs each request with AWS SigV4 off the ambient credential chain. Run Ethos on AWS Bedrock covers the model id, the IAM permission, and how credentials resolve.

Output cap for Claude models

With provider: anthropic, each request's max_tokens is the model's documented maximum output, taken from the bundled model catalog: 128,000 for Claude Fable 5.1, Fable 5, Mythos 5.1, Mythos 5, Opus 5.5, Opus 5, Opus 4.8, 4.7 and 4.6, Sonnet 5 and Sonnet 4.6; 64,000 for Claude Haiku 4.5, Sonnet 4.5 and Opus 4.5. A Claude model the catalog does not list gets 8,096. The request always streams, so a large cap does not hit the SDK's request timeout.

To send a different cap for one model, set it in ~/.ethos/config.yaml. The config value beats the catalog:

models.anthropic/claude-opus-5.maxOutputTokens: 32000

The cap follows the model that answers. When a personality role routes a turn to another Claude model, that model's cap is sent, not the configured model's. The catalog cap applies only to provider: anthropic, because catalog rows are keyed by provider. The same Claude model through openrouter, azure or bedrock gets no catalog cap. openrouter honours models.openrouter/<model>.maxOutputTokens. azure and bedrock do not read that key.

Local endpoints (Ollama and vLLM)

Pull and run the model before pointing Ethos at it:

ollama pull llama3.1:8b
ollama serve   # leave running

Then in ~/.ethos/config.yaml:

provider: ollama
model: llama3.1:8b
apiKey: ollama
baseUrl: http://localhost:11434/v1
personality: researcher

For vLLM, the same shape with provider: vllm and baseUrl: http://localhost:8000/v1.

A local server needs serving flags before it works well for agent turns — context length, prefix caching, tool-call parsing. Set them per Configure local model serving, then score the model with Qualify a local model.

Optional — a fallback chain

Stack two providers so Ethos fails over automatically when the first one rate-limits or 5xx's. The chain triggers when two or more providers.<n>.* blocks are present.

provider: anthropic
model: claude-sonnet-5
apiKey: sk-ant-XXXXXXXXXXXX
personality: researcher

providers.0.provider: anthropic
providers.0.apiKey: sk-ant-XXXXXXXXXXXX
providers.0.model: claude-sonnet-5

providers.1.provider: openrouter
providers.1.apiKey: sk-or-XXXXXXXXXXXX
providers.1.model: anthropic/claude-sonnet-5

The top-level provider, model, and apiKey keys stay in place — they're used when the chain has fewer than two entries.

Verify

Run the health check and then one turn:

ethos doctor

doctor reports the active provider, the model, whether the SDK module is installed, and whether the API key is reachable.

Then:

ethos chat -q "respond with the single word 'ok'"

A streamed ok and a non-zero usage line means the provider, key, and model resolved end-to-end.

Troubleshoot

Unknown provider 'foo'. Did you mean 'anthropic'? — ethos doctor rejects provider strings outside the catalog. Set provider: to one of anthropic, openai, codex, openrouter, azure, bedrock, xai, ollama, vllm.

401 Unauthorized from the provider. — The key is wrong, expired, or missing the right scope. Regenerate at the provider console and re-run ethos setup auth.

ECONNREFUSED 127.0.0.1:11434 with provider: ollama. — ollama serve is not running. Start it in another terminal or check lsof -i :11434.

model not found from OpenRouter. — OpenRouter model ids are namespaced (anthropic/claude-sonnet-5, not claude-sonnet-5). Copy the exact id from the OpenRouter model page.

Empty stream, no error. — The base URL points at an endpoint that accepts requests but returns nothing useful (common with custom OpenAI-compatible gateways). Run ethos doctor and compare baseUrl against the provider's docs.

Rate-limited on Anthropic. — Add a rotation key with ethos keys add (Anthropic only) or fall back via the providers.<n> chain shown above.