OpenAI provider
Standalone — usable in any Node project, no
@warlock.js/corerequired.
@warlock.js/ai-openai is the OpenAI provider adapter for
@warlock.js/ai. It turns OpenAI Chat Completions into
a vendor-neutral ModelContract you pass to any agent, workflow, or
supervisor.
It also works with any OpenAI-compatible endpoint — Azure OpenAI,
OpenRouter, local LLM gateways — by passing a custom baseURL.
Install
Section titled “Install”npm install @warlock.js/ai @warlock.js/ai-openaiyarn add @warlock.js/ai @warlock.js/ai-openaipnpm add @warlock.js/ai @warlock.js/ai-openaiConstruct
Section titled “Construct”import { OpenAISDK } from "@warlock.js/ai-openai";
const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY!,});
// Or route through an OpenAI-compatible endpoint:const openrouter = new OpenAISDK({ apiKey: process.env.OPENROUTER_API_KEY!, baseURL: "https://openrouter.ai/api/v1", provider: "openrouter",});Use as a model
Section titled “Use as a model”const model = openai.model({ name: "gpt-4o-mini" });// Pass to any @warlock.js/ai agent / workflow / supervisor.Use as an embedder
Section titled “Use as an embedder”const embedder = openai.embedder({ name: "text-embedding-3-small" });const { vector } = await embedder.embed("Hello world");Capabilities
Section titled “Capabilities”- Structured output — on by default (
response_format: json_schema, strict: true); passresponseFormat: "json_object"or"text"on the model call to downgrade to the soft “respond in JSON only” system-prompt hint instead. - Vision — auto-detected for
gpt-4o*,gpt-4-turbo*,gpt-4.1*,o1*,o3*,chatgpt-4o*. Override with thevisionflag.contextWindowsetscapabilities.contextWindow(used by window-relative compaction triggers); on the directopenaiprovider it is filled forgpt-4.1*,gpt-4o*,gpt-4,gpt-4-turbo*,gpt-3.5-turbo*,gpt-5,gpt-5-*,gpt-5.*andgpt-6*(400K, the documented gpt-5 figure; setcontextWindowfor a larger window),o1*,o3*ando4-mini*, and left unset for any other id. Other OpenAI-compatible endpoints takecontextWindowfrom the model config only. - Streaming — token deltas, tool-call fragments, terminal
done. - Pricing — built-in list prices for flat-priced OpenAI models, or a
pricingmap (USD per million tokens) for a cost breakdown on everyusage.
List models
Section titled “List models”await openai.models() lists the models your key can use from GET /v1/models
and returns ModelInfo[]: id (pass it as name), label, provider, kind
("chat", "embedding", "image", "audio", "moderation" or "other"), and
created (Unix seconds) when reported. OpenAI’s listing carries no modality
field, so kind is classified from the model id: treat it as a filter hint for
ids it does not know. Pagination is followed to the end and failures throw the
usual provider errors (for example ProviderAuthError), never an empty list.
Works the same on OpenAI-compatible gateways that implement GET /models, with
provider set to the SDK’s label. Pass { signal } to cancel.
const chat = (await openai.models()).filter((model) => model.kind === "chat");Pricing and usage
Section titled “Pricing and usage”Every response reports token usage (input, output, total), on
model.complete() and the streaming done chunk as well as on agent
reports. OpenAI’s prompt_tokens already contains the cached tokens, so
the usage is marked inputIncludesCache: true: prompt-cache hits surface
as cachedTokens, billed cache writes (on the models that charge for
them) as cacheWriteTokens, and both are billed out of input exactly
once. Reasoning-model thinking tokens appear as reasoningTokens
(counted within output). When the model is priced, usage.cost holds
the per-channel USD breakdown; sum it with totalCost(usage.cost) from
@warlock.js/ai.
Built-in list prices. With the default "openai" provider label,
flat-priced models are priced out of the box from OPENAI_MODEL_PRICING
— taken from OpenAI’s pricing page
(Standard tier), as of OPENAI_MODEL_PRICING_AS_OF (2026-11-06):
gpt-6.1-terra, gpt-6-cyber, gpt-6-astra, gpt-6-luna,
gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini,
gpt-5.4-nano, gpt-5.2, gpt-5.1, gpt-5, gpt-5-mini, gpt-5-nano,
the gpt-4.1 family, gpt-4o, gpt-4o-mini, o3, o4-mini, o3-mini
and o1. The lookup is by exact id (openAIPricingFor(name)).
The gpt-6, gpt-5.6, gpt-5.5 and gpt-5.4 entries carry tiers:
when a request’s prompt is longer than 272K tokens, every token of that
request bills at the tier’s higher rates (computeCost picks the tier from
the usage; resolvePricing from @warlock.js/ai does the same by hand).
Left unpriced on purpose — pass pricing for them: the -pro models,
gpt-6-sol and gpt-5.6-sol (promotional pricing until 2026-11-30), dated
snapshot ids, Batch / Flex / Priority pricing, the data-residency uplift,
and any SDK with a custom provider label.
Model lifecycle. OPENAI_MODEL_LIFECYCLE (lookup:
openAILifecycleFor(name)) records each priced model’s status from
OpenAI’s deprecation page:
o1, o3-mini, o4-mini and gpt-4.1-nano are retired (2026-10-23),
gpt-5, gpt-5-mini, gpt-5-nano and o3 are retiring on 2026-12-11,
gpt-4.1 and gpt-4.1-mini on 2027-01-14, and gpt-5.1 and
gpt-5.4-nano on 2027-04-01. Each entry is
{ status: "current" | "retiring" | "retired", retiresOn?, replacement? }.
Retired ids stay in the price table so old reports keep their cost.
Resolution: per-model pricing > SDK pricing registry > built-in
table > unpriced (Usage.cost stays undefined).
import { totalCost } from "@warlock.js/ai";import { OpenAISDK } from "@warlock.js/ai-openai";
const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY ?? "", pricing: { // Tiered models have no built-in rate; contract rates win over the table. "gpt-5.5": { input: 5, output: 30, cachedInput: 0.5 }, },});
// Or per model, overriding the SDK entry and the built-in table:const model = openai.model({ name: "gpt-4o", pricing: { input: 2.5, output: 10 } });
const { usage } = await model.complete([{ role: "user", content: "hi" }]);console.log(usage.cachedTokens, totalCost(usage.cost));An overloaded API (HTTP 503 / server_is_overloaded) surfaces as
ProviderOverloadedError (code: "PROVIDER_OVERLOADED",
retryable: true). Every provider error carries status, retryable
and billed (false for 401 / 402 / 403, otherwise "unknown").
An account that is out of credit surfaces as QuotaExceededError
(code: "PROVIDER_QUOTA_EXCEEDED", not retryable), whatever the status. It is
recognized from the error code (insufficient_quota,
credit_balance_exhausted, billing_hard_limit_reached, billing_not_active)
or from type: "insufficient_quota", so a 429 for spent credit is not retried
as a rate limit. A plain 429 stays ProviderRateLimitError. The default
fallback chain does not advance on PROVIDER_QUOTA_EXCEEDED; pass it in
alsoRetryOn of the fallback model to switch providers when credit runs out. The same mapping covers
the OpenAI-compatible providers (DeepSeek, Groq, xAI, Mistral).
Prompt-cache hints
Section titled “Prompt-cache hints”openai.model({ name, promptCacheKey, promptCacheRetention }) sends
prompt_cache_key and prompt_cache_retention, and
cacheControl.breakpoints adds explicit prompt_cache_breakpoint
markers. A per-call cacheControl: { key } is sent as prompt_cache_key
and takes precedence over promptCacheKey. Wrapper SDKs reuse this transport: xAI and Mistral receive
prompt_cache_key (both document it), but never the OpenAI-only
retention or breakpoint fields; Groq and DeepSeek cache automatically
and receive no hint; a custom provider label receives none of them.
Need an offline token estimate before a call? openai.count(text) returns a fast character-heuristic approximation — good for budgeting and quota guards, not for billing.
Response format override
Section titled “Response format override”By default the adapter emits strict response_format: { type: "json_schema", strict: true } whenever an agent passes an output schema, so the JSON always matches the shape. Some OpenAI-compatible targets reject strict json_schema (older models, certain OpenRouter routes, Ollama’s OpenAI-compat endpoint, some fine-tunes). For those, set responseFormat on the model call to pick a looser wire mode:
const model = openai.model({ name: "some-legacy-model", responseFormat: "json_object" });responseFormat | Wire response_format | Shape enforcement |
|---|---|---|
"json_schema" | { type: "json_schema", strict: true } | Token-level — the model can only emit JSON matching the schema. The default when omitted. |
"json_object" | { type: "json_object" } | Valid JSON, but not the shape — re-communicated via the agent’s soft system-prompt hint. |
"text" | (none emitted) | No response_format on the wire — relies entirely on the soft prompt hint. |
Picking "json_object" or "text" also flips the model’s inferred capabilities.structuredOutput to false, so the agent runtime re-injects a soft “respond in JSON only” hint. Client-side validation against your output schema still runs, so result.data is shape-checked either way.
For the full surface (pricing config, streaming caveats, embedding
dimension truncation, token counting), see the
warlock-js-ai-openai skill.
Image generation
Section titled “Image generation”openai.image({ name }) returns an ImageModelContract for the
ai.image() output verb — prompt in, images out, in the
same never-throws { data, error, usage, report } envelope every
executable returns. OpenAI ships two metering models behind one factory:
import { ai } from "@warlock.js/ai";import { OpenAISDK } from "@warlock.js/ai-openai";
const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY! });
// gpt-image-1 — token-metered; always returns base64 bytes.const gpt = openai.image({ name: "gpt-image-1", pricing: { input: 5, output: 40 } });
// dall-e-3 — per-image pricing; base64 by default.const dalle = openai.image({ name: "dall-e-3", pricing: { perImage: 0.04 } });
const { data, error } = await ai.image({ model: gpt, prompt: "an isometric office desk, soft studio lighting", size: "1024x1024", quality: "high",});
if (error) { console.warn(error.code); // typed AIError (content-filter / rate-limit / …)} else { for (const img of data.images) { if (img.type === "base64") save(Buffer.from(img.base64, "base64"), img.mediaType); else download(img.url); }}gpt-image-*is token-metered — price it with{ input, output }(USD per 1M tokens). The adapter never sendsresponse_format(the API rejects it), so it always returns base64 bytes.dall-e-*is per-image — price it with{ perImage }(orperImageBySize). It defaults to base64; opt into a hosted URL withoptions: { responseFormat: "url" }.- The model id is not validated locally.
openai.image({ name })passes it straight toclient.images.generate— the same goes foropenai.speech()andopenai.transcribe()— so an id OpenAI does not serve fails at the provider (as a typedAIError), not at construction. There are no model-id helpers to consult —isOpenAIImageModel(),OPENAI_IMAGE_MODEL_PREFIXES,isOpenAISpeechModel()andisOpenAITranscriptionModel()were all removed in 4.13.0. Branch on your own id list if you need one.
The image spend folds into the same Usage.cost rollup as text — no second
accounting path. See ai.image for the full verb surface
(options, cost-truth, GeneratedImage).
Speech (text-to-speech)
Section titled “Speech (text-to-speech)”openai.speech({ name }) returns a SpeechModelContract for the
ai.speech() verb — text in, audio out.
// tts-1 / tts-1-hd — billed per INPUT CHARACTER.const classic = openai.speech({ name: "tts-1", voice: "alloy", pricing: { perMillionCharacters: 15 },});
// gpt-4o-mini-tts — billed per TOKEN; supports `instructions` tone steering.const steered = openai.speech({ name: "gpt-4o-mini-tts", pricing: { input: 0.6, output: 12 } });
const { data, error } = await ai.speech({ model: steered, text: "Welcome aboard. Let's get you set up.", voice: "verse", // overrides the model default format: "wav", // "mp3" | "opus" | "aac" | "flac" | "wav" | "pcm" speed: 1.25, // OpenAI 0.25–4.0 instructions: "calm, warm", // gpt-4o-mini-tts only});
if (error) { console.warn(error.code);} else { const { base64, mediaType } = data.audio; // GeneratedAudio, always base64 today await fs.writeFile("welcome.wav", Buffer.from(base64, "base64"));}The OpenAI container defaults to mp3 (→ audio/mpeg); speed and
instructions are only sent when set, and the default voice is alloy when
neither the call nor the model config supplies one. tts-1 reports no token
usage, so its cost is priced entirely from the input character count.
Transcription (speech-to-text)
Section titled “Transcription (speech-to-text)”openai.transcribe({ name }) returns a TranscriptionModelContract for the
ai.transcribe() verb — audio in, text out. Build the
AudioInput with ai.audioFromFile(path) (reads disk, infers the media type,
recognizes WhatsApp .ogg / .opus and iOS .m4a) or
ai.audioFromBuffer(bytes, mediaType).
// whisper-1 — defaults to verbose_json (segments + duration); billed PER MINUTE.const whisper = openai.transcribe({ name: "whisper-1", pricing: { perMinute: 0.006 } });
// gpt-4o-transcribe — defaults to json; billed PER TOKEN.const gpt4o = openai.transcribe({ name: "gpt-4o-transcribe", pricing: { input: 2.5, output: 10 } });
const audio = await ai.audioFromFile("./voice-note.ogg");
const { data, error } = await ai.transcribe({ model: whisper, audio, language: "en", // BCP-47 hint — improves accuracy + latency});
if (error) { console.warn(error.code);} else { console.log(data.text); // full transcript for (const seg of data.segments ?? []) { console.log(seg.start, seg.end, seg.text); // whisper verbose_json segments }}segments and report.durationSeconds appear only in whisper’s
verbose_json mode — use them to build subtitles or jump-to-timestamp. The
adapter wraps the base64 bytes in an uploadable via the SDK’s toFile,
declaring the codec from audio.mediaType / audio.filename. whisper-1
cost derives from durationSeconds; if the provider reports no duration, cost
stays undefined — no guessing.
PDF and audio input (opt-in)
Section titled “PDF and audio input (opt-in)”Vision is auto-detected, but PDF and audio input parts are off by default — OpenAI accepts them only on specific model families, so the capability is honest-off until you opt in:
const doc = openai.model({ name: "gpt-4o", pdf: true }); // PDF file partsconst listen = openai.model({ name: "gpt-4o-audio-preview", audio: true }); // input_audio| Content part | OpenAI wire mapping | Requires |
|---|---|---|
{ type: "pdf", source: { base64, mediaType } } | { type: "file", file: { filename, file_data } } | .model({ pdf: true }) |
{ type: "audio", source: { base64, mediaType } } | { type: "input_audio", input_audio: { data, format } } | .model({ audio: true }) |
PDF and audio reach the wire only when the model declares the matching
capability — the agent’s modality gate throws otherwise, so capability ≡
behavior. Only wav / mp3 audio media types are accepted, and a remote-URL
PDF/audio source raises a typed InvalidRequestError (OpenAI has no remote
file/audio source — resolve to base64 first). An explicit flag always wins
over inference.
Related
Section titled “Related”ai.image/ai.speech/ai.transcribe— the modality verb surfaces these factories feed.- Google provider — Gemini and Imagen image
generation, and Gemini
inlineDatamultimodal input. - API reference — every public export.
Provider body options and input-token counts
Section titled “Provider body options and input-token counts”Native body fields merge after neutral options. When both values are plain
objects, their immediate keys merge and caller-provided keys win; untouched
neutral fields remain. Arrays, scalars and null replace the field. This merge
is one level deep: a nested object explicitly supplied by the caller replaces
that nested value. When both objects have string type discriminators and the
caller changes the type, the whole object is replaced. For example, overriding
response_format from json_schema to json_object removes the old
json_schema fields; switching tool_choice variants removes old variant fields
as well. The rule also applies to OpenAI-compatible adapters using this helper.
providerOptions.openai.body adds native request fields to Chat Completions or
Responses, merged after the neutral options. For example:
const options = { providerOptions: { openai: { body: { service_tier: "default" } } } };await model.complete(messages, options);The key stays openai for compatible gateways with a custom provider label.
Other provider keys are ignored. OpenAICallOptions is exported. The body cannot
replace model, messages, input, instructions, tools, stream,
stream_options, previous_response_id, conversation, store, include, functions, function_call, prompt or background;
those fields protect the adapter’s conversation, tools and stateless replay.
Models using api: "responses" expose native countTokens(messages, options?)
through responses.inputTokens.count: the same input, system instructions,
tools, reasoning, text format, tool choice, parallel-tool setting and truncation
used for generation are counted. The endpoint supports only input-shaping fields;
output caps and generation-only options are omitted. The Chat Completions model
has no countTokens method. sdk.count(text) remains a separate offline estimate.
For budget preflight, countInputTokens(model, messages, { tools }) from
@warlock.js/ai returns { tokens, source }. It estimates only when native
counting is absent or explicitly throws InvalidRequestError with
reason: "count-unsupported"; this adapter preserves that signal. System and
tools are supported natively. Ordinary validation, auth, network and rate-limit
errors propagate. See the cost/budget guide for reserve mode.
if (model.countTokens) { const inputTokens = await model.countTokens(messages, options);}Built-in tools and streamed reasoning
Section titled “Built-in tools and streamed reasoning”With api: "responses", OpenAI’s hosted tools (web_search, file_search, code_interpreter, image_generation, mcp) go in tools through ai.providerTool({ provider: "openai", definition }), appended verbatim after the function tools. Built-ins your code must execute (shell, apply_patch, computer, custom) are not supported: the adapter throws InvalidRequestError when the model emits one. Chat Completions only takes function / custom tools from OpenAI; the OpenAI-compatible wrappers send a provider tool when its provider equals their label (e.g. "groq"). See Provider-defined tools.
const model = openai.model({ name: "gpt-5.6", api: "responses" });const webSearch = ai.providerTool({ provider: "openai", definition: { type: "web_search" } });With streamChunks (on by default under agent.stream()), the Responses model yields reasoning for reasoning-summary and reasoning-text deltas (summaries need body: { reasoning: { summary: "auto" } }) and tool-args for response.function_call_arguments.delta. The Chat model yields tool-args for argument fragments and reasoning for a compatible vendor’s reasoning_content / reasoning delta. See Stream reasoning and tool arguments.
Native web search chooses the API for you
Section titled “Native web search chooses the API for you”Chat Completions cannot run OpenAI’s hosted web_search. On the direct
"openai" provider, a call that carries a web_search tool (or its dated and
_preview variants) is served by the Responses API automatically, with the same
model name, client and pricing; calls without it stay on Chat Completions.
const model = openai.model({ name: "gpt-4.1" }); // no `api` neededawait model.complete(messages, { tools: [webSearch] }); // sent to POST /v1/responsesNothing falls back silently. The adapter throws InvalidRequestError before any
request when:
- the model is pinned with
api: "chat"(remove it, or useapi: "responses"); - the model is one OpenAI documents as unable to search:
gpt-3.5*,gpt-4andgpt-4-turbo,gpt-4.1-nano,o1-mini,o1-preview,o3-mini, and non-chat ids such as embeddings,whisper,tts,dall-eandgpt-image*; - a
gpt-5*call asks forreasoning.effort: "minimal", which OpenAI’s search tool does not accept.
Any other id, including ones this list does not know, is sent and OpenAI
decides. gpt-4o-mini and gpt-4.1-mini can search; OpenAI bills extra
search-content tokens on them, so check the price before enabling search there.
Hosted-search metering and metadata
Section titled “Hosted-search metering and metadata”On Responses, each structured web_search_call output item contributes one
usage.serverToolUse.webSearchRequests. Streaming item events and terminal
output are reconciled as snapshots, so the same call is counted once.
response.providerMetadata.openai.output (also on streaming done) retains
hosted output items with their action and sources. The adapter automatically
requests web_search_call.action.sources when web-search tools are supplied.
Agent reports preserve this metadata on each trip; it is not replayed or
passed to function dispatch.
Explicitly set pricing.serverTools: OPENAI_SERVER_TOOL_PRICING (exported by
this package) for standard web_search: $0.01 per tool call, plus search-content
tokens at model rates. Non-reasoning web_search_preview has a different fee
and needs an explicit override. These prices were verified on 2026-10-18 from
OpenAI pricing;
see the official search output schema
and cost tracking.
Chat Completions does not map hosted-search billing counters.
Qwen’s enableSearch flag and Kimi’s standalone kimiSearchTool() /
kimiFetchTool() have separate controls; Kimi’s legacy builtin retires on
20 October 2026 and is unsupported; see OpenAI-compatible native search.