OpenAI provider

Standalone — usable in any Node project, no @warlock.js/core required.

@warlock.js/ai-openai is the OpenAI provider adapter for @warlock.js/ai. It turns OpenAI Chat Completions into a vendor-neutral ModelContract you pass to any agent, workflow, or supervisor.

It also works with any OpenAI-compatible endpoint — Azure OpenAI, OpenRouter, local LLM gateways — by passing a custom baseURL.

Install

npm

npm install @warlock.js/ai @warlock.js/ai-openai

yarn

yarn add @warlock.js/ai @warlock.js/ai-openai

pnpm

pnpm add @warlock.js/ai @warlock.js/ai-openai

Construct

import { OpenAISDK } from "@warlock.js/ai-openai";

const openai = new OpenAISDK({
  apiKey: process.env.OPENAI_API_KEY!,
});

// Or route through an OpenAI-compatible endpoint:
const openrouter = new OpenAISDK({
  apiKey: process.env.OPENROUTER_API_KEY!,
  baseURL: "https://openrouter.ai/api/v1",
  provider: "openrouter",
});

Use as a model

const model = openai.model({ name: "gpt-4o-mini" });
// Pass to any @warlock.js/ai agent / workflow / supervisor.

Use as an embedder

const embedder = openai.embedder({ name: "text-embedding-3-small" });
const { vector } = await embedder.embed("Hello world");

Capabilities

• Structured output — on by default (response_format: json_schema, strict: true); pass responseFormat: "json_object" or "text" on the model call to downgrade to the soft “respond in JSON only” system-prompt hint instead.
• Vision — auto-detected for gpt-4o*, gpt-4-turbo*, gpt-4.1*, o1*, o3*, chatgpt-4o*. Override with the vision flag.
• Streaming — token deltas, tool-call fragments, terminal done.
• Pricing — built-in list prices for flat-priced OpenAI models, or a pricing map (USD per million tokens) for a cost breakdown on every usage.

Pricing and usage

Every response reports token usage (input, output, total), on model.complete() and the streaming done chunk as well as on agent reports. OpenAI’s prompt_tokens already contains the cached tokens, so the usage is marked inputIncludesCache: true: prompt-cache hits surface as cachedTokens, billed cache writes (on the models that charge for them) as cacheWriteTokens, and both are billed out of input exactly once. Reasoning-model thinking tokens appear as reasoningTokens (counted within output). When the model is priced, usage.cost holds the per-channel USD breakdown; sum it with totalCost(usage.cost) from @warlock.js/ai.

Built-in list prices. With the default "openai" provider label, flat-priced models are priced out of the box from OPENAI_MODEL_PRICING — taken from OpenAI’s pricing page (Standard tier), fetched 2026-10-17: gpt-5.4-mini, gpt-5.4-nano, gpt-5.2, gpt-5.1, gpt-5, gpt-5-mini, gpt-5-nano, the gpt-4.1 family, gpt-4o, gpt-4o-mini, o3, o4-mini, o3-mini and o1. The lookup is by exact id (openAIPricingFor(name)).

Left unpriced on purpose — pass pricing for them: models priced by prompt length (a higher rate above 272K input tokens: the gpt-6, gpt-5.6, gpt-5.5 and gpt-5.4 families and the -pro models), dated snapshot ids, promotional / Batch / Flex / Priority pricing, the data-residency uplift, and any SDK with a custom provider label.

Resolution: per-model pricing > SDK pricing registry > built-in table > unpriced (Usage.cost stays undefined).

import { totalCost } from "@warlock.js/ai";
import { OpenAISDK } from "@warlock.js/ai-openai";

const openai = new OpenAISDK({
  apiKey: process.env.OPENAI_API_KEY ?? "",
  pricing: {
    // Tiered models have no built-in rate; contract rates win over the table.
    "gpt-5.5": { input: 5, output: 30, cachedInput: 0.5 },
  },
});

// Or per model, overriding the SDK entry and the built-in table:
const model = openai.model({ name: "gpt-4o", pricing: { input: 2.5, output: 10 } });

const { usage } = await model.complete([{ role: "user", content: "hi" }]);
console.log(usage.cachedTokens, totalCost(usage.cost));

An overloaded API (HTTP 503 / server_is_overloaded) surfaces as ProviderOverloadedError (code: "PROVIDER_OVERLOADED", retryable: true). Every provider error carries status, retryable and billed (false for 401 / 402 / 403, otherwise "unknown").

Prompt-cache hints

openai.model({ name, promptCacheKey, promptCacheRetention }) sends prompt_cache_key and prompt_cache_retention, and cacheControl.breakpoints adds explicit prompt_cache_breakpoint markers. Wrapper SDKs reuse this transport: xAI and Mistral receive prompt_cache_key (both document it), but never the OpenAI-only retention or breakpoint fields; Groq and DeepSeek cache automatically and receive no hint; a custom provider label receives none of them.

Need an offline token estimate before a call? openai.count(text) returns a fast character-heuristic approximation — good for budgeting and quota guards, not for billing.

Response format override

By default the adapter emits strict response_format: { type: "json_schema", strict: true } whenever an agent passes an output schema, so the JSON always matches the shape. Some OpenAI-compatible targets reject strict json_schema (older models, certain OpenRouter routes, Ollama’s OpenAI-compat endpoint, some fine-tunes). For those, set responseFormat on the model call to pick a looser wire mode:

const model = openai.model({ name: "some-legacy-model", responseFormat: "json_object" });

responseFormat	Wire response_format	Shape enforcement
"json_schema"	{ type: "json_schema", strict: true }	Token-level — the model can only emit JSON matching the schema. The default when omitted.
"json_object"	{ type: "json_object" }	Valid JSON, but not the shape — re-communicated via the agent’s soft system-prompt hint.
"text"	(none emitted)	No response_format on the wire — relies entirely on the soft prompt hint.

Picking "json_object" or "text" also flips the model’s inferred capabilities.structuredOutput to false, so the agent runtime re-injects a soft “respond in JSON only” hint. Client-side validation against your output schema still runs, so result.data is shape-checked either way.

For the full surface (pricing config, streaming caveats, embedding dimension truncation, token counting), see the warlock-js-ai-openai skill.

Image generation

openai.image({ name }) returns an ImageModelContract for the ai.image() output verb — prompt in, images out, in the same never-throws { data, error, usage, report } envelope every executable returns. OpenAI ships two metering models behind one factory:

import { ai } from "@warlock.js/ai";
import { OpenAISDK } from "@warlock.js/ai-openai";

const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY! });

// gpt-image-1 — token-metered; always returns base64 bytes.
const gpt = openai.image({ name: "gpt-image-1", pricing: { input: 5, output: 40 } });

// dall-e-3 — per-image pricing; base64 by default.
const dalle = openai.image({ name: "dall-e-3", pricing: { perImage: 0.04 } });

const { data, error } = await ai.image({
  model: gpt,
  prompt: "an isometric office desk, soft studio lighting",
  size: "1024x1024",
  quality: "high",
});

if (error) {
  console.warn(error.code); // typed AIError (content-filter / rate-limit / …)
} else {
  for (const img of data.images) {
    if (img.type === "base64") save(Buffer.from(img.base64, "base64"), img.mediaType);
    else download(img.url);
  }
}

• gpt-image-* is token-metered — price it with { input, output } (USD per 1M tokens). The adapter never sends response_format (the API rejects it), so it always returns base64 bytes.
• dall-e-* is per-image — price it with { perImage } (or perImageBySize). It defaults to base64; opt into a hosted URL with options: { responseFormat: "url" }.
• The model id is not validated locally. openai.image({ name }) passes it straight to client.images.generate — the same goes for openai.speech() and openai.transcribe() — so an id OpenAI does not serve fails at the provider (as a typed AIError), not at construction. There are no model-id helpers to consult — isOpenAIImageModel(), OPENAI_IMAGE_MODEL_PREFIXES, isOpenAISpeechModel() and isOpenAITranscriptionModel() were all removed in 4.13.0. Branch on your own id list if you need one.

The image spend folds into the same Usage.cost rollup as text — no second accounting path. See ai.image for the full verb surface (options, cost-truth, GeneratedImage).

Speech (text-to-speech)

openai.speech({ name }) returns a SpeechModelContract for the ai.speech() verb — text in, audio out.

// tts-1 / tts-1-hd — billed per INPUT CHARACTER.
const classic = openai.speech({
  name: "tts-1",
  voice: "alloy",
  pricing: { perMillionCharacters: 15 },
});

// gpt-4o-mini-tts — billed per TOKEN; supports `instructions` tone steering.
const steered = openai.speech({ name: "gpt-4o-mini-tts", pricing: { input: 0.6, output: 12 } });

const { data, error } = await ai.speech({
  model: steered,
  text: "Welcome aboard. Let's get you set up.",
  voice: "verse", // overrides the model default
  format: "wav", // "mp3" | "opus" | "aac" | "flac" | "wav" | "pcm"
  speed: 1.25, // OpenAI 0.25–4.0
  instructions: "calm, warm", // gpt-4o-mini-tts only
});

if (error) {
  console.warn(error.code);
} else {
  const { base64, mediaType } = data.audio; // GeneratedAudio, always base64 today
  await fs.writeFile("welcome.wav", Buffer.from(base64, "base64"));
}

The OpenAI container defaults to mp3 (→ audio/mpeg); speed and instructions are only sent when set, and the default voice is alloy when neither the call nor the model config supplies one. tts-1 reports no token usage, so its cost is priced entirely from the input character count.

Transcription (speech-to-text)

openai.transcribe({ name }) returns a TranscriptionModelContract for the ai.transcribe() verb — audio in, text out. Build the AudioInput with ai.audioFromFile(path) (reads disk, infers the media type, recognizes WhatsApp .ogg / .opus and iOS .m4a) or ai.audioFromBuffer(bytes, mediaType).

// whisper-1 — defaults to verbose_json (segments + duration); billed PER MINUTE.
const whisper = openai.transcribe({ name: "whisper-1", pricing: { perMinute: 0.006 } });

// gpt-4o-transcribe — defaults to json; billed PER TOKEN.
const gpt4o = openai.transcribe({ name: "gpt-4o-transcribe", pricing: { input: 2.5, output: 10 } });

const audio = await ai.audioFromFile("./voice-note.ogg");

const { data, error } = await ai.transcribe({
  model: whisper,
  audio,
  language: "en", // BCP-47 hint — improves accuracy + latency
});

if (error) {
  console.warn(error.code);
} else {
  console.log(data.text); // full transcript
  for (const seg of data.segments ?? []) {
    console.log(seg.start, seg.end, seg.text); // whisper verbose_json segments
  }
}

segments and report.durationSeconds appear only in whisper’s verbose_json mode — use them to build subtitles or jump-to-timestamp. The adapter wraps the base64 bytes in an uploadable via the SDK’s toFile, declaring the codec from audio.mediaType / audio.filename. whisper-1 cost derives from durationSeconds; if the provider reports no duration, cost stays undefined — no guessing.

PDF and audio input (opt-in)

Vision is auto-detected, but PDF and audio input parts are off by default — OpenAI accepts them only on specific model families, so the capability is honest-off until you opt in:

const doc = openai.model({ name: "gpt-4o", pdf: true }); // PDF file parts
const listen = openai.model({ name: "gpt-4o-audio-preview", audio: true }); // input_audio

Content part	OpenAI wire mapping	Requires
{ type: "pdf", source: { base64, mediaType } }	{ type: "file", file: { filename, file_data } }	.model({ pdf: true })
{ type: "audio", source: { base64, mediaType } }	{ type: "input_audio", input_audio: { data, format } }	.model({ audio: true })

PDF and audio reach the wire only when the model declares the matching capability — the agent’s modality gate throws otherwise, so capability ≡ behavior. Only wav / mp3 audio media types are accepted, and a remote-URL PDF/audio source raises a typed InvalidRequestError (OpenAI has no remote file/audio source — resolve to base64 first). An explicit flag always wins over inference.

Related

• ai.image / ai.speech / ai.transcribe — the modality verb surfaces these factories feed.
• Google provider — Gemini and Imagen image generation, and Gemini inlineData multimodal input.
• API reference — every public export.

Provider body options and input-token counts

Native body fields merge after neutral options. When both values are plain objects, their immediate keys merge and caller-provided keys win; untouched neutral fields remain. Arrays, scalars and null replace the field. This merge is one level deep: a nested object explicitly supplied by the caller replaces that nested value. When both objects have string type discriminators and the caller changes the type, the whole object is replaced. For example, overriding response_format from json_schema to json_object removes the old json_schema fields; switching tool_choice variants removes old variant fields as well. The rule also applies to OpenAI-compatible adapters using this helper.

providerOptions.openai.body adds native request fields to Chat Completions or Responses, merged after the neutral options. For example:

const options = { providerOptions: { openai: { body: { service_tier: "default" } } } };
await model.complete(messages, options);

The key stays openai for compatible gateways with a custom provider label. Other provider keys are ignored. OpenAICallOptions is exported. The body cannot replace model, messages, input, instructions, tools, stream, stream_options, previous_response_id, conversation, store, include, functions, function_call, prompt or background; those fields protect the adapter’s conversation, tools and stateless replay.

Models using api: "responses" expose native countTokens(messages, options?) through responses.inputTokens.count: the same input, system instructions, tools, reasoning, text format, tool choice, parallel-tool setting and truncation used for generation are counted. The endpoint supports only input-shaping fields; output caps and generation-only options are omitted. The Chat Completions model has no countTokens method. sdk.count(text) remains a separate offline estimate.

For budget preflight, countInputTokens(model, messages, { tools }) from @warlock.js/ai returns { tokens, source }. It estimates only when native counting is absent or explicitly throws InvalidRequestError with reason: "count-unsupported"; this adapter preserves that signal. System and tools are supported natively. Ordinary validation, auth, network and rate-limit errors propagate. See the cost/budget guide for reserve mode.

if (model.countTokens) {
  const inputTokens = await model.countTokens(messages, options);
}

Built-in tools and streamed reasoning

With api: "responses", OpenAI’s hosted tools (web_search, file_search, code_interpreter, image_generation, mcp) go in tools through ai.providerTool({ provider: "openai", definition }), appended verbatim after the function tools. Built-ins your code must execute (shell, apply_patch, computer, custom) are not supported: the adapter throws InvalidRequestError when the model emits one. Chat Completions only takes function / custom tools from OpenAI; the OpenAI-compatible wrappers send a provider tool when its provider equals their label (e.g. "groq"). See Provider-defined tools.

const model = openai.model({ name: "gpt-5.6", api: "responses" });
const webSearch = ai.providerTool({ provider: "openai", definition: { type: "web_search" } });

With streamChunks (on by default under agent.stream()), the Responses model yields reasoning for reasoning-summary and reasoning-text deltas (summaries need body: { reasoning: { summary: "auto" } }) and tool-args for response.function_call_arguments.delta. The Chat model yields tool-args for argument fragments and reasoning for a compatible vendor’s reasoning_content / reasoning delta. See Stream reasoning and tool arguments.

Hosted-search metering and metadata

On Responses, each structured web_search_call output item contributes one usage.serverToolUse.webSearchRequests. Streaming item events and terminal output are reconciled as snapshots, so the same call is counted once. response.providerMetadata.openai.output (also on streaming done) retains hosted output items with their action and sources. The adapter automatically requests web_search_call.action.sources when web-search tools are supplied. Agent reports preserve this metadata on each trip; it is not replayed or passed to function dispatch.

Explicitly set pricing.serverTools: OPENAI_SERVER_TOOL_PRICING (exported by this package) for standard web_search: $0.01 per tool call, plus search-content tokens at model rates. Non-reasoning web_search_preview has a different fee and needs an explicit override. These prices were verified on 2026-10-18 from OpenAI pricing; see the official search output schema and cost tracking. Chat Completions does not map hosted-search billing counters.

Qwen’s enableSearch flag and Kimi’s standalone kimiSearchTool() / kimiFetchTool() have separate controls; Kimi’s legacy builtin retires on 20 October 2026 and is unsupported; see OpenAI-compatible native search.