Skip to content
TanStackAI
Adapters

Groq

The Groq adapter calls Llama, gpt-oss, Qwen, and Kimi with low latency. It also calls Whisper for transcription and Orpheus for speech.

1. Install

shell
npm i @tanstack/ai-groq

2. Set the API key

Put the key in the environment of your server:

shell
GROQ_API_KEY=gsk_...

groqText and the other short factories read GROQ_API_KEY. To pass the key in code, use the create* factory of the same adapter:

ts
import { createGroqText } from "@tanstack/ai-groq";

const adapter = createGroqText("openai/gpt-oss-120b", process.env.GROQ_API_KEY!);

3. Stream a chat from the server

ts
import {
  chat,
  chatParamsFromRequest,
  toServerSentEventsResponse,
} from "@tanstack/ai";
import { groqText } from "@tanstack/ai-groq";

export async function POST(request: Request) {
  const { messages, threadId, runId } = await chatParamsFromRequest(request);

  const stream = chat({
    adapter: groqText("openai/gpt-oss-120b"),
    messages,
    threadId,
    runId,
  });

  return toServerSentEventsResponse(stream);
}

4. Render the chat

The client is the same for every provider. Point useChat at your route:

tsx
import { useState } from "react";
import { useChat, fetchServerSentEvents } from "@tanstack/ai-react";

export function Chat() {
  const [input, setInput] = useState("");
  const { messages, sendMessage } = useChat({
    connection: fetchServerSentEvents("/api/chat"),
    threadId: "support-chat",
  });

  return (
    <form
      onSubmit={(event) => {
        event.preventDefault();
        sendMessage(input);
        setInput("");
      }}
    >
      {messages.map((message) => (
        <p key={message.id}>
          {message.parts
            .map((part) => (part.type === "text" ? part.content : ""))
            .join("")}
        </p>
      ))}
      <input value={input} onChange={(event) => setInput(event.target.value)} />
    </form>
  );
}

For Vue, Solid, Svelte, Angular, and the other frameworks, see Quick Start.

You now have a streaming chat on Groq. Tools and Structured Outputs work the same as with every adapter. For the Groq details of both, see Tool calls that Groq rejects and Structured output.

Transcription

groqTranscription turns speech into text with Whisper:

ts
import { generateTranscription } from "@tanstack/ai";
import { groqTranscription } from "@tanstack/ai-groq";

const result = await generateTranscription({
  adapter: groqTranscription("whisper-large-v3-turbo"),
  audio: "https://example.com/recording.mp3",
  language: "en",
  threadId: "thread-1",
});

console.log(result.text);

// verbose_json (the default) fills language, duration, and timestamped segments
for (const segment of result.segments ?? []) {
  console.log(`[${segment.start}s to ${segment.end}s] ${segment.text}`);
}
  • audio takes a File, a Blob, an ArrayBuffer, a base64 string, a data URL, or an https:// URL. The adapter sends a URL to Groq as it is, with no new upload.
  • Models: whisper-large-v3-turbo, whisper-large-v3.
  • responseFormat takes json, text, or verbose_json (the default). Groq does not support srt or vtt.

See Transcription.

Speech

groqSpeech turns text into speech with the Orpheus models:

ts
import { generateSpeech } from "@tanstack/ai";
import { groqSpeech } from "@tanstack/ai-groq";

const result = await generateSpeech({
  adapter: groqSpeech("canopylabs/orpheus-v1-english"),
  text: "Hello from Groq!",
  threadId: "thread-1",
  voice: "autumn",
  format: "wav",
});

console.log(result.audio); // Base64-encoded audio
  • Models: canopylabs/orpheus-v1-english and canopylabs/orpheus-arabic-saudi.
  • English voices: autumn (the default), diana, hannah, austin, daniel, troy.
  • Arabic voices: fahad, sultan, lulwa, noura.
  • Formats: flac, mp3, mulaw, ogg, and wav (the default). modelOptions.sample_rate sets the sample rate.

See Text-to-Speech.

Summarization

ts
import { summarize } from "@tanstack/ai";
import { groqSummarize } from "@tanstack/ai-groq";

const result = await summarize({
  adapter: groqSummarize("openai/gpt-oss-120b"),
  text: "Your long text to summarize...",
  threadId: "thread-1",
  maxLength: 100,
  style: "concise", // "concise" | "bullet-points" | "paragraph"
});

console.log(result.summary);

No image generation

Groq does not make images. For images, use OpenAI or Gemini. Groq has no provider tools of its own. Define your tools with toolDefinition().

Models and options

Model options

Provider options go in modelOptions. This includes the sampling options temperature, top_p, and max_completion_tokens (the token limit of Groq):

ts
import { chat } from "@tanstack/ai";
import { groqText } from "@tanstack/ai-groq";

const stream = chat({
  adapter: groqText("openai/gpt-oss-120b"),
  messages: [{ role: "user", content: "Hello!" }],
  threadId: "thread-1",
  modelOptions: {
    temperature: 0.7,
    max_completion_tokens: 1024,
    top_p: 0.9,
  },
});

Sampling options do not go at the root of chat(). See Moving Sampling Options into modelOptions.

Reasoning

Set how hard a reasoning model thinks with reasoning on chat():

ts
import { chat } from "@tanstack/ai";
import { groqText } from "@tanstack/ai-groq";

const stream = chat({
  adapter: groqText("openai/gpt-oss-120b"),
  messages: [{ role: "user", content: "Hello!" }],
  threadId: "thread-1",
  reasoning: "medium",
});

It goes out as reasoning_effort:

  • openai/gpt-oss-120b, openai/gpt-oss-20b, openai/gpt-oss-safeguard-20b: low, medium, high.
  • qwen/qwen3.8-27b: off (sent as none), low, medium, high.

To pick how the thinking text comes back, set reasoning_format in modelOptions.

Models

The package exports the full lists as GROQ_CHAT_MODELS, GROQ_TRANSCRIPTION_MODELS, and GROQ_TTS_MODELS.

GroupModels
OpenAI gpt-ossopenai/gpt-oss-120b (reasoning), openai/gpt-oss-20b, openai/gpt-oss-safeguard-20b (safety-tuned)
Qwenqwen/qwen3.8-27b (reasoning, vision)
Safetymeta-llama/llama-prompt-guard-2-86m and meta-llama/llama-prompt-guard-2-22m (prompt injection detection)

adapter.inputModalities lists the input kinds of the selected model. See Check what a model accepts.

Advanced

Configuration

ts
import { createGroqText, type GroqTextConfig } from "@tanstack/ai-groq";

const config: Omit<GroqTextConfig, "apiKey"> = {
  baseURL: "https://api.groq.com/openai/v1", // Optional, for custom endpoints
};

const adapter = createGroqText("openai/gpt-oss-120b", process.env.GROQ_API_KEY!, config);

Need a header on each request, or a log of each model call? This adapter supports wrapFetch.

Tool calls that Groq rejects

Groq can reject a tool call from the model with tool_use_failed. If the error has a tool call in failed_generation that the adapter can rebuild, the adapter does not run the call. It returns the provider error as the result of that tool. The agent loop can then repair the call on its next step. Other provider errors still end the run.

Structured output

Some Groq models, for example meta-llama/llama-prompt-guard-2-86m, support JSON mode but not strict JSON schema. For these models, chat({ outputSchema }) sends json_object and adds your schema to the system prompt. Read When strict JSON schema is rejected.

API reference

Each factory pair has the same shape. The short factory reads GROQ_API_KEY from the environment. The create* factory takes an explicit API key. Both take model as the first argument.

FactoriesCreates
groqText(model, config?) / createGroqText(model, apiKey, config?)Chat adapter
groqSummarize(model, config?) / createGroqSummarize(model, apiKey, config?)Summarization adapter
groqTranscription(model, config?) / createGroqTranscription(model, apiKey, config?)Speech-to-text adapter
groqSpeech(model, config?) / createGroqSpeech(model, apiKey, config?)Text-to-speech adapter

config.baseURL sets a custom base URL for the API requests. adapter.inputModalities lists the input kinds of the selected model. It is undefined for a model that this package does not know.

Edit on GitHub
Grok (xAI)
Mistral