Structured Output From LLMs: Get Reliable JSON With Validation and Retries

Sooner or later every AI feature needs data back from the model instead of prose: a list of tags, a sentiment score, fields pulled from an invoice. The model will usually give you JSON when asked. "Usually" is the problem. A missing field, a number sent as a string or a stray sentence before the opening brace will break your code at the worst moment.

This tutorial shows a short, reliable pattern that works with any OpenAI-compatible API:

1. Ask for JSON in a way the API enforces.
2. Validate the result against a schema.
3. If validation fails, send the errors back and let the model fix its own output.

Setup

npm install zod
export LLM_BASE_URL="https://api.openai.com/v1"
export LLM_API_KEY="sk-..."
export LLM_MODEL="your-model-name"

Step 1: Define the shape you want

Write the schema once, in code. It's both your validator and your documentation. Here we extract structured details from a support email.

import { z } from "zod";

const Ticket = z.object({
  category: z.enum(["billing", "bug", "feature_request", "other"]),
  urgency: z.number().int().min(1).max(5),
  summary: z.string().min(10).max(200),
  customerEmail: z.string().email().nullable(),
});
type Ticket = z.infer<typeof Ticket>;

Be strict. Enums instead of free text, integer ranges instead of "a number", explicit nullable() when a field may be missing. Every rule here is a bug you'll catch before it reaches your database.

Step 2: Ask for JSON mode

Most OpenAI-compatible providers support response_format: { type: "json_object" }, which forces the reply to be syntactically valid JSON. It doesn't guarantee your fields, which is why we still validate. Some APIs require the word "JSON" to appear in your messages when this mode is on, so we always describe the format in the system prompt.

const SYSTEM = `Extract a support ticket as JSON with exactly these fields:
- category: one of "billing", "bug", "feature_request", "other"
- urgency: integer 1 (low) to 5 (critical)
- summary: one sentence, 10-200 characters
- customerEmail: the sender's email address, or null if absent
Return only the JSON object.`;

async function callModel(messages: { role: string; content: string }[]): Promise<string> {
  const res = await fetch(`${process.env.LLM_BASE_URL}/chat/completions`, {
    method: "POST",
    headers: { authorization: `Bearer ${process.env.LLM_API_KEY}`, "content-type": "application/json" },
    body: JSON.stringify({
      model: process.env.LLM_MODEL,
      messages,
      temperature: 0,
      response_format: { type: "json_object" },
    }),
  });
  if (!res.ok) throw new Error(`LLM error ${res.status}: ${await res.text()}`);
  return (await res.json()).choices[0].message.content;
}

Use temperature: 0 for extraction tasks. You want the same input to give the same output.

Step 3: Validate, and retry with feedback

This is the heart of the pattern. When validation fails, we don't just ask again and hope. We show the model exactly what was wrong. Models are good at fixing a specific, named error.

export async function extractTicket(email: string, maxRetries = 2): Promise<Ticket> {
  const messages = [
    { role: "system", content: SYSTEM },
    { role: "user", content: email },
  ];

  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const raw = await callModel(messages);

    let parsed: unknown;
    try {
      parsed = JSON.parse(raw);
    } catch {
      parsed = undefined;
    }

    const result = Ticket.safeParse(parsed);
    if (result.success) return result.data;

    const problems = parsed === undefined
      ? "The reply was not valid JSON."
      : result.error.issues.map((i) => `${i.path.join(".") || "root"}: ${i.message}`).join("\n");

    messages.push({ role: "assistant", content: raw });
    messages.push({ role: "user", content: `That JSON is invalid:\n${problems}\nReturn the corrected JSON object only.` });
  }
  throw new Error("Model did not return valid ticket JSON after retries");
}

Why this works

• JSON mode removes the most common failure: chatty text around the JSON.
• Schema validation catches the subtle failures: wrong types, missing fields, values outside the allowed set.
• Error feedback turns a retry from a dice roll into a correction. One retry fixes the large majority of remaining problems.
• A retry cap and a thrown error mean bad data never slips through quietly. Your caller decides what to do: queue for a human, fall back to a default, or show an error.

Going further

• Native structured outputs. Some providers accept a full JSON Schema (response_format: { type: "json_schema", ... }) and constrain the model to it. Use it where it's available. Keep the Zod validation anyway: it costs microseconds and protects you if you switch providers.
• Log every failure. Save the raw output and the validation errors. Patterns in failures show you where your prompt or schema is unclear.
• Validate tool-call arguments the same way. Agent tool calls are JSON written by the model too, so the same schema-and-retry pattern applies.

The rule to remember: never trust model output as data until your code has checked it. A few lines of schema are the cheapest reliability you'll ever add.