Reliable JSON requires more than asking for JSON in prose. Define a schema with required fields and allowed values, use the provider's current structured-output feature when appropriate, parse and validate the response in code, and route failures to retry, correction, or human review before taking action.
Who this is for: Developers and technical operators connecting model output to forms, databases, reports, or automated workflows.
- Keep the schema as small and semantically precise as the downstream task permits.
- Treat schema conformance and factual correctness as two separate checks.
- Never let unvalidated model output directly trigger a consequential write or action.
Design the data contract first
Start from the receiving system, not from the model. Identify required fields, optional fields, allowed enum values, nested relationships, and null behavior. If priority can only be low, medium, or high, encode those choices. Avoid a free-text field when downstream code expects a stable category.
Field descriptions should explain meaning and evidence. A due_date field might mean a date explicitly stated in the source, not a model estimate. Decide whether missing evidence produces null, an omitted property, or a rejected response. Consistent absence rules prevent fake precision and reduce special cases in application code.
Use constrained generation carefully
Current model platforms provide ways to request output that follows a supplied schema. These features can substantially reduce syntax and shape errors, but capabilities, supported schema features, and request syntax vary. Check official provider documentation when implementing or updating the integration rather than relying on an old code sample.
A valid structure does not establish valid content. The model may return an allowed category that is unsupported by the source. Include evidence fields when traceability matters, and describe the grounding rule in the prompt. Keep untrusted source text clearly separated from system instructions so embedded text cannot redefine the schema or workflow.
Validate in application code
Parse the response with a JSON parser, then validate it with the same schema used by the application. Check types, ranges, enums, string lengths, and cross-field rules. A date can be syntactically valid but precede the source event; a total can be numeric but disagree with its line items.
Return structured error information rather than silently coercing unexpected values. Decide which errors permit a bounded retry and which require a person. Log the schema version, model configuration, validation outcome, and a privacy-safe reference to the input. Do not log sensitive raw content by default.
Design failure handling
A repair request can show validation errors and ask for a corrected object, but it should not run indefinitely. Set a small retry limit and preserve the original evidence. If a required fact is absent, retries cannot create it. The correct result may be a flagged record for human completion.
Downstream actions should use an explicit approval boundary. Extraction into a review queue is different from automatically updating payroll, sending a customer promise, or deleting a record. Test malformed input, prompt injection inside documents, missing fields, extra fields, and plausible but unsupported values before production use.
Extract an invoice for review
An accounts team wants invoice PDFs converted into draft records, but no record should be posted without validation and approval.
- Define a schema for vendor_name, invoice_number, currency, line_items, subtotal, tax, total, and source evidence locations.
- Mark unknown values as null and prohibit calculation of values that are not printed on the invoice.
- Request schema-constrained output using the current documented provider method.
- Validate types, allowed currency codes, arithmetic relationships, and required evidence in application code.
- Send valid drafts to an accountant review queue and send failed records with clear error reasons to manual entry.
Structured-output readiness checklist
Apply this checklist before model-generated data reaches another system.
- Schema: required, optional, enum, null, range, and nesting rules are explicit.
- Meaning: every field says whether it is extracted, inferred, calculated, or reviewer supplied.
- Validation: parser, schema checks, business rules, and evidence checks run outside the model.
- Failure path: retries are bounded and missing facts move to a person rather than fabrication.
- Action boundary: consequential writes require authorization after validation.
Common mistakes
- Equating valid JSON syntax with accurate extraction from the source.
- Using a broad string field where downstream code assumes a small set of values.
- Automatically repairing or coercing invalid values without recording what changed.
Try one
A model returns {status: 'approved', amount: 5000} in valid JSON, but the document contains no approval decision. What should the pipeline do?
It should reject or flag the record because schema validity does not supply missing evidence. A good response identifies the semantic validation failure, preserves the source, records an unsupported approval error, and routes the item to human review. Evaluation should not accept retrying until the model invents a differently worded approval or treating an allowed enum as proof.
Sources
- OpenAI structured outputs guideOfficial guidance for producing responses that conform to a JSON schema.
- Gemini API structured output guideOfficial Google reference for schema-constrained JSON responses.