Response healing

11Response healing

Models asked for JSON sometimes return almost-JSON: wrapped in a markdown fence, a trailing comma, or truncated mid-object by max_tokens. Add plugins: [{ "id": "response-healing" }] to a request that uses response_format and the gateway repairs those deterministically: fences stripped, trailing commas removed, truncated brackets and quotes closed. No second model call, no extra cost, no change to latency worth measuring.

{
  "model": "google/gemini-3.6-flash",
  "response_format": { "type": "json_object" },
  "plugins": [{ "id": "response-healing" }],
  "messages": [{ "role": "user", "content": "List 3 colours as JSON" }]
}

// response, only when a repair was actually made:
// "healing_applied": true

Valid JSON is never touched, requests without response_format are never touched, and a broken answer that cannot be repaired deterministically is returned as-is rather than guessed at. healing_applied: true appears only when a repair happened, so you can count how often your model actually needed it. Non-streaming requests only.