PDF, audio and video

07PDF, audio and video inputs

Chat messages can carry more than text and images. Three additional content part shapes are accepted on POST /chat/completions, and every model can take them: where the provider reads the media natively it is passed through, everywhere else Rapid Router converts it server-side before routing.

{ "role": "user", "content": [
  { "type": "text", "text": "Summarise this document." },
  { "type": "file", "file": {
      "filename": "report.pdf",
      "file_data": "data:application/pdf;base64,JVBERi0x..." } }
] }

# file_data also accepts a plain https URL (or use file.file_url).
# Up to 20MB per file; multiple file parts per message are fine.

PDF. Google and Anthropic models read the PDF natively, pages, layout, images and all, so scanned documents work there. On every other provider the text is extracted server-side and delivered to the model as Contents of <filename>: ..., capped at 200,000 characters with a truncation note. A corrupt or empty PDF answers 400 rather than a garbled prompt.

One body, every model. The vendor-specific shapes are normalised at the gateway, so the same request works whichever model serves it. Anthropic's { "type": "document" } and { "type": "image" } blocks, Google's PDF-as-image_url data URL, and the standard image_url image part are all accepted on every model; Rapid Router rewrites each into whatever the serving provider takes. Swapping google/gemini-3.1-pro-preview for anthropic/claude-sonnet-5 needs no change to how the media is attached.

Audio. Google models hear the audio natively. For every other model the clip is transcribed by the cheapest connected speech-to-text model and the part becomes [Audio transcript]: ... in the prompt. The transcription is billed as its own line in your usage log at the STT model's normal per-minute price, on top of the chat request's tokens. If no speech-to-text provider is connected the request answers 400 saying so.

Video. Supported on Google models only, which watch the video from its URL. Any other model answers 400: video input is only supported on Google models. There is no server-side conversion for video.