The Gemini API can now read long video by letting the model jump around the timeline and pull only the frames, audio, and transcript it needs, rather than sampling the whole clip up front. Google's video understanding documentation calls this agentic video understanding, and it changes how the model handles footage longer than a few minutes.
- Agentic mode navigates; static mode samples. Static processing extracts frames at a fixed 1 FPS across the entire clip in a single pass. Agentic mode instead explores the timeline, inspecting transcripts and adjusting frame rate and resolution on the fly based on the prompt.
- Google reports up to 88% fewer tokens and about 7% higher quality on long-form content. Both figures come from Google's documented measurements, not independent tests.
- Four models support it. Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash, and 3.5 Flash Lite handle agentic processing.
- Google recommends agentic mode as the starting point for long videos and queries aimed at specific moments. Static mode stays the pick for clips under five minutes or when you need frame-level precision across the whole clip.
- You can confirm agentic processing ran. The response exposes
processing_call and processing_result steps that show which segments the model loaded.
How agentic mode differs from static 1 FPS processing
In static mode, Gemini tokenizes every second of video regardless of the question. At default low resolution that runs about 100 tokens per second, or about 300 at high resolution, so a long clip fills the context window before the model answers anything.
Agentic mode drops the up-front pass. The model reasons about the prompt, requests the transcript or specific frames it needs, and synthesizes an answer from what it pulled. Google accounts for the navigation reasoning as thought tokens and the loaded frames, audio, and transcript as tool-use tokens, which is where the token savings on long content come from.
The two modes are not exclusive within a request. Google's examples set agentic mode on a long lecture and static mode on a short experiment clip in the same call, then ask the model to compare them.
Where static mode still fits
Agentic navigation can slightly increase time to first token on clips under five minutes, since the model does internal reasoning and load round-trips before it starts generating. Google flags static as the better choice there, and for any task that needs every frame across the full clip.
Static mode also keeps the fine-grained controls. Clipping intervals and custom frame-rate sampling, such as one frame every two seconds, are only supported in static processing, according to the documentation. Google recommends streaming or background execution for long videos or complex agentic prompts to keep the connection open and surface intermediate steps.
What agentic mode could mean for logging and archive search
For production teams, the payoff is cost and reach on footage that was impractical to feed a model whole. Models with a 1M context window can take video up to three hours long at low media resolution, and agentic mode makes querying that length viable without paying to tokenize every second.
That points toward footage logging, archive retrieval, compliance review, and clip discovery, where the useful question is usually about a few moments buried in hours of material. Google does not benchmark those specific workflows, so treat the fit as our read rather than a documented result. The processing_call and processing_result steps also give a reviewable trace of what the model actually inspected, which is worth having when an answer needs to be checked against the source.
We looked at Gemini inside Avid Media Composer through Google Cloud, and the same video-understanding push lands here at the API level, where anyone building a media tool can call it directly.
