The Toolkit·No. 24·September 4, 2026·Read by 100K+ subscribers
Gemini stopped watching the whole video
Google's new video mode reads the transcript first and only pulls the frames it needs. Here's the notes machine to build on top of it.
Welcome to the Toolkit, the Friday half of The AI Debrief. Tuesday is what happened. Friday is what to do about it: one tool worth your time, one play you can run today, and nothing you have to buy. Here's what we're getting into this week. Tuesday was about who controls the model menu. Today is smaller and more useful: Google quietly changed how Gemini watches video, and it turns the most expensive kind of input into the cheapest way to take notes.
Gemini agentic video understanding Every video you have ever handed to a model was processed the same way: sampled into frames at a fixed rate, every frame turned into tokens, whether or not anything happened in it. A one-hour screen recording of someone talking over a static slide cost the same as an hour of action. That is why "just summarize this video" has always been a bill you do not run twice. On September 1 Google switched on agentic video understanding across Gemini 3.8 Flash, 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. You flip one setting,
Google's numbers: up to 88% fewer total tokens and roughly 7% higher quality on long-form content versus the old static mode. Those are Google's numbers on Google-chosen videos, so hold them the way we held Cloudflare's last week. The mechanism is the part I believe, because it is the same trick that made every good agent cheaper this year: do not read everything, read the index, then read what matters. The parts that are not marketing. The navigation happens as thought tokens and whatever it loads is billed as tool-use tokens, so the bill floats with how hard your question is, and a vague prompt makes it wander. It is in Preview and Google says costs are likely to change. Limits: the 1M-context models take videos up to three hours at low resolution or one hour at high; files go in through the File API at 2 GB each on the free tier and 20 GB paid; YouTube URLs go in directly, capped at eight hours of video a day on the free tier, uncapped on paid, up to ten links in a single request. Who this is not for. Anything where every frame matters: QA of a UI, counting things, anything forensic. Static mode still exists for that and you should keep using it. This is for one question, "what did they say and when", asked of content too long to watch. Video understanding docs → · the Sep 1 changelog entry
What it replaces: a transcription subscription, or the forty minutes of scrubbing per video you do instead. What it costs after: nothing, as long as you stay inside eight hours of YouTube a day on the free tier. 1. Get a key. Google AI Studio, Get API key. The free tier is enough for everything below. 2. Write the prompt once, and make it demand timestamps. This is the whole play:
3. One call. Model 4. Save the reply as a Markdown file named after the video, into whichever folder your notes app already watches. Obsidian, Notion, a plain directory, it does not matter. The point is that the notes land where you will actually see them. 5. Batch it. Put your links in a list and loop, one call per video with a short pause between them. Up to ten links can share one request, which is how you ask cross-video questions: "which of these five tutorials actually shows the setup step?"
Why it fits the free tier: eight hours of YouTube a day is more than you watch. Static mode would have burned through that allowance on the first long video. Agentic mode skips, which is the whole reason this is a free play now and was not a month ago. What it saves: call it thirty minutes for every hour-long video you would otherwise scrub, plus the ones you were never going to get to.
The full script, the prompt, and a second version that watches a YouTube channel's feed and files notes on every new upload automatically are going in the vault.
Free. Same place the rest of the Toolkit builds live. If you build only one piece of this, run the single call on a video you have already watched. Compare the notes to what you remember. That tells you in five minutes whether to trust it with the ones you have not. Honest worst case: the preview pricing changes, and you own a script that costs a few cents more per video than it did this week. That's the Toolkit. See you Tuesday for the Debrief. From Drew and The AI Debrief team P.S. What's the video you keep meaning to watch and never do - a two-hour tutorial, a conference talk, an earnings call? Hit reply and tell us. We read every one. P.P.S. New here, or skipped one? Every past edition is in the archive. |