Google Research's video co-director keeps characters consistent for minutes
Serguey Asael Shinder2 min readGoogle Research has described an "AI video co-director", a multi-agent framework that sits on top of its Gemini and Veo models and aims to produce coherent videos several minutes long. The post, published on 24 September, names four pieces of work behind it: Co-Director, due at COLM 2026; CANVAS, due at EMNLP 2026; and two more called A²RD and VQQA.
The problem it addresses is familiar to anyone who has chained generative video clips. Existing pipelines prompt each shot independently, so characters' clothes and settings shift from shot to shot, and an error early in the chain - a flawed asset, say - spoils everything built on it. Google frames this as a credit-assignment problem: when the final video fails, it is hard to tell which prompt caused it.
The co-director's answer is to plan globally rather than shot by shot. An orchestrator agent uses a multi-armed bandit to choose a creative configuration across three dimensions - strategy, narrative mode and aesthetic - and pushes that choice into the prompts of all the sub-agents. A pre-production agent turns it into a storyboard; production agents generate keyframes, video and audio. A multimodal model then judges the finished cut on the same three dimensions and feeds a reward back to the bandit, so the next attempt improves. CANVAS adds explicit memory: it keeps structured records of characters, locations and object states, and retrieves visual anchors from that memory to keep a scene consistent. Because the system only orchestrates Google's models, the post notes, its output carries the same SynthID watermarking.
What it means
The research is less about a new model than about software architecture around models. Keeping a story consistent is treated as state management - a record of what exists in the world, updated as the plot moves - plus a feedback loop that scores the whole result instead of each piece. Those are ideas any team building multi-step generation pipelines can borrow, whatever the medium: explicit shared state beats hoping the model remembers, and a judge that evaluates the end product catches failures that per-step checks miss.
The limits are the usual ones for a research post. The claims of "substantial gains" come from the authors' own evaluations, and two of the four papers are still forthcoming. An automated judge scoring its own system's output also deserves scrutiny; it is a useful optimisation signal, not the same thing as independent human evaluation.