InfinityEdit: Infinite Video Editing
with a Lightweight Edit-Ignition Adapter

Yunze Tong1,∘, Mushui Liu1,2,∘,†, Canyu Zhao1, Shiyi Zhang2, Didi Zhu1, Peng Zhang1,2, Wanggui He2, Jinlong Liu2, Ying Chen2, Hao Jiang2,†, Pipei Huang2, Bo Zheng2
1Zhejiang University   2Alibaba Group
∘ Equal contribution   † Corresponding author
Paper arXiv Code BibTeX

One instruction after another, applied to a live stream — edits stack infinitely.

Abstract

With large pretrained models, existing methods have effectively improved instruction-based video editing. However, most of them rely on an in-place editing assumption: they align the edited video with the given source clip frame by frame over a fixed time span. This pattern fails for open-ended streams, e.g., restyling a live game or applying a camera move to an ongoing shot. In such cases, edits must extend to future frames as they arrive, rather than be applied to a static input clip. We study this setting and name it infinite video editing: given a preceding segment and an edit request, a model must generate the next segment that continues the stream while applying the requested edit. This process repeats as an unbounded sequence of edit instructions arrives. This task brings two challenges: the edit must be a faithful continuation rather than a frame-wise rewrite, and generation quality must remain stable as edits accumulate. To address them, we first design a data-collection pipeline for infinite video editing. Based on the collected data, we propose InfinityEdit, a lightweight edit adapter that equips a streaming video generator with unbounded editing ability. The adapter contains three attention modules. History cross-attention guides the denoising frames using the input frames. Temporal causal self-attention keeps temporal cues flowing only from earlier frames to later ones. Edit cross-attention injects the edit request into generation. During inference, the adapter is activated only in the chunk where an edit request arrives. Subsequent chunks are generated by the original model with a reset anchor frame. This scheme applies the edit while preserving the original model's infinite generation ability. Extensive experiments show that InfinityEdit faithfully continues the stream under each edit, and stays stable over unbounded edit sequences.

Single-Video Continued Editing

A single instruction applied to a source clip produces the next, edited segment that continues it. We group cases by edit type; hover (or tap) an instruction to read it in full.

Long Video Generation

Our model generates long videos while a few instructions are applied in sequence. Each row is one case: the left-most clip is the source, and every arrow applies a new instruction on top of the previous result.

Infinite Sequential Editing

Stacking many instructions on a single continuous stream. Each row is one case: the left-most clip is the source, and every arrow applies a new instruction on top of the previous result.

Comparison with Baselines

The same sequence of instructions applied to one source (zoom out → move up → zoom in). Each row is a different method; Ours is shown first. Hover (or tap) an instruction to read it in full.

† denotes pure T2V generation methods that do not take visual input; all other methods condition on the same source video.

Method

To achieve infinite video editing, InfinityEdit attaches a lightweight Edit-Ignition Adapter to a frozen streaming video generator. The backbone generates one latent video chunk at a time while conditioning on a multi-scale history window and an anchor frame. Adapter blocks are inserted after transformer layers and update the current-chunk features while leaving the history tokens unchanged.

Each adapter block has three stages. History Cross-Attention anchors leading denoising frames to the provided history. Temporal Causal Self-Attention propagates information forward along time. Edit Cross-Attention injects the edit instruction into all current-chunk tokens. During inference, the adapter ignites only the first chunk after an edit request arrives. Later chunks are continued by the frozen backbone with a sliding history window and a reset anchor frame, keeping memory bounded while allowing edits to stack over time.

InfinityEdit Edit-Ignition Adapter architecture

Architecture. The Edit-Ignition Adapter is composed of three attention modules: History Cross-Attention, Temporal Causal Self-Attention, and Edit Cross-Attention.

BibTeX

Coming soon.