Mix Forcing

Mix Forcing

Causal Pre-Training via Mixture Attention for Interactive World Models

Core method of LingBot-World-Infinity

Zelin Gao1 Hao Ouyang2,‡ Jiapeng Zhu2 Qiuyu Wang2 Jingye Chen1 Zichen Liu1
Xing Zhu2 Yujun Shen2 Yinghao Xu1,†

1The Hong Kong University of Science and Technology (HKUST)

2Robbyant

‡Project Lead†Corresponding Author

Paper GitHub Code Hugging Face
A rollout from our causally pretrained video model.
MOMENTS ALONG THE WAYVideo time · click a frame to jump

Overview

Mix Forcing combines teacher forcing and bidirectional training to learn causal video generation while preserving pretrained visual priors. With chunk-wise prompt conditioning and four-step distillation, it enables autoregressive world simulation with camera control, online text updates, and one-hour rollouts. Trained with MoBA masks, the model generates video chunks using current inputs and cached history.

Mix Forcing generates successive video chunks from an initial image, camera poses, local prompts, and a shared environment prompt, using a causal video model with MoBA attention and a KV cache.

Mix the training modes

MoBA attention combines teacher forcing for causal next-chunk prediction with bidirectional training to preserve visual priors.

Condition each chunk

Chunk-wise cross-attention uses shared environment and current local prompts, without accessing future instructions.

Distill for rollout

Consistency and distribution matching distillation turn the causal initialization into a four-step autoregressive generator.

Qualitative Comparisons

Compare six interactive world models under the same scene and control sequence.
Choose camera-controlled exploration or online prompt interaction, then play the results together.

SELECT A SCENE
Scene prompt

00:00 / 00:00

Choose a scene, then play all six videos together.

Long-Horizon Interaction

A recorded interactive model rollout.
Follow the journey, or jump to a moment below.

ONE-HOUR INTERACTIVE ROLLOUT
MOMENTS ALONG THE WAYVideo time · click a frame to jump

Scene Revisiting

A landmark revisit sequence.The camera looks away, then returns to the waterfall and red foliage seen earlier.

A WaterfallB Red foliageSame sequence · actual frame timestamps
A→B→look away→B→A

Click a frame to jump to that moment in the video.

Citation

Download .bib 
@misc{gao2026mixforcing,
  title = {Mix Forcing: Causal Pre-Training via Mixture Attention for Interactive World Models},
  author = {Gao, Zelin and Ouyang, Hao and Zhu, Jiapeng and Wang, Qiuyu and Chen, Jingye and Liu, Zichen and Zhu, Xing and Shen, Yujun and Xu, Yinghao},
  year = {2026},
  eprint = {2610.33589},
  archivePrefix = {arXiv},
  primaryClass = {cs.CV},
  url = {https://arxiv.org/abs/2610.33589}
}

Demo