OptiWorld: planning better futures

Optimal Control for Video World Generation under Physical Constraints

A video world model can imagine what happens next. OptiWorld adds a decision layer that shapes the motion it is asked to render.

OakMind / SixteenMiles Labs · NeurIPS 2026 · Preprint submitted May 30, 2026

Project & demos ↗ Read the paper ↗ PDF ↗

OptiWorld overview showing how optimized trajectories guide physically constrained video generation — OptiWorld overview. Figure from the original project page.

Choosing the motion before rendering it

Convincing pixels do not automatically give an object a good route through a scene. OptiWorld brings classical optimal control into video generation at inference time: recover a task-relevant state, optimize a trajectory under physical constraints, then use it to guide rendering. The framework targets goal-conditioned generation, motion editing, and counterfactual futures. Paper abstract ↗

Understand, plan, generate

Understand the scene. Geometry, segmentation, and vision-language reasoning assemble a compact 3D state. Editing also uses tracks from the source video.
Plan the motion. A continuous manifold expresses goals, hazards, and efficiency as geometry. Risky regions become costly to traverse; smoothness regularizes the path. A voxel-graph seed is refined into a continuous trajectory.
Generate the video. The first frame, prompt, and optimized 3D tracks guide a controllable generator. The paper uses Diffusion as Shader; the framework supports other generators with a compatible trajectory interface.

This shared planning geometry makes constraints comparable within one optimization. The modeled state is deliberately compact, rather than a complete physical simulation. Method in the paper ↗

Three-stage OptiWorld pipeline: extract the 3D world state, optimize a trajectory on a constraint-induced manifold, and render video with trajectory guidance — The understanding, planning, and generation pipeline. Figure from the project’s framework section.

Three ways to shape a future

Move an object toward a goal

Starting from one image, OptiWorld plans the selected object’s movement toward a target. The project provides six comparisons with Cosmos-Predict, VLIPP, HunyuanVideo, and Wan2.2, plus nine additional plan-and-video examples.

Edit the dynamics of a video

Source tracks anchor an optimized route that reduces unnecessary travel and abrupt motion. Five editing cases compare the source, Wan FLF2V, and OptiWorld, with track overlays showing the changes.

Explore counterfactuals

Changing a goal or safety constraint changes the planned future. The counterfactual gallery includes three scenes with alternative goals and five safety examples.

What the experiments report

The paper evaluates 60 images in OptiBench-I2V and 30 videos in OptiBench-V2V, drawn from DROID and collected scenes. Motion metrics use reconstructed 3D tracks; visual metrics use VBench. Experimental setup ↗

Goal-conditioned image-to-video generation
Method	Goal error ↓	Viol.@succ. ↓	Acceleration ↓	Jerk ↓	Energy ↓	Motion smoothness ↑	Background consistency ↑	Flicker score ↑
HunyuanVideo-1.5	0.548	0.523	0.0118	0.0197	0.0576	0.996	0.961	0.995
Wan2.2	0.549	0.523	0.0172	0.0304	0.0860	0.993	0.958	0.990
Cosmos-Predict2.5	0.733	0.286	0.0104	0.0186	0.0398	0.994	0.936	0.985
VLIPP	0.331	0.511	0.0099	0.0176	0.0419	0.994	0.964	0.992
OptiWorld	0.310	0.260	0.0070	0.0116	0.0126	0.997	0.982	0.996

Video dynamics editing
Method	Track deviation ↓	Acceleration ↓	Jerk ↓	Path length ↓	Energy ↓	Motion smoothness ↑	Background consistency ↑	Flicker score ↑
Source video	—	0.0124	0.0205	0.647	0.0269	0.993	0.969	0.988
Wan2.1 FLF2V	0.123	0.0157	0.0266	0.733	0.0387	0.995	0.957	0.993
OptiWorld	0.113	0.0107	0.0181	0.524	0.0236	0.996	0.967	0.992

Source: project I2V results and editing results. ↓ means lower is better; ↑ means higher is better. “Viol.@succ.” is the reported safety-violation measure. Flicker is a consistency score, so higher is better. See the paper’s metric definitions for evaluation details.

OptiWorld leads every displayed I2V metric. In editing, it improves motion metrics over Wan2.1 FLF2V, while the source retains the higher background-consistency score and Wan has the higher flicker score.

Our reading: an explicit, inspectable motion plan is a useful interface between scene understanding and visual synthesis. These benchmark results support that direction; they do not establish a general safety guarantee for generated scenes.

The research team

Yu Yuan¹, Jianhao Yuan², Xijun Wang¹, Daiqing Li³, Liu He¹, Lu Ling¹, Stanley H. Chan¹

¹ Purdue University
² University of Oxford
³ SixteenMiles Labs

Daiqing Li represents SixteenMiles Labs in this collaboration. The project thanks Yichen Sheng for discussion and credits the Nerfies website template.

Read, explore, and cite

Original project page — figures, interactive comparisons, and all demo cases.
arXiv:2606.00499 · PDF · Full HTML paper · TeX source · DOI.
Code repository — coming soon on the project page; not publicly accessible when checked.
Benchmark — coming soon; the project currently provides no separate download link. Check the project page for release updates.

Links and release status checked October 2, 2026. This article summarizes the authors’ work; the paper contains the full method, ablations, and benchmark details.

BibTeX

@article{Yuan_2026_OptiWorld,
  title={{OptiWorld}: Optimal Control for Video World
         Generation under Physical Constraints},
  author={Yuan, Yu and Yuan, Jianhao and Wang, Xijun
          and Li, Daiqing and He, Liu and Ling, Lu
          and Chan, Stanley H.},
  journal={arXiv preprint arXiv:2606.00499},
  year={2026}
}