RULE-GOVERNED GENERATIVE WORLDS

WorldRuler.

Interactive World Modeling under Configurable Rules

YOUR WORLD. YOUR RULES.Discover WorldRuler

Mingliang Zhai1,2Chuanhao Li1,†Jiaming Tan1,2Zhen Li1,4Yuwei Wu2Yunde Jia5Zian Meng3Yuanyang Yin3Kaipeng Zhang1,3

1 Alaya Lab2 Beijing Institute of Technology3 Shanghai Innovation Institute4 The University of Tokyo5 Shenzhen MSU-BIT University

† Corresponding author

01 / THE IDEA

A world that follows your rules.

Change what an action means. Keep what the world remembers.

A switch can open a door before it comes into view. A defeated monster can leave a key that is still there after you return. Generative worlds need to preserve the logic behind these interactions—and make its consequences visible.

WorldRuler connects rule execution, video generation, and perceptual feedback through an executable 3D rule graph.

A schematic rule chain: defeat a monster, collect its key, and open a door, with the rules and world state saved together.
RULE-STATE PERSISTENCE Save the rules and the state behind the next action.
01 / PERSIST

Resume the world.

Restore rule definitions, dependencies, and object states together—not just the last image.

02 / GROUND

Find the object.

Bind rule entities to visual instances, including objects that appear only later.

03 / REALIZE

Make the state visible.

Guide designated attributes while preserving compatible appearance, motion, and unrelated content.

02 / THE METHOD

Execute the rule. Realize the consequence.

A persistent rule graph and a visual generation loop connect symbolic state to what you see.

WorldRuler method: executable 3D rule graph, dynamic object binding, rule-guided generation, and perceptual feedback.
ARCHITECTURE Rule execution → object binding → video generation → perceptual feedback.
RULE GRAPH

Persistent, executable state.

Rules, dependencies, object identities, attributes, and spatial information live in a shared representation.

DYNAMIC BINDING

State before first sight.

Stored rule targets constrain newly visible objects from their first appearance, with feedback refining correspondences.

ATTRIBUTE REPLACEMENT

Local, targeted control.

A frozen reader estimates object attributes. A trained writer applies rule-directed corrections during generation.

Counterfactual reconstruction and attribute swapping

Counterfactual reconstruction and bidirectional attribute swapping train the replacement operator using paired videos. The video generator and attribute reader remain frozen.

Diagram of local feature corrections, counterfactual reconstruction, and bidirectional attribute swapping.

03 / IN THE WORLD

See the rules in action.

Compare GTA-V and Habitat videos with rules disabled and enabled.

Bullet impact → tank explosion

RULE-CONDITIONED GENERATION19 seconds
Rule disabled
Rule enabled
0:00 / 0:19

04 / A RULE CHAIN, VISUALIZED

From an action to a world state.

A temple walkthrough combines generated footage, reconstructed 3D geometry, and synchronized rule annotations.

GENERATED VIDEOFRAME 000
RECONSTRUCTED WORLD + RULES
01 · DEFEAT

Defeat goblin → Key appears

Key not visible
02 · COLLECT

Collect key → Carry key

Unconfirmed
03 · UNLOCK

Use key → Open gate

Unconfirmed
Loading 3D view…
Drag to orbit · Scroll to zoom · Shift / right-drag to pan · Touch: drag / pinch. Keyboard: arrow keys orbit, Shift and arrows pan, plus and minus zoom, Home resets the view.
PlayerGoblinKeyGate
Approach0.00 / 19.50 s

05 / THE DATASET

RuleVerse.

One world. Two possible outcomes.

Paired rule-driven interactions. Collected in Habitat and GTA-V.

Each pair shares the scene configuration and action intent, while a designated rule mechanism is enabled or disabled. The resulting videos capture different state evolutions, with annotations of states, events, and rule outcomes.

Scene-aware splits separate training and test groups. The collection pipeline covers object attributes, spatial relationships, and effects that begin outside the current view.

RuleVerse collection pipeline across Habitat and GTA-V, from scene grounding to annotated rule-enabled and rule-disabled video pairs.
DATA COLLECTION Shared scene and action intent. Controlled rule intervention. Paired observations.
3,200
Paired interactions
6,400
Videos
2,900 / 300
Training / test pairs
Habitat/GTA-V
Simulation environments
Dataset statistics

Rule combinations

Pairs (%)
Habitat
Trigger / effect Attribute Geometry Presence Topology
Action 7.3 7.8 4.9 4.5
Spatial 5.2 5.6 2.2 3.7
Relation 5.6 5.4 3.6 2.6
State 6.6 7.1 5.0 3.4
Sequence 5.9 5.9 3.7 3.9
GTA-V
Trigger / effect Attribute Geometry Presence Behavior
Action 15.3 10.9 2.9 12.7
Spatial 5.6 4.2 1.5 5.6
Relation 2.7 2.0 0.7 2.5
State 5.9 3.6 1.4 5.6
Sequence 5.6 4.3 1.3 5.7
Object bindingShare (%)
BindingHabitatGTA-V
Single-object18.541.1
Cross-object60.240.2
Environment21.318.7
Rule activation timingMedian · video progress (%)
EnvironmentMedian activation
Habitat65.5
GTA-V55.5
CITE THIS PROJECT PAGE
@article{zhai2026worldruler,
  title={WorldRuler: Interactive World Modeling under Configurable Rules},
  author={Mingliang Zhai and Chuanhao Li and Jiaming Tan and Zhen Li and Yuwei Wu and Yunde Jia and Zian Meng and Yuanyang Yin and Kaipeng Zhang},
  journal={arXiv preprint arXiv:2611.24869},
  year={2026}
}