Programmable
World Model

Alaya Lab Team

What

Programmable World Model is an explicit-state-driven framework: it decouples world-state maintenance and evolution from visual observation generation. Inside the loop, a state executor maintains the world state and advances it under player actions and world rules; a deterministic compiler projects the state, represented as state-augmented 3D OBBs, into pixel-aligned controls; a video model renders the observation; and a VLM-based agent writes the executable world specification from a reference image and open-ended language.

Why

Video world models are powerful interactive renderers, but plausible observations alone are not a world engine. Their state lives implicitly in visual histories, difficult to inspect, edit, or verify; controls stop at cameras and prompts without reaching entity-level state; and the world must persist beyond the current view, where off-screen entities and latent facts such as health or relations are silently forgotten. These hidden facts and rules can be expressed compactly in language, structured records, and executable code, exactly what an agent can construct and update.

What a world enables

Persistent state under events. The engine applies the death event to the canonical world state.
INPUTGENERATED
W
A
S
D
ENGINEt 000
Prompt

Third-person view of a misty jungle clearing after rain: wet mud, moss-covered roots and rocks, dense ferns and vines, soft light filtering through the canopy. Three yellow-furred minotaur-like creatures with horns wear grey tactical vests and carry rifles. The one in the foreground faces away from the camera, aiming ahead; two more stand across the clearing with weapons raised, facing it.

The world exists off-screen. A −130° pan reveals entities that were maintained in the world state all along.
INPUTGENERATED
W
A
S
D
ENGINEt 000
Prompt

A bald man in tactical gear aims a firearm at two other armed individuals on an airport tarmac. One of the targets is down on the ground, while the other is standing and firing back. The scene is set on a wide, paved area with airport equipment and distant mountains in the background. Third-person view: a man stands in the foreground at the center of the frame, back to the camera.

Rule-driven interaction. A vehicle enters the scene and stops exactly as the world program's event triggers dictate; the direction map tells the renderer the entity's own motion, separate from camera motion.
INPUTGENERATED
W
A
S
D
ENGINEt 000
Prompt

A military airfield at night, lit by cold floodlights and a sweeping searchlight beam. A large HANGAR 17 sign spans the buildings, an orange fire burns at the right edge, and a yellow RESTRICTED warning board stands on the wet apron below dark mountains. Third-person view: a man stands in the foreground at the center of the frame, back to the camera.

Open-domain worlds. The same OBB interface and state compiler drive an entirely different programmed world, preserving the flexibility of open-domain video generation.
INPUTGENERATED
W
A
S
D
ENGINEt 000
Prompt

A dark bay at night beneath storm clouds, motorboats on open water with a glittering city skyline on the horizon. Third-person chase view: a speedboat in the foreground at the center of the frame, seen from behind. The boat throttles up and speeds toward the city.

Long-horizon rollout. A 30-second autoregressive rollout with a crowd of entities — everyone enters, acts, and falls exactly as the engine's canonical state dictates, with no drift across 897 frames.
INPUTGENERATED
W
A
S
D
ENGINEt 000
Prompt

Three armed individuals are engaged in a firefight on a wet tarmac during sunset. One person is shooting while another is falling to the ground, and a third stands nearby aiming. In the background, airport buildings and an approaching airplane are visible. Third-person view: a man stands in the foreground at the center of the frame, back to the camera.

Gallery

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a luminous underwater Art Deco city with coral towers, glass domes, fish, glowing sea plants, and turquoise light shafts. Several non-human figures, each designed as a translucent river spirit with a humanoid silhouette, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as gouache-and-ink animation with graphic geometry and jewel tones.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a vast coral cavern lit by shafts of ocean light, colorful reefs, sea turtles, and submerged ruins. Several non-human figures, each designed as a plush toy-like bear guardian with fabric seams and button details, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as richly textured colored ink illustration with translucent layered washes.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a retro-futurist lunar colony with glass domes, crater ridges, Earthrise, and silent rover tracks. Several non-human figures, each designed as a paper-folded origami figure with crisp geometric limbs, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as mid-century science-fiction illustration with clean shapes and grainy gouache texture.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a moonlit European fantasy castle courtyard with ivy, fountains, lanterns, and deep blue sky. Several non-human figures, each designed as a porcelain mannequin with painted joints and a blank mask, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as romantic storybook gouache with delicate linework and silver-blue lighting.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a whimsical fairy-tale village inside a giant hollow tree, bridges, warm windows, moss, and tiny gardens. Several non-human figures, each designed as a glass-and-light holographic humanoid with no human facial features, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as intricate storybook ink-and-watercolor miniature illustration.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a vast red-rock desert canyon at sunset with mesas, wind-carved arches, and a distant caravan trail. Several non-human figures, each designed as a tall straw scarecrow with stitched cloth limbs, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as bold screen-printed travel-poster illustration with long cinematic shadows.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a snowy arctic fjord with icebergs, pine forests, frozen waterfalls, and luminous aurora ribbons. Several non-human figures, each designed as a sandstone golem carved with simple geometric motifs, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as soft pastel oil painting with luminous atmospheric perspective.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in rolling golden wheat fields under a huge summer sky, a windmill, dirt lanes, and distant blue hills. Several non-human figures, each designed as a sunflower-headed garden sentinel with leafy articulated limbs, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as classic pastoral oil illustration with luminous impasto and gentle animation contours.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a wide African savanna beneath dramatic storm clouds, acacia trees, grassland, and distant kopjes. Several non-human figures, each designed as a fox-masked festival spirit with a long tail and robe-like body, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as cinematic gouache landscape with energetic skies and naturalistic color.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a cozy mountain tea village in peak autumn, red maple trees, wooden houses, terraces, and morning mist. Several non-human figures, each designed as a theatrical wooden marionette with visible strings and joint knobs, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as cozy textured colored-pencil and watercolor illustration.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in an enchanted bioluminescent forest with giant mushrooms, glowing flowers, ancient roots, and firefly light. Several non-human figures, each designed as a glowing mushroom-folk wanderer with a cap-shaped head, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as dreamlike digital painting with luminous fantasy color and soft edges.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a layered paper-cutout seaside landscape with rolling waves, cliffs, sailboats, and a lighthouse. Several non-human figures, each designed as a tiny winged gargoyle guardian with a stone body, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as handcrafted paper collage animation with visible cut edges and soft shadows.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a floating steampunk sky-port above cloud seas, brass towers, airships, and sunset clouds. Several non-human figures, each designed as a retro-futurist astronaut in a rounded pressure suit, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as detailed brass-and-copper concept art with ink outlines and painterly light.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a serene floating lotus city above golden clouds, pagodas, waterfalls, and distant mountain peaks. Several non-human figures, each designed as a coral-bodied sea golem with shell armor and bioluminescent seams, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as ornate ink wash and silk-painting animation with elegant flowing forms.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a bright Mediterranean harbor with whitewashed houses, blue sea, fishing boats, bougainvillea, and cliffs. Several non-human figures, each designed as a bronze bird-headed sentinel with articulated wings folded at its sides, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as sunny plein-air watercolor with loose impressionist strokes.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a completely new elevated neon city in heavy rain with monorails, holographic billboards, and reflective streets. Several non-human figures, each designed as a whimsical mushroom-armored knight with an oversized helmet, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as stylized anime cyberpunk painting with saturated magenta and cyan light.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in an enormous Victorian glass greenhouse filled with exotic plants, fountains, winding paths, and sunbeams. Several non-human figures, each designed as a living stone statue with a carved helmet and rune-like markings, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as botanical engraving blended with luminous hand-painted animation.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a lush tropical rainforest after rain with a waterfall, broad leaves, orchids, and misty mountains. Several non-human figures, each designed as a moss-covered woodland guardian made of bark and vines, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as vibrant botanical watercolor with expressive brushwork.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a sunlit fantasy valley village with a winding river, green hills, giant trees, flower meadows, stone paths, and cottages. Several non-human figures, each designed as a friendly brass clockwork automaton, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as warm watercolor backgrounds and gentle hand-painted cel animation.

INPUTGENERATED
W
A
S
D
Prompt

Third-person cinematic view in a dramatic volcanic island coast with black sand, lava glow, palms, ocean spray, and storm clouds. Several non-human figures, each designed as a small dragon-person wearing a traveler cloak, clearly non-human, occupy the scene; the central figure faces away from the camera and holds a compact rifle. The figures move naturally while preserving their distinct positions. The camera moves smoothly while keeping the group framed. During the sequence, one figure falls to the ground and remains motionless. Rendered as dynamic matte-painting brushwork with fiery complementary colors.