01Definition
Instead of learning purely from trial and error in the real world — slow and risky for a robot or a self-driving car — an agent can "imagine" the outcome of an action inside its world model first, then plan or refine its behavior before ever touching real hardware.
02How it works
03How it's trained
Most world models are trained on large amounts of video or sensor data using a self-supervised prediction objective: given the recent past, predict what happens next. One family of models does this by generating the next video frame pixel by pixel, autoregressively, the way Genie 3 and GAIA-1 both work. A newer family, exemplified by Meta's V-JEPA 2, skips pixel generation entirely — it predicts the next state in a compressed, abstract "embedding" space instead, which is cheaper to compute and, its creators argue, closer to how the physical outcome actually needs to be represented for planning.
04Notable examples
Cosmos
A world foundation model platform announced at the same CES 2025 keynote that popularized "physical AI" — generates physics-based simulation video for training robots and autonomous vehicles, and is already used by Figure AI, Skild AI, and Agility Robotics.
Genie 3
A general-purpose foundation world model that generates a navigable, real-time interactive environment from a single text prompt — running at 24 frames per second and staying consistent for a few minutes, a marked jump over the earlier Genie 2.
GAIA-1
A world model built specifically for autonomous driving, generating realistic driving video from video, text, and action inputs to train and evaluate driving policies.
V-JEPA 2
A world model trained on over a million hours of video, used for zero-shot robot planning and manipulation without generating any video pixels itself.
05Why it matters
This matters because real-world trial and error doesn't scale: a robot can't crash a car or drop a fragile object thousands of times to learn. A world model lets an agent run that trial and error internally, compressed and safe, then apply what it learned.
06Related terms
07FAQ
Q.How is a world model different from a hand-built physics engine (like in a video game)?
A.A physics engine encodes explicit, hand-coded rules (gravity, collision, friction coefficients). A world model is learned from data instead — capturing messy real-world dynamics that would be impractical to hand-code, at the cost of being an approximation rather than an exact simulation.
Q.Are world models only used for self-driving cars?
A.No — they're used broadly across robotics and humanoid planning too, as the general-purpose Genie 3 and V-JEPA 2 examples show. GAIA-1 is a driving-specific example, not the only application.
Q.Does a world model need to generate video to be useful?
A.No. Generating video (as Genie 3 and GAIA-1 do) is useful for creating training environments and for humans to visually inspect what the model predicts, but V-JEPA 2 shows that predicting future states in an abstract embedding space — with no pixels at all — is enough for planning and control, and is cheaper to run.