What is Spatial AI?
Spatial AI refers to systems designed to reconstruct a volumetric representation of their surroundings, filling in missing elements while preserving the integrity of the result.
For instance, when provided with a photograph of a room, a model might generate the elements not visible in the image, such as the wall behind the photographer, the back of a chair, or the floor beneath a table. The coordinates of these additions are fixed, ensuring that when the original viewpoint is revisited, the chair remains in place and retains its form.
The term used for this concept is spatial intelligence.
Most systems of this type operate in a similar manner:
- Input can be a photograph, a short video, a textual description, or a rough layout of volumes;
- Each frame is linked to a specific point and angle of capture, creating a cohesive geometry from these connections;
- Areas not represented in any frame are filled in based on accumulated knowledge about the characteristics of rooms, streets, and objects;
- The final output is saved, allowing a return along a known path to yield the same view.
This class of systems is referred to as world models. Fei-Fei Li, a professor at Stanford University and the creator of ImageNet, summarized the requirements for these models into three key points:
- The generated space must not contradict geometric or physical laws;
- Input can be received in any form, even down to minimal gestures;
- For each action, the model must return the subsequent state of the world.
One of the first commercial products in this domain was launched in November 2025, shortly after Li's publication. World Labs introduced Marble, enabling users to export a prepared world using "Gaussian splats" or polygonal meshes.
Currently, there is no universal method for training spatial AI. While language models are trained on a single task—predicting the next token—there is no similarly straightforward method for three-dimensional data.
World Labs showcased their next development, Atlas, on September 1, 2026, with access granted to a limited circle of partners upon application.
How Do World Models Differ from Language Models?
Language and spatial models process different types of data. Text unfolds in a linear manner, where each subsequent word depends on its predecessor. In contrast, a spatial arrangement remains constant regardless of whether it appears in a frame.
This leads to different consistency requirements:
- A language model needs only to avoid contradicting previous statements;
- A spatial model must ensure geometric consistency: if the camera circles a room and returns to the starting point, the image must replicate the initial frame, or else a window might appear on the wrong wall.
Conceptually, the internal structure of language and spatial models is relatively similar: Atlas integrates input data into a unified picture and completes the continuation. The distinction lies in the attachment; every image is associated with a specific point and angle of capture, allowing all accumulated data to be constructed volumetrically rather than linearly. World Labs refers to this as spatial context.
As explained by Li, language consists of a sequence of symbols, while describing a world necessitates accounting for numerous interconnections: gravity influences movement, and the interaction of light with materials determines the color and brightness of objects. A world model must coherently reproduce geometry, physics, and environmental changes over time, requiring new approaches to data representation and training.
Evaluating results is also challenging—there is no standardized test for world models. When presenting Atlas, World Labs reported on two tasks: generating a view based on a specified camera trajectory and reconstructing a scene from several images. Both criteria do not assess whether the model accurately handles the physics of objects or long scenarios.
Why Do Language Models Get Confused in Space?
A transformer can confidently navigate a city route, even without a coherent map of the area.
In an experiment, researchers from MIT, Harvard, and Cornell trained a system on recordings of taxi trips through Manhattan. The model not only charted existing routes between unfamiliar points but also identified the shortest paths.
This success, however, conceals a gap: the task itself does not require a coherent map of the city. At each step, it suffices to identify one allowable turn at the current point, and neighboring intersections often share similar options. Thus, the model can respond correctly even without distinguishing between one fork and another.
While the route remained within familiar parameters, the gap was not apparent. However, when 1% of streets were closed, accuracy dropped from approximately 100% to 67%.
Following this, researchers reconstructed the map—not based on actual streets in Manhattan, but on the routes constructed by the model itself. This led to the identification of passages that do not exist and overpasses crossing other routes at arbitrary angles.
An unexpected observation emerged: systems trained on random trips built a more accurate representation than those that only saw optimal routes, as unpredictable trajectories included more turns.
The gap between human and machine remains evident in recent tests. On the MMSI-Bench dataset, humans achieved a score of 97%, while the strongest closed model scored around 40% and the best open model approximately 30%.
It is important not to generalize this finding to all neural networks. The Manhattan experiment evaluated a specially trained transformer, not universal chatbots. The internal structure of spatial models within such systems is not disclosed by developers.
Who Came Up with Training AI in Imaginary Environments?
The idea originated in psychology. British researcher Kenneth Craik proposed in 1943 that the brain maintains a miniature copy of reality to simulate the outcomes of various actions before deciding on a course of action.
In machine learning, this approach took shape in 2018. David Ha and Jürgen Schmidhuber developed a three-part system trained on arcade games:
- The first part compresses the screen image into a small set of numbers;
- The second part predicts what will happen next based on this set;
- The controller selects an action.
The controller was quite small—only 867 parameters.
Subsequently, the researchers paused the game. The agent continued training on images generated by the predictive component—within the system's own imagination rather than in the actual game.
This learned behavior strategy was then transferred back to the actual engine. In a shooter game requiring dodging fireballs, the autonomous assistant survived around 1100 steps with a passing threshold of 750. In a race on randomly generated tracks, the agent scored 906 points against a requirement of 900.
However, this approach also revealed vulnerabilities. While training in an imaginary environment, the system discovered a way to move such that monsters ceased to attack, even though the original arcade game continued firing.
This led to an engineering principle: the more accurately a model reproduces an environment, the easier it is for an autonomous assistant trained within it to find loopholes that do not exist in reality. Ha and Schmidhuber attempted to mitigate this artifact by introducing noise into the simulation, but a universally effective solution has yet to be found.
How Are Three-Dimensional Scenes Created from Photographs?
A typical image generator selects the next frame to appear plausible alongside the previous one. However, Atlas works differently: it tags each input image with the shooting location and the direction of the lens. The system then operates not with a set of images but with points distributed in space.
This leads to a significant distinction from its predecessors. Marble accepted a single image as input and generated the rest. In contrast, Atlas utilizes over 100 images, dramatically reducing the amount of generated content.
The difference between the models is evident in a specific example. Given a single photograph of a country house with a garden, the model might invent both the layout of the yard and what lies beyond the fence. However, when a second and third image are added, these fragments align with reality.
The more images involved, the less the model needs to invent. Source: World Labs.The output is provided in three forms: new perspectives, depth maps, and point clouds. The latter are compiled into "Gaussian splats," which can be loaded into game engines or mixed reality glasses.
World Labs measured the accuracy of reconstruction based on multiple images: Atlas achieved an average error of 25.3 compared to 28.7–47.7 for five specialized models. Independent assessments are not yet available.
The second skill being tested is camera control. The model must navigate to specified points: a command like "eight meters forward, turn right" should yield the same frame that would arise from real shooting from that location.
Which Companies are Developing Spatial AI?
Three main teams stand out in the sector:
World Labs. In January 2026, Fei-Fei Li's company launched a public interface for developers. In February, the project secured $1 billion with participation from AMD, Autodesk, Emerson Collective, Fidelity, Nvidia, and Sea. In July, World Labs acquired SceniX, a team focused on simulation for robotics.
Google DeepMind. The Genie 3 model, introduced in summer 2025, generates interactive environments in real-time: 24 frames per second at 720p resolution, allowing several minutes of continuous interaction. However, its visual memory is limited to about 60 seconds; scenes left off-screen for too long appear differently upon return.
The model was initially released as a limited research preview, with later open access via Google Labs. There is no developer interface for Genie 3.
AMI Labs. Yann LeCun left Meta in November 2025 and, along with Alexandre LeBrun, founded their startup in Paris. The company emerged from the shadows on March 10, 2026, with a seed round of $1.03 billion, aiming to create an architecture that learns from video and predicts environmental states rather than the next word.
It is too early to assess the competitive landscape in this sector: there are few commercial implementations, and no public usage statistics have been disclosed by any of the three developers.
Where are World Models Already Making an Impact?
The first real application is in autonomous transportation. On February 6, 2026, Waymo introduced its own world model based on Genie 3.
This simulator provides not only camera images but also lidar data—a complete set of signals that a vehicle receives on the road.
A key feature is counterfactual scenarios. Engineers take a trip record and modify it: how events would unfold if the vehicle had not yielded the right of way. The entire scene then unfolds differently, including the behavior of other road participants.
Similarly, external videos are processed. Footage from a dashcam or smartphone is transformed into an environment where the vehicle retraces its route.
Previous reconstruction methods using "Gaussian splats" (Waymo's former technique) did not allow for such flexibility: they fell apart if the modeled route deviated from the original footage. To ensure robustness against such deviations, the company adopted a generative model.
The second area is robotics. World Labs recorded two large spaces using a standard phone, extracting 24 frames from each recording. This was sufficient for the system to render everything that the robot's cameras would see on any route within such a space.
In July 2026, the company demonstrated the next step. Policies were trained entirely in simulation, without a single real-world example, and then transferred to physical robots. Several tasks—such as packaging a box with both hands, working with cables, and extracting thin objects from a pile—were autonomously executed for an hour without human intervention.
One real task transforms into thousands of training variations for the robot. Source: World Labs.The benefits of training in simulation are still unclear. Waymo does not disclose how it affects real-world safety metrics, and World Labs does not compare these policies with those trained using traditional methods.
Why Does a Realistic Video Diverge from Physics?
Researchers from Google DeepMind and the Bulgarian institute INSAIT recorded 66 domestic scenes using three cameras: showing how liquids flow, fabrics crumple, and objects collide. Each recording was cut off after three seconds, and the model was asked to fill in the next five seconds of footage, comparing the results with the actual continuation of the same episode.
This set of scenarios was termed Physics-IQ. The best of eight evaluated systems scored 29.5 out of 100. The sample included models available by early 2025: Sora, Runway Gen 3, Pika 1.0, Lumiere, Stable Video Diffusion, and VideoPoet participated.
A more significant observation was that no statistically significant correlation was found between the plausibility of the image and adherence to physics. A video that a multimodal model perceives as real footage violates mechanical laws just as frequently as a video with obvious flaws.
The reason lies in the very task posed to the video generator. It is configured to produce a sequence of frames that statistically resembles examples from the training set. This requirement is indifferent to whether a liquid retains its volume or whether a thrown object travels in an arc.
Developers acknowledge these limitations. Atlas was tested in two ways: generating a view based on a specified camera trajectory and reconstructing a scene from photographs. The physics of objects was not evaluated during these tests.
The gap between humans on spatial tasks and the difference from real footage in physical terms. Source: ForkLog based on Physics-IQ and MMSI-Bench data.Why Do Autonomous Agents Need Spatial AI?
An AI assistant working with text and interfaces can function without geometry. Booking a ticket or checking tables does not require knowledge of what lies beyond a door. However, the situation changes when the task extends into physical space.
For a robot, merely having a description of a room is insufficient. It needs answers to questions about what will happen after an action: will a shelf hold the weight of a box, or how will a disturbed object move? These answers are precisely what a world model provides.
The economics of training also shift. Capturing an environment on a phone is cheaper than manually constructing a three-dimensional scene, and from a single recording, thousands of variations can be generated for training: lighting, object placement, and background can all be set arbitrarily.
What a spatial model adds to an autonomous assistant. Source: ForkLog.The question of accuracy remains open. World Labs claims that the generative system itself reproduces the dynamics of contact between a manipulator and an object, citing successful transfers of grasping policies from simulation to real robots. However, there has yet to be an independent verification of these results.
There is no intersection with blockchain infrastructure in this direction. None of the developers have publicly announced plans to connect world models with on-chain calculations, although payment protocols for autonomous assistants are developing in parallel.