AI World Models Have an Amnesia Problem. This One Hired a Programmer to Fix It.

Give any AI world model demo enough time and it will eventually contradict itself. A door you closed swings back open. A character you shot wanders back into frame, unbothered. A red car you've been staring at for two minutes quietly becomes blue. None of this is a bug in the usual sense. It's a structural consequence of how these systems work: they're not simulating a world, they're guessing what the next frame probably looks like, and "probably" has no obligation to remember what happened thirty seconds ago.

A small academic team thinks it has a real answer, and it's a strange one: stop asking the video model to remember anything at all, and put a coding agent in charge of the facts instead. The project is called Code World Model (CWM), out of Westlake University's AGI Lab, and the paper, titled "Code World Model: Coding Agent as World Brain," is a three-author effort by Yiwen Chen and Chi Zhang of Westlake and Guosheng Lin of Nanyang Technological University. It landed on arXiv on August 26, 2026, with code and weights released the same week. (One small note on provenance: the GitHub account hosting the code, buaacyw, reads like a reference to Beihang University, but the paper's actual institutional affiliations are Westlake and NTU. Worth knowing before you assume a URL tells you who did the work.)

Two jobs that shouldn't be one job

Most of what gets called a "world model" right now, Genie, Oasis, the assorted Minecraft-style demos, is an autoregressive video predictor. Autoregressive just means each new frame is generated by conditioning on the last handful of frames, the same trick an LLM uses to predict the next word from the words before it. That architecture is genuinely impressive at short range and structurally bad at long range, because there's no ledger anywhere in the system that says "this door is unlocked" or "this character is dead." The only memory the model has is whatever visual residue survives in its own recent output, and video models are about as reliable at holding onto that as you'd be trying to recall the exact shade of a stranger's jacket ten minutes after they walked past you.

CWM's answer is to stop asking one model to do both remembering and rendering. The system has two halves. A coding agent, an LLM operating in an agentic loop, plays the role the authors call the "world brain." When something happens in the simulated world, the agent reasons about it in plain language and then writes and runs actual code that updates a persistent world-state object: positions, inventory, health, whatever the rules of that particular world are. That object gets compiled into what the paper calls a proxy representation, a stripped-down, per-frame description of the scene (something closer to a top-down map with labeled bounding boxes than a picture). Only at that point does a video model take the proxy and turn it into an actual frame you'd recognize as a game or a video. In CWM's release, that renderer is a LoRA fine-tune, meaning a small set of additional trained weights layered on top of an existing model rather than a full retrain, built on MiniMax's flagship video generator, MiniMax-H3. It was trained on proxy representations paired frame-for-frame with real footage, so its only job is learning to turn an abstract layout into something convincing to look at.

If you've ever thought about how a video game actually works, this split should sound familiar, because it's the same one a game engine draws between logic and graphics. In a normal game, the engine is the authority: it knows exactly where everything is and enforces the rules, and the graphics pipeline just draws whatever the engine hands it. Nobody worries that a game's renderer might "forget" a door is open, because the renderer was never in charge of remembering it in the first place. CWM is an attempt to rebuild that same division entirely out of learned components: a language model standing in for the engine, a diffusion model standing in for the graphics card.

Here are four rollouts in the project page's "Coding agents update and control world states" section: the world brain keeps track of what's where, whatever art style the renderer is asked to draw it in.

Why the split actually matters

The reason this is more than an architecture diagram is that it separates two problems a pure video model is forced to solve at once: keeping the state of the world correct, and rendering that state convincingly. A video-only model has to do both from pixels alone, with nothing to check its own memory against, which is exactly why long rollouts degrade the way they do. Small errors compound because there's no ground truth anywhere in the loop, only the model's increasingly unreliable memory of its own prior guesses.

CWM's proxy layer functions something like a save file. The code side can always state precisely what actually happened, because it's just code, executing deterministically, the same way an actual game server would. That leaves the video model with a narrower and more tractable job: take a scene description and make it look real and stylistically coherent, which happens to be something video models are already fairly good at. It's also why the released demos show a scene's visual style shifting, say from photoreal to painterly, while the underlying objects and events stay identical. Style is a rendering decision. State is not, and under this design, the renderer never gets a vote on it.

Where this sits next to Genie, Oasis, and Marble

The obvious comparison is Google DeepMind's Genie 3, currently the highest-profile video-only world model around. DeepMind's own writeup claims Genie 3 holds interactive 720p environments together at 24fps for "several minutes," a real improvement over Genie 2's roughly 10-second ceiling. But DeepMind's own limitations section is careful to say the interaction window still tops out "at a few minutes, not hours," and independent coverage has documented physics glitches and outright hallucinations, people appearing to walk backward, objects quietly changing shape, that look a lot like symptoms of the exact memory problem CWM is trying to design around. Genie 3 also remains a closed research preview with no public release date, so there's no way to actually run the two systems head to head. This is an architectural comparison, not a benchmark table, and it's worth being upfront that no benchmark table currently exists to make it one.

Elsewhere in the field, Oasis, Decart and Etched's real-time playable Minecraft-style model, proved that an autoregressive, video-only world model could run interactively at all. It's a genuinely impressive demo of responsiveness and a textbook case of the same drift problem: it's proof that the system reacts, not that it remembers. World Labs' Marble, Fei-Fei Li's venture, takes a genuinely different route out of the same problem: it generates actual navigable 3D scene geometry instead of a video stream, which sidesteps memory loss by making the world a persistent 3D object rather than a sequence of guesses. The tradeoff is that Marble reads closer to a 3D reconstruction and generation tool than an anything-can-happen simulator. CWM occupies an odd middle point between the two: no real 3D geometry, but no reliance on visual memory either. Its state is explicit, inspectable, and, crucially, just code you could in principle read.

I'd flag one thing plainly, because it's the kind of detail that's easy to skate past in a paper this new: every result CWM reports is self-published. There's no independent evaluation comparing rollout length, or how quickly errors accumulate, against Genie 3 or Oasis under matched conditions. The paper had picked up 28 upvotes on Hugging Face's papers board at the time of writing, days after release. That's a genuinely interesting academic idea, not yet a claim that's survived contact with anyone outside the team that made it. Self-reported numbers in this field have a way of looking better in the paper than in someone else's hands, and there's no reason to assume CWM is the exception until someone actually checks.

The part that isn't a click and go download

The team released code, a paper, and weights, which is real openness and more than DeepMind has offered for Genie 3. And it isn't a couple of cherry-picked clips: the project page's gallery runs the same pipeline through dozens of characters and art styles.

But released isn't the same thing as accessible. The project's own setup instructions were tested on an H800, a single GPU with 80GB of VRAM, which is a data-center accelerator, not something sitting in most people's desktops. The H800 itself is Nvidia's export-compliant, cut-down version of the H100 that Chinese labs have largely standardized on since the more capable chip became hard to buy. The renderer weights are distributed as LoRA checkpoints rather than a full fine-tuned model, which keeps the download reasonably small, but you still need the full base MiniMax-H3 video model loaded before those checkpoints do anything. I'd expect a quantized, lower-VRAM path to show up eventually, since that's the usual second act for a release like this, but nothing like that is documented in the repo as of this writing. Anyone hoping to try this at home should plan on renting cloud compute, not booting it up locally this week.

What could still go wrong

It's worth being honest about what "released" means here: five commits and 44 stars on GitHub at the time of writing. This is days-old academic work wearing a flashy demo, not a mature toolkit, and the flashiness is doing a bit of the persuading. The persistent world state the coding agent maintains is only as reliable as the code that agent writes in the moment, and coding agents hallucinate and produce logic errors of their own. Trading visual drift for buggy generated game logic isn't obviously a win, it's a different failure mode, and the paper doesn't yet offer hard numbers on how often that second kind of error shows up. The proxy representation is also, by design, a lossy simplification of the world: whatever it doesn't specify, the renderer has to invent, which is precisely where visual inconsistency can sneak back in through the side door the team just spent the whole architecture trying to close.

Even so, the framing here is the right one, which is why it earned its own post instead of a line in a roundup. If AI-generated interactive worlds are ever going to be more than tech demos, "the video model is hallucinating physics" was always going to be the wall everyone eventually hit. You can't build a game, or a robotics training environment, which is the other obvious use case for a system like this, on top of something that can't reliably remember whether a door is open. Whether a general-purpose coding agent is the right brain for that job, as opposed to a more constrained symbolic engine or state machine built for exactly this, is a genuinely open question the paper doesn't settle. But CWM is one of the first releases to take "the video model shouldn't be the one doing the remembering" seriously enough to build an entire pipeline around it, and that's worth watching well before it's something you can run on a single consumer GPU.