LeCun's AMI Labs open-sources H-JEPA, a hierarchical world model that plans in abstract latent space and lifts maze success from 18% to 73%

Why it matters

【Key points】

AMI Labs (Advanced Machine Intelligence), the company founded by Yann LeCun, with NYU, INRIA Paris and Brown University, released "H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning" (arXiv 2610.06805) in early October, with code and all trained checkpoints under the MIT license. It extends LeCun's JEPA world models from a single timescale to a hierarchy: upper levels look further ahead and keep only slow, task-relevant state, while lower levels handle concrete actions.

【Verifiable】

Per the project page and paper, H-JEPA stacks action-conditioned JEPAs, each predicting farther ahead in its own learned latent space, trained jointly end to end with SIGReg against collapse. Planning runs top-down: the top level plans toward the goal, its predictions become subgoals for the level below, and the lowest level outputs primitive actions. Across Visual AntMaze, FourRoom, OGBench Cube and Push-T, hierarchical planning beats the flat LeWM baseline; on Visual AntMaze a three-level model lifts success from 18% to 73% with less planner compute. The authors list limits: deeper than two levels hurts on Push-T, manipulation data shows little selective abstraction, and DROID real-robot results are offline path fidelity, not closed-loop success. Hugging Face hosts 93 checkpoints across five settings.

【Our read】

The paper gives reproducible evidence for long-horizon planning in abstract latent space rather than by generating pixels: more abstraction, cheaper planning, which is LeCun's core argument against the autoregressive LLM route. The evidence is still simulated mazes and simple manipulation, with no closed-loop real-robot success, so general agents remain far off. AMI Labs releasing code and weights in full matters; whether the gains hold in closed-loop real-robot control is the test of whether this route can compete head-on.