Plan in latent state space, decode actions afterwards, verify with the world model.
World models such as LeWM plan with the Cross-Entropy Method: for every state–goal pair they search over raw
action sequences from scratch, querying the world model thousands of times per replanning step and reusing
nothing across queries. Action sequences are low-level and highly multimodal, since many of them realize
nearly the same state trajectory. Latent state trajectories are smoother and already organized by the
world model's encoder, so LeFlow learns its planning prior in that space instead.
1
Latent flow planner
A conditional rectified-flow model generates the interior of a latent path whose endpoints are clamped to
$z_{\text{start}}$ and $z_{\text{goal}}$. Its
near-straight transport needs only 16 Euler steps, and $N{=}64$ candidates are
sampled in parallel.
2
Inverse dynamics decoder
A lightweight MLP maps each latent transition, together with its displacement, to an executable action
chunk: $a_t = g_\omega([z_t,\, z_{t+1},\, z_{t+1}-z_t])$. The planner decides
where to go; the decoder decides how.
3
Rollout reranking
Each decoded action sequence is rolled out through the frozen LeWM predictor. The candidate whose
actual rollout lands closest to the goal is executed. This projects generated paths back onto the
latent manifold the world model can actually control.
Training objective
LeWM stays frozen. Only the flow planner $v_\psi$ and the decoder
$g_\omega$ are trained, jointly, on offline trajectories:
$$\mathcal{L} = \mathcal{L}_{\text{flow}} + \mathcal{L}_{\text{inv}} + \lambda_{\text{cons}}\,\mathcal{L}_{\text{cons}}, \qquad \mathcal{L}_{\text{cons}} = \mathbb{E}\,\big\lVert \mathrm{pred}_\phi(z_t, \hat a_t) - z_{t+1} \big\rVert_2^2$$
The consistency term decodes each generated transition, rolls it one step through the frozen predictor and
penalizes the gap. This distills the world model's dynamics into the planner at training time, while reranking
selects among samples at inference. A small weight ($\lambda_{\text{cons}}{=}0.1$) is
enough; a large one collapses diversity onto the predictor's one-step behavior.