COLM 2026 · NonAR-LM Workshop

Feature Evolution in
Diffusion Language Models

Meliksah Yorulmazlar · Justice Sefas · Seunghyun Yoo · Lulu Shao

Purdue University · UC San Diego · Rensselaer Polytechnic Institute

Explore how sparse autoencoder features change across denoising in Dream and LLaDA.

The same LLaDA feature across 16 bins: late MASK activation and persistent revealed activation, on a shared 0–100% incidence scale.
Same feature, different states · LLaDA L1 f4937 · Measured positive incidence

Explore the temporal atlas

  1. Choose a model

    Switch between Dream and LLaDA.

  2. Move through time

    Move the 16-bin slider. Feature positions remain fixed; activity changes.

  3. Inspect a feature

    Hover or tap a point for a short candidate description.

Interactive atlas34 × 34-inch posterDLM-Scope SAE codeRelated paper: DLM-ScopeAtlas UI source