Continuous diffusion models have long benefited from a powerful mathematical tool known as the probability flow ODE, a deterministic ordinary differential equation that reproduces the same marginal distributions as the underlying stochastic diffusion process. This property has enabled faster sampling techniques and consistency distillation, both now widely used in image generation. Whether an equivalent formulation could exist for discrete diffusion models, which operate on symbolic data such as text or code rather than continuous values, remained an open question.
This paper answers that question by introducing exponential residual flows (ERFs), a construction that augments each discrete token with a continuous residual variable evolving according to a piecewise-defined ODE. The discrete state changes precisely when this residual hits zero, and the authors formally prove that this dynamic preserves the marginal distributions of the original diffusion process. The vector field driving these equations depends on the forward and backward rate matrices of the given diffusion model, grounding the approach in the existing theoretical framework for discrete models.
Beyond the theoretical contribution, the authors derive a practical discretization of their sampler and apply it to two common variants of discrete diffusion: masked diffusion and uniform diffusion. The deterministic trajectories produced by ERFs also support consistency distillation, a technique that can sharply cut the number of generation steps required without compromising output quality.
Experiments on text and code generation benchmarks show measurable improvements over existing sampling approaches for this class of models. While the work remains foundational and is not tied to a deployed product, it fills a notable theoretical gap: discrete diffusion, seen as a potential alternative to autoregressive models for text generation, now has a deterministic framework comparable to the one that drove progress in continuous diffusion for images.