Research
My research interests lie at the intersection between deep learning, generative modeling, and mathematics, currently working on discrete diffusion and reasoning tasks.
Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning Tasks
Preprint (under review), 2026
Abstract
Discrete diffusion models iteratively denoise a noisy sequence of categorical data such as text, enabling any-order generation with self-correction and in-filling capabilities. This makes them promising candidates for reasoning, planning, and constrained-generation tasks like coding and math. Yet masked and uniform diffusion erase a token's identity in a single corruption event, so their noisy states cannot explicitly represent partially resolved uncertainty in which several alternatives remain plausible. We close this gap with Binary Discrete Diffusion, a family of continuous-time discrete-diffusion processes whose noisy state is a set of candidate tokens per position, represented as a binary membership vector. Binary diffusion changes candidate sets through additions and removals, allowing information to be lost and recovered gradually while the diffusion state itself represents partial uncertainty. On Sudoku-Extreme, Boxoban, and graph coloring, the symmetric formulation of binary diffusion consistently outperforms strong, well-tuned masked and uniform baselines under matched architectures and controlled training, sampling, and tuning budgets, with its largest gains on the hardest instances, while remaining competitive on language modeling (Text8, TinyStories, OpenWebText). These results support candidate sets as an effective intermediate representation for iterative reasoning and show that the corruption kernel can materially change the computation learned by the reverse model.
BibTeX
@misc{casadonoguerales2026binarydiffusion, title = {Binary Discrete Diffusion: Candidate-Set Diffusion for Logic and Planning Tasks}, author = {Casado Noguerales, Rodrigo and von R{\"u}tte, Dimitri and Sch{\"o}lkopf, Bernhard and Hofmann, Thomas and Raoufi, Aran}, year = {2026}, eprint = {2610.28543}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2610.28543}, }
What Does a Discrete Diffusion Model Learn? One Reverse Process in Three Coordinates
Oral @ NeurIPS 2026 Workshop on Diffusion Language Models
Abstract
What does a discrete diffusion model actually learn? The literature trains denoisers, bridge plug-ins, and concrete scores, each deriving its own ELBO objective under different conventions. This fragmentation makes the object being learned unclear and ELBO values difficult to interpret and compare. All three methods learn the same object in different coordinates: the exact time-reversal of the noising process. We first prove the Oracle Distance identity: for any noising process, the negative ELBO is not just a bound on the likelihood; it equals the data entropy plus the path KL divergence from the reversed noising process to the learned one. Next, for token-factorizing noise, we show that denoiser, bridge plug-in, and concrete score are three coordinates of this reverse process, derive exact conversions between them, and identify the probability distribution learned by each. In particular, contrary to its standard interpretation, the bridge plug-in learns not a denoiser but the cavity distribution: the clean token given the noisy sequence excluding its own position. We confirm the theory by training all three coordinates with masked, uniform, and GIDD kernels on OpenWebText under a shared recipe, and by training uniform-diffusion solvers on four logic tasks. The learned coordinates agree after conversion, while confusing the cavity with the denoiser can severely degrade sampling quality. Together with exact boundary-term formulas, closed-form NELBO calibrations at initialization, and a clock interpretation of time schedules, these results provide a conceptual framework and a mechanical recipe for deriving, implementing, and comparing discrete diffusion processes.
BibTeX
@misc{casadonoguerales2026discretediffusion, title = {What Does a Discrete Diffusion Model Learn? One Reverse Process in Three Coordinates}, author = {Casado Noguerales, Rodrigo and Sch{\"o}lkopf, Bernhard and Hofmann, Thomas and Raoufi, Aran}, year = {2026}, eprint = {2607.05381}, archivePrefix = {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2607.05381}, }
