Google researchers explain why diffusion models can be creative instead of just copying their training data

Google researchers published a paper explaining why diffusion models, the technique underlying many modern image generators, can produce genuinely novel outputs instead of simply reproducing their training data, according to a post on the Google Research blog. Diffusion models work by learning to reverse a noise adding process, transforming random noise back into a coherent image, and the paper addresses a puzzle at the heart of that process: if a model learns to denoise perfectly based only on the examples it saw during training, it should in principle just reproduce those exact training examples rather than create new, meaningful images, a failure mode known as memorization. The researchers frame image generation as a task of manifold recovery, where the model must infer the shape of the underlying, hidden data manifold from only a finite number of training examples sampled from it, then generate new points on that manifold corresponding to novel images. The paper identifies a property called score smoothing, arising naturally from how neural networks are trained, as the key mechanism that lets a model interpolate between training examples to produce new, coherent images rather than only memorizing what it has already seen. The work, titled On the Interpolation Effect of Score Smoothing in Diffusion Models, was presented at the International Conference on Learning Representations in 2026, and offers a more rigorous theoretical account of a capability, creative generalization beyond training data, that has so far mostly been observed empirically rather than explained mechanistically.