Abstract:  

Diffusion models have recently risen to prominence for a variety of inverse-imaging problems. Many such models use what is commonly known as conditional diffusion which effectively samples from the distribution of the desired image conditioned on some known side information, often a degraded or lower-resolution version of the target image. However, an alternative paradigm has recently emerged in the form of constrained diffusion in which explicit constraints between the target image and the side information are iteratively incorporated into the diffusion reconstruction during inference. While prior literature has considered conditional and constrained diffusion to effectively be mutually exclusive, I present a diffusion algorithm that combines the two within the widely-used denoising diffusion probabilistic models (DDPM) framework. The resulting approach – constrained conditional denoising diffusion – inputs both the target and side information into the diffusion network during both training and inference similar to conditional diffusion but also applies explicit constraints during inference like constrained diffusion. The proposed approach is evaluated for the task of fusing a hyperspectral image, possessing high spectral resolution, with a multispectral image, having high spatial resolution, to yield an image with high resolution both spatially and spectrally, an inverse-imaging problem called hyperspectral-multispectral fusion. Experimental results demonstrate that, not only can constrained and conditional diffusion operate complementarily and achieve performance superior to either used alone, but also that the proposed constrained conditional denoising diffusion outperforms other state-of-the-art approaches for hyperspectral-multispectral fusion.

Bio:

James E. Fowler received the B.S degree in computer and information science engineering and the M.S and Ph.D degrees in electrical engineering from The Ohio State University, Columbus, OH, USA, in 1990, 1992, and 1996, respectively. He was a program director in the Computer & Information Science & Engineering (CISE) directorate of the U.S National Science Foundation (NSF) from 2022-2025. He is currently the head of the Department of Electrical & Computer Engineering, Mississippi State University, Starkville, MS, USA, where he is a William L. Giles Distinguished Professor and also holds the James W. Bagley Chair. Dr. Fowler was previously the Chair of both the Computational Imaging Technical Committee and the Image, Video, and Multidimensional Signal Processing Technical Committee of the IEEE Signal Processing Society. He was a General Co-Chair of the 2014 IEEE International Conference on Image Processing, Paris, and is currently a General Chair of the Data Compression Conference. He was formerly the Editor-in-Chief of IEEE Signal Processing Letters and previously served as a Senior Area Editor of IEEE Transactions on Image Processing and as an Associate Editor of IEEE Transactions on Computational Imaging, IEEE Transactions on Image Processing, IEEE Transactions on Multimedia, and IEEE Signal Processing Letters. He is a Fellow of the IEEE.