When:
April 22, 2022 1:00 PM — 2:00 PM in-person
Where:
CDM 222
Speaker: Sathya Ravi, University of Illinois at Chicago, USA
Abstract:
Starting with the work of Einstein and Smoluchowski on Brownian motion, the mathematical theory of Stochastic Differential Equations (SDEs) was built by Ito and Stratonovich to describe the fluctuating motion of microscopic particles such as atoms, and gas molecules. In the current times, as data centric decisions permeate our everyday lives, it may not be a stretch to imagine that mathematical results in SDE literature might be useful for computational purposes. In this talk, I will discuss my recent work on various ways of integrating SDEs during deep network training for various generalization, personalization, and computational benefits. We will focus on two instances of utilizing SDEs in the context of training feedforward architectures: (i) efficient feature augmentation; (ii) low memory personalization. In the first part of the talk, I will introduce the concept of generators of a SDE, and then describe in detail how generators can be used as a generic add-on to any layer in the neural network to adjust the predictions based on other samples in the minibatch. I will discuss various applications in small sample settings such as few shot learning, point cloud processing, and variational segmentation. Interestingly, all this can be accomplished without solving a single SDE. In the second part of the talk, I will switch focus to generative tasks such as image generation. Using Information Bottleneck Principle, I will explain a novel correspondence between noise level (time) and layers of encoders of a deep generative model that we have identified. We show that this correspondence can then be used to expand the range of a generator for downstream tasks, with intermediate iterates of the SDE which can be efficiently obtained. I will discuss experiments on image generation task, and advantages of using range expanded generators on a downstream denoising task.
Bio:
Dr. Ravi holds a Bachelor’s degree in Computer Engineering from NIT, Trichy, India, M.S in Industrial and Systems Engineering, MA in Mathematics, and PhD in Computer Science all from University of Wisconsin, Madison. Dr. Ravi is interested in Numerical Optimization of Deep Learning systems and in using Deep Learning to solve vision problems efficiently. For more information about Dr. Ravi’s research work please visit his research lab.