Total Variation Optimization Layers for Computer Vision

Raymond A. Yeh¹   Yuan-Ting Hu   Zhongzheng Ren   Alexander G. Schwing  

Toyota Technological Institute at Chicago¹   University of Illinois Urbana-Champaign  

Abstract

Optimization within a layer of a deep-net has emerged as a new direction for deep-net layer design. However, there are two main challenges when applying these layers to com- puter vision tasks: (a) which optimization problem within a layer is useful?; (b) how to ensure that computation within a layer remains efficient? To study question (a), in this work, we propose total variation (TV) minimization as a layer for computer vision. Motivated by the success of total variation in image processing, we hypothesize that TV as a layer pro- vides useful inductive bias for deep-nets too. We study this hypothesis on five computer vision tasks: image classifica- tion, weakly supervised object localization, edge-preserving smoothing, edge detection, and image denoising, improving over existing baselines. To achieve these results we had to address question (b): we developed a GPU-based projected- Newton method which is 37× faster than existing solutions

Materials

[Paper]

[Poster]

Code

[GitHub]

Presentation

Citation

@inproceedings{YehCVPR2022,
               author = {R.~A. Yeh and Y.-T. Hu and Z. Ren and A.~G. Schwing},
               title = {Total Variation Optimization Layers for Computer Vision},
               booktitle = {Proc. CVPR},
               year = {2022},
}