Training Energy-Based Normalizing Flow with Score-Matching Objectives

” In this paper, we establish a connection between the parameterization of flow-based and energy-based models, and present a new flow-based modeling approach called energy-based normalizing flow (EBFlow). We demonstrate that by optimizing EBFlow with score-matching objectives, the computation of Jacobian determinants for linear transformations can be entirely bypassed. This feature enables the use of arbitrary linear layers in the construction of flow-based models without increasing the asymptotic complexity of each training iteration. In addition to the reduction in runtime, we enhance the training stability and empirical performance of EBFlow through a number of techniques developed for score-matching methods. Our experimental results demonstrate that our approach exhibits improved efficiency compared to maximum likelihood estimation, and outperforms the other flow-based models trained using score-matching methods in recent literature. “