Improving convergence of restricted Boltzmann machines via a learning adaptive step size

Authors

Abstract

Restricted Boltzmann Machines (RBMs) have recently received much attention due to their potential to integrate more complex and deeper architectures. Despite their success, in many applications, training an RBM remains a tricky task. In this paper we present a learning adaptive step size method which accelerates its convergence. The results for the MNIST database demonstrate that the proposed method can drastically reduce the time necessary to achieve a good RBM reconstruction error. Moreover, the technique excels the fixed learning rate configurations, regardless of the momentum term used.

Keywords

Restricted Boltzmann Machines, Deep Belief Networks, Deep learning, Adaptive step size

Subject

Deep learning, Deep Belief Networks, Restricted Boltzmann Machines

Conference

17th Iberoamerican Congress on Pattern Recognition (CIARP 2012), LNCS vol. 7441, pp: 511-518, September 2012

DOI