Towards Expanding Relevance Vector Machines to Large Scale Datasets

Authors

Abstract

In this paper we develop and analyze methods for expanding automated learning of Relevance Vector Machines (RVM) to large scale text sets. RVM rely on Bayesian inference learning and while maintaining state-of-the-art performance, offer sparse and probabilistic solutions. However, efforts towards applying RVM to large scale sets have met with limited success in the past, due to computational constraints.

We propose a diversified set of divide-and-conquer approaches where decomposition techniques promote the definition of smaller working sets that permit the use of all training examples. The rationale is that by exploring incremental, ensemble and boosting strategies, it is possible to improve classification performance, taking advantage of the large training set available. Results on Reuters-21578 and RCV1 are presented, showing performance gains and maintaining sparse solutions that can be deployed in distributed environments.

Keywords

Large scale learning; text classification; relevance vector machines

Subject

Relevance Vector Machines

Related Project

CATCH - Inductive Inference for Large Scale Data Bases Text CATegorization

Journal

International Journal of Neural Systems, Vol. 18, #1, pp. 45-58, World Scientific Publishing Company, February 2008