The Graphics Processing Unit (GPU) originally designed for rendering graphics and which is difficult to program for other tasks, has since evolved into a device suitable for general-purpose computations. As a result graphics hardware has become progressively more attractive yielding unprecedented performance at a relatively low cost. Thus, it is the ideal candidate to accelerate a wide variety of data parallel tasks in many fields such as in Machine Learning (ML). As problems become more and more demanding, parallel implementations of learning algorithms are crucial for a useful application. In particular, the implementation of Neural Networks (NNs) in GPUs can significantly reduce the long training times during the learning process. In this paper we present a GPU parallel implementation of the Back-Propagation (BP) and Multiple Back-Propagation (MBP) algorithms, and describe the GPU kernels needed for this task. The results obtained on well-known benchmarks show faster training times and improved performances as compared to the implementation in traditional hardware, due to maximized floating-point throughput and memory bandwidth. Moreover, a preliminary GPU based Autonomous Training System (ATS) is developed which aims at automatically finding high-quality NNs-based solutions for a given problem.
Journal
International Journal of Neural Systems (IJNS), Vol. 21, #1, pp. 31-47, Hojjat Adeli, February 2011
DOI
Cited by
Year 2016 : 1 citations
Fazanaro, F. I., Soriano, D. C., Suyama, R., Madrid, M. K., de Oliveira, J. R., Muñoz, I. B., & Attux, R. (2016). Numerical characterization of nonlinear dynamical systems using parallel computing: The role of GPUs approach. Communications in Nonlinear Science and Numerical Simulation, 37, 143-162.
Year 2015 : 2 citations
Zhao, L., Lu, J., Chen, D. F., & Wang, W. (2015). The Research on the Multi-Sensor Information Fusion Identifying of Alcohol based on Modified PCA and ANN. International Journal of u-and e-Service, Science and Technology, 8(8), 55-64.
Wang, Y., Tang, P., An, H., Liu, Z., Wang, K., & Zhou, Y. (2015, November). Optimization and Analysis of Parallel Back Propagation Neural Network on GPU Using CUDA. In Neural Information Processing (pp. 156-163). Springer International Publishing.
Year 2014 : 1 citations
Menke, Nathan Benjamin, et al. "A retrospective analysis of the utility of an artificial neural network to predict ED volume." The American journal of emergency medicine 32.6 (2014): 614-617.
Year 2013 : 3 citations
S. Melih Nigdeli and M. Hasan Boduroglu. Active tendon control of torsionally irregular structures under near-fault ground motion excitation. Computer-Aided Civil and Infrastructure Engineering, 28(9):718–736, 2013.
F. Hejazi, I. Toloue, M. S. Jaafar, and J. Noorzaei. Optimization of earthquake energy dissipation system by genetic algorithm. Computer-Aided Civil and Infrastructure Engineering, 28(10):796–810, 2013.
Nigdeli, S. Melih, and M. Hasan Boduro?lu. "Active Tendon Control of Torsionally Irregular Structures under Near?Fault Ground Motion Excitation." Computer?Aided Civil and Infrastructure Engineering 28.9 (2013): 718-736.
Year 2012 : 4 citations
Karl Pauwels and Marc M. Van Hulle. Head-centric disparity and epipolar geometry estimation from a population of binocular energy neurons. International Journal of Neural Systems, 22(3), 2012.
Álvaro Herrero, Urko Zurutuza, and Emilio Corchado. A neural-visualization IDS for honeynet data. International Journal of Neural Systems, 22(2), 2012.
M. A. H. Akhand and K. Murase. Ensembles of neural networks based on the alteration of input feature values. International Journal of Neural Systems, 22(1):77-87, 2012.
Juan Pablo Balarini, Sergio Nesmachnow, and Martín Rodríguez. Facial recognition using neural networks over GPGPU. CLEI Electronic Journal, 15(3), 2012.
