Training echo estate neural network using harmony search algorithm

Abstract

Echo State Networks (ESN) are a special form of recurrent neural networks (RNNs), which allow for the black box modeling of nonlinear dynamical systems. A unique feature of an ESN is that a large number of neurons (the “reservoir”), whose synaptic connections are generated randomly, is used in such that only the connections from the reservoir to the output modified by learning. The computation of optimal weights can then be achieved by a simple linear regression in an offline manner. ESNs have been applied to a variety of tasks from time series prediction to dynamic pattern recognition with great success. In many tasks, however, an online adaptive learning of the output weights is required. Harmony Search (HS) algorithm shows good performance when the search space is large. Here we propose HS algorithm for training echo state network in an online manner. In our simulation experiments, the ESNs are trained for predicting of three different time series including Mackey-Glass, Lorenz chaotic and Rossler chaotic time series with four different algorithms including Recursive Least Squares (RLS-ESN), Particle Swarm Optimization (PSO-ESN), and our proposed methods (HS-ESN and HS-RLS-ESN). Simulation results show that HS-ESN is significantly the fastest algorithm for training ESN whereas can effectively meet the requirements of the output precision. HS-RLS-ESN algorithm firstly uses HS to close to solution region then it uses RLS to obtain less error. HS-RLS-ESN is slower than HS-ESN and faster than RLS-ESN, but its generality power is very close to RLS-ESN. © 2017 [International Journal of Artificial Intelligence].