Backpropagation networks for forecasting | |
| Author | Songyot Sureerattanan |
| Call Number | AIT Diss. no.CS-00-02 |
| Subject(s) | Back propagation (Artificial intelligence) Forecasting |
| Note | A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Technical Science, School of Advanced Technologies |
| Publisher | Asian Institute of Technology |
| Series Statement | Dissertation ; no. CS-00-02 |
| Abstract | In recent years, artificial neural networks (ANNs) have been extensively used in various fields. Among them, backpropagation (BP) networks appear to be most p0pular and have been widely used in many applications such as forecasting and classification. To predict the future outcome values with an acceptable level of accuracy, a BP network has to be trained with a large sample of historical data that have been collected over a given time period. The BF network will then learn to generalize and extrapolate from new data to predict the future outcomes. However, there have been several problems encountered. One is how to determine the appropriate structure of the network for a particular problem. Another problem is its slew convergence (and no convergence in some cases), so that many iterations are required to train even a simple network. The structure of the network seriously affects the performance of the network model. As the network becomes more complex, the training time will increase. Therefore, the network should be kept as simple as possible. As the number of nodes in the input and output layers are application-dependent, the remaining problems are how to optimally choose the number of hidden layers and the number of hidden nodes. For many applications, they are determined by trial—and—error. Generally, when the number of parameters (the number of weights and biases) increases, the mean squared error (MSE) should be reduced. Therefore, it is difficult to determine the best network model by using only MSE. Instead, the Bayesian Information Criterion (BIC) was proposed in this study to select the best model from the candidate models having different numbers of parameters. It should be noted that the BIC penalizes the model for having more parameters and therefore tends to result in a smaller model. A new stopping rule was proposed to systematically determine the appropriate network structure using a procedure that gradually increases the network complexity until the current value of BIC is greater than the previous one or the decrease in the value of BIC becomes very small. Two new algorithms were devised to Speed up the convergence of BP networks: 0 The first proposed algorithm was obtained by applying the adaptive neural model with the temperature momentum term to the Kalman filter (KF) with the momentum term. 0 For advanced refinement, the nonlinear neural network problem can be partitioned into the nonlinear part in the weights of the hidden layers and the linear part in the weights of the output layer. By employing the conjugate gradient method for the nonlinear part and the KP algorithm for the linear part, we arrived at the second prOposed algorithm. After the weights of the hidden layers are obtained by using the conjugate gradient method, the weights of the output layer (in a linear problem) are readily solved by KF. The partitiOn allows the nonlinear and linear parts of the search to be conducted in a reduced dimensional space, resulting in acceleration of the training process. Consequently, the second proposed algorithm can greatly improve the convergence speed. From simulation experiments with three data sets; namely, daily streamflow (rainfall- runofi) data, quarterly data on exports and gross domestic product (GDP) of Thailand, and daily data on stock prices in Thai market, it was found that the BIC and these algorithms could perform satisfactorily in all cases considered. The BIC criterion and the two algorithms were introduced without any conditions. Consequently, they should be generally applicable to any type of data. |
| Year | 2000 |
| Corresponding Series Added Entry | Asian Institute of Technology. Dissertation ; no. CS-00-02 |
| Type | Dissertation |
| School | School of Advanced Technologies (SAT) |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Computer Science (CS) |
| Chairperson(s) | Huynh Ngoc Phien; |
| Examination Committee(s) | Tabucanon, Mario T.;Sadananda, Ramakoti;Oshita, Shinjiro;Tanaka, Kiyoshi; |
| Degree | Thesis (Ph.D.) - Asian Institute of Technology, 2000 |