Forecasting with Neural Networks – A Review

Raymond J. Ballard
Texas A&M University-Commerce

Source: www.nssa.us/nssajrnl/NSSJ2003%2020_2/html/03Ballard_Raymond.htm

Most forecasting techniques used today are based upon traditional linear or nonlinear statistical models, such as regression analysis. Although these models are useful and have been utilized for many years to predict, the models are somewhat limited in their ability to forecast in certain situations. Given the changing nature of technology and the globalization of business and financial markets, it is becoming increasingly important to be able to more quickly and accurately predict trends and patterns in data in order to maintain competitiveness. More specifically, it is becoming increasingly important for forecasting models today to be able to detect nonlinear relationships while allowing for high levels of noisy data and chaotic components.

In the light of changing business environments, managers are seeing the need for more flexible forecasting models. Specifically, forecasting models need to better allow for processing large amounts of data originating from many sources and many locations. Currently, however, most of the processing of information that occurs in business is done by utilizing computers that process information sequentially from a single central processing unit. This processing unit generates a result directly from the hard coding of the problem into the computer by a programmer. Due to the need for improved and advanced processing, businesses have turned their focus to the idea that there is the potential for information processing to take place through mechanisms other than traditional models.

One specific method of information processing that is being focused upon today utilizes a system that mirrors the organization and structure of the human nervous system. Such a system would consist of "parallel" computers equipped with multiple processing elements aligned to operate in parallel to process information. This method of forecasting is referred to as neural networking and is accomplished through a forecasting tool that has become known as an artificial neural network (ANN). Such networks have been used in the medical, science, and engineering fields. Now this technology is expanding to additional disciplines and is increasing in popularity as a superior forecasting tool in today’s business environment.


The Structure of A Neural Network:

A neural network is an information processing model based upon the functioning of neurons, or nerve cells, found in the human brain and nervous system, researchers are able to create an information processing system for use in forecasting that operates in the same manner as the human nervous system. Scientists have learned that the human brain holds over 100 billion neurons connected by fibers that form networks which are then used to transfer signals and process information. In a neural network model, information is processed through interconnected networks that work to transfer information through a signaling process. The most common type of neural network model utilized today is known as the "back propagation network" due to the back propagation algorithm used in the calculations within the model. These back propagation neural networks are made up of three layers of neurons; input layers, hidden layers, and output layers. Also within this model are assigned weights representing the knowledge base of the system and a transfer function that is used to process the data and represents the nonlinear properties of the neuron.

Although these three layers, the assigned weights, and the transfer function make up the typical neural network structure, there are many choices when designing such a network. For example, all layers and neurons can vary in number depending on the characteristics, such as amount and nature of the data to be used and the desired output result. More specifically, although using one hidden layer is adequate to map any input/output relationship, it may not be the most appropriate design in certain forecasting circumstances. This is also the case with the output layer.


Operation of a Neural Network:

The input layer is the first layer of neurons and is where all of the known external variables and data is input. Each input neuron represents a separate variable and various line connectors join these input layers with the middle, or hidden layers of the network. The connectors are assigned weights according to the level of importance since they are the location of the knowledge pool that exists within the network. The basic process that occurs between the input and hidden layers begins with the inputting of the external data. The values of this input data are then multiplied by the appropriate weights, as determined by the back propagation algorithm, and summed within the hidden layer. This sum is then converted through a transfer function into an output value. This output value is in the last layer, known as the output layer and typically contains only one neuron since only one output is usually requested. One output request is typically recommended because determining more than one output has proven to generate less than desirable results.

Since the neural network operates to process data in the same manner as the human brain functions, the neural network needs the ability to "learn" information as opposed to being "programmed." This learning ability of the neural network is accomplished through intense training of the network by providing it with numerous, reliable, and correct examples. It is essential that the examples provided to the neural network be accurate and complete since this is the only way to reach a desirable output result. As this learning process takes place, the many examples are combined to produce a "training set" of data. For each example presented to the network, there is a mathematical value representing both the input data and the preferred output data.

During the training phase, the overall goal is to determine the most accurate weights to be assigned to the connector lines. Also during training, the output is computed repeatedly and the result is compared to the preferred output generated by the training data. Any variance is considered a training error and it is important for this training error to be as small as possible so that the forecasted output is reliable. In order to minimize this error, the originally assigned weights are adjusted until the error declines. This weight adjustment is accomplished through the use of algorithm. By adjusting the weights, the error is minimized continually until a point is reached that represents the least amount of acceptable error. At this point an accurate forecast can be produced.


Advantages and Disadvantages of Neural Networks:

The purpose of using neural networks is to be able to forecast data patterns that are too complex for the traditional statistical models. Although neural networks are quickly becoming the wave of the future for forecasting, they continue to have both advantages and disadvantages. A strong advantage of neural networks is that, when properly trained, they can be considered experts with regard to the particular project for which they were designed to examine. This network structure can even be used to "provide projections given new situations and answer "what if" questions."

In addition, the learning ability of neural networks allows them to adjust to dynamic and changing market environments and is a much more flexible forecasting tool than traditional statistical models. An example of this level of flexibility is in the area of forecasting net asset values of mutual funds. When compared to regression analysis, neural network forecasting in this area was shown to generate a 40% increase in accuracy. This higher performance level is mainly attributed to the flexibility of the neural network and the ability to take into account all variables, internal and external, as well as the relationships between them. Furthermore, neural network systems are able to detect patterns and trends in any set of data they are given, including highly unorganized and variable data. As such, many businesses are continuing to increase the usage of neural networks in various areas of their operations in order to increase and improve their competitiveness.

There are, however critics who point out the disadvantages of using neural networks as forecasting tools. First, the design of the neural network is a very complex procedure that still relies mostly on trial and error. This is due to the necessity to determine the appropriate input variables and the necessary level of training of the system. The training process is time-consuming, and must be continuously repeated to account for changes in values of the variables. Without the recurring retraining, the accuracy of the neural network will decline. In addition, because the neural network can only produce accurate results if provided with an accurate training set, it is necessary to use large volumes of precise examples in the training phase. It is also important for the network designer to recognize the possibility of over-training when designing the neural network. This "over-fitting" occurs as a result of the large amounts and chaotic nature of the data coupled with few training sets. The danger here is that if over-fitting occurs, the neural network forecast will be too inaccurate.

Neural networks are also criticized because of the unstable nature of problem solving and the often inability to repeat a process and obtain the same results. Because the data are constantly changing, it is very difficult to repeat a solution to a problem. This, therefore, makes it very difficult to follow the methods the network is using to reach the output forecast. Thus, the most often disadvantage of the neural network is the inherent "black-box" nature of its’ operations. Neural networks, although able to generate a solution to many problems, are unable to explain how they arrive at their results. As such, neural networks are considered "black boxes" whose "rules of operation are completely unknown." This causes many to be wary of relying heavily on the results from a system they cannot truly understand.


Applications in Business:

The use of neural networks in business environments has been increasing over the last few years. Many areas of business, especially finance, utilize neural networks to improve forecasting of their business applications and to create new methods of evaluating financial data and investment decisions.

Neural networks are being used specifically by companies for improved forecasting capabilities in analysis of the stock market. Neural network systems are being used to predict short-term stock performance. For example, a neural network system referred to as the "Short Term Stock Selector" is available for use through an interactive web site. This web site utilizes neural network software applications to generate a decision regarding the near-term trading of a particular stock. More specifically, this service bases its information and predictions on a powerfully trained neural network system. Along the same lines, neural network systems have also been trained to forecast the performance of the Standard & Poors 500. For example, these systems have been specifically trained from historical data to notify the user with a buy signal if the index is predicted to go up. Currently, the neural network software used in this area is on a constant and continuous training schedule in order to keep the most recent activity of the S&P 500 in its training data.

Another example of neural network application in the stock market involves Daiwa Securities Company. This company utilizes neural network technology to learn and recognize stock price patterns for use in price forecasting. Currently, the neural network systems used by Daiwa Securities Company has been trained to analyze 1,134 company’s stock price patterns. This information is then used to forecast future stock price fluctuations for these particular companies.

In addition, neural network technology has also been applied to forecasting activities for entire financial markets and financial indexes. O’Sullivan Brothers Investments. Ltd. in Connecticut has utilized neural network software to follow over twenty different financial markets and generate reports regarding the behaviour of any of the studied markets. More specifically, this particular neural network system will generate information such as the probability of a certain price occurring in the next time period, as well as the most optimal target long and short-term prices. Still another product of this particular neural network can notify the user as to what level must be reached for the overall market to experience an increase. Similarly, in the futures market, predictions are also being made with neural network systems. Neural network technology has been applied in predicting corn futures. In such applications, the study concluded that when compared to traditional forecasting models used in this area, the neural network produced between 18 and 40 percent fewer errors.

Neural networks have also been used in determining bond ratings. Traditionally, bond ratings have been chosen as a result of statistical regression analysis. By utilizing neural networks to determine a bond’s rating, additional important factors affecting the potential default risk, can be accounted for. Unlike statistical regression analysis, the less obvious variables affecting default risk can be taken into account. Studies show this method of determining bond ratings has provided between 95% and 100% accuracy, while statistical regression analysis has been 85% accurate. In addition to bond ratings, bond prices can also be predicted with neural networks. Currently, G.R. Pugh & Company of New Jersey utilizes neural networks to predict bond prices of 115 public utility companies. The company relies heavily on this technique when advising clients of potential investment opportunities.

Bank loan decisions are another area in which neural networks are proving useful. Because the decision to make or deny a loan is very subjective or non-linear in nature, the use of neural networks resulted in a significant improvement in this decision-making process. By utilizing a neural network, the banker can rely on the network to "recognize certain similarities and patterns" and make a more accurate prediction with regard to potential default on a loan.

The ability to forecast server downtime has been advantageous to companies such as Computer Associates because such predictions make it possible for the company to fix any potential network problem prior to complete computer network failure.

Unlike the area of finance, there are currently few available and reliable studies that have been completed in the area of forecasting macroeconomic variables with neural networks. However, there appears to be a growing interest in using neural network systems to forecast macroeconomic variables. As a result, there are discussions regarding the need for studies that examine the effectiveness of neural networks in this area. One study that has been completed in this area looked at the forecasting accuracy of neural networks for Gross Domestic Product predictions. In this study, forecasting Gross Domestic Product with neural networks was proven to provide more accurate predictions when compared to traditional statistical forecasting techniques. Specifically, in the study completed by Greg Tkacz and Sarah Hu of the Bank of Canada, it was determined that on average, the best neural networks yield forecasts that art 15 to 19 percent more accurate than the corresponding linear models for the 4-quarter growth rate of real output.

In addition to applications mentioned above, neural network technology had been utilized in other areas of business. For example, neural network systems are being used by manufacturers to better determine adequate raw material levels and credit card companies are utilizing the technology for discovering and monitoring fraudulent activities. Sales forecasts are also being improved through neural network technology at both the wholesale and retail levels. Chase Manhattan Bank maintains a large neural network system that they utilize for determining the creditworthiness of various potential loan recipients. It is important for the bank to be better able to assess such creditworthiness since they are such a large supplier of funds for businesses.


Summary & Conclusion:

Neural network technology is another move toward increased efficiencies and improvements within the ever-increasing competition and global nature of today’s business environment. It is important for business to continue to adjust and adapt to changes is the business environment in which they operate and be able to deal with the dynamic nature of the global market. By utilizing neural networks, businesses are hoping to improve their abilities to predict patterns and trends in data. This would then allow them to better analyze business decisions, as well as offer improved products and services to their customers.

When compared to traditional forecasting models, the few reports currently available view neural network forecasting tools as more accurate than traditional statistical models. It should be noted, however, that because of the fairly recent increase in usage of neural networks in the various fields of business, there are only a limited number of research studies available analyzing their accuracy. Furthermore, there remain many who promote the usage of neural network technology and many who are eager to point out the limitations of utilizing this forecasting tool. Overall, however, within the business world there seems to be more proponents for neural network technology than opponents.

As researchers have pointed out, it is important to remember that neural network technology does not have to be considered a replacement for the traditional forecasting tools. In other words, it does not have to be considered a take-it or leave-it approach to forecasting. Researchers have declared that the best way to view neural network systems is as an important complement to other forecasting tools. As noted in a recent study on macroeconomic forecasting neural networks should be viewed as an additional tool to be included in the toolbox of macroeconomic forecasters. With this in mind, utilizing neural network systems together with other forecasting techniques can be considered yet another valuable advancement in the age of technology.