Українська   Русский
DonNTU   Masters' portal

Abstract

This master's work is not completed yet. Final completion: May 2020. The full text of the work and materials on the topic can be obtained from the author or his head after this date.

Content

Introduction

Neural networks are one of the areas of research in the field of artificial intelligence, aimed at reproducing the human nervous system. Namely: the ability of the nervous system to learn and correct mistakes.

Neural networks (artificial neural network) are a system of connected and interacting simple processors. Such processors are usually quite simple. Each processor of such a network deals only with the signals that it periodically receives and the signals that it periodically sends to other processors. And, nevertheless, being connected to a sufficiently large network with controlled interaction, these processors together are capable of performing rather complex tasks.

1.Relevance of the topic

The advantage of large artificial neural networks over classical codes with error detection and correction is due to the fact that at the time of training, they are able to take into account the real distributions of multidimensional probabilities of biometric data, whereas all classical codes with detection and correction of errors were built in the hypothesis of an equiprobable distribution of errors.

2. Types of Artificial Neural Networks

The neural network consists of two main layers - receiving (it is also distribution) signals and processing. However, if a neural network consists only of these two layers, then it is single-layer, if there are more layers, then it is multi-layer [1].

Single-layer ANN (Fig. 1) - incoming signals are immediately transmitted from the input layer to the output layer, which processes them and gives a finished result. In the image, the distribution layer is shown in circles, and the processing layer is shown in squares.

Picture 1 — Single Layer Neural Network

A multilayer ANN (Fig. 2) is a network that consists of input, hidden, and processing layers. The signal from the distribution layer is partially processed by the hidden layer, after which it is transmitted to the last layer of neurons, which calculates the final result.

Picture 2 — Multilayer neural network

Scientists have learned to teach hidden layers of ANNs recently and this is a big step forward, since multilayer neural networks superior in performance and capabilities single-layer.

Networks work in two directions - direct distribution and reverse. Direct distribution ANNs make it possible to successfully solve most of the problems: forecasting, clustering, and recognition. In such neural networks, the signal is transmitted only forward, it is not possible to return back.

In such networks, part of the signal from neurons can be returned back and this principle of operation significantly expands the capabilities of neural networks. Such ANNs may have short-term memory as in humans. (fig. 3)

Picture 3 — Feedback ANN

3. Ways to train neural networks

3.1 Teacher training

Teaching with a teacher (supervised learning) requires a complete set of tagged data for training the model at all stages of its construction.

The presence of a fully marked dataset means that each example in the training set corresponds to the answer that the algorithm should receive. Thus, a labeled dataset of flower photographs will teach the neural network, which depicts roses, daisies or daffodils. When the network receives a new photo, it will compare it with examples from the training dataset to predict the answer. [2].

Basically, teaching with a teacher is used to solve two types of problems: classification and regression.

In classification problems, the algorithm predicts discrete values ??corresponding to class numbers to which the objects belong. In the training dataset with photos of flowers, each image will have a corresponding label - “daisy”, “rose” or “buttercup”. The quality of the algorithm is evaluated by how accurately it can correctly classify new photos with daisies and scoops.

Regression objectives are related to continuous data. One example, linear regression, calculates the expected value of the variable y, given the specific values ??of x.

3.2 Обучение с частичным привлечением учителя [3].

Semi-supervised learning is characterized by its name: the training dataset contains both labeled and unallocated data. This method is especially useful when it is difficult to extract important features from the data or mark out all objects - a laborious task.

This machine learning method is common for analyzing medical images, such as computed tomography scans or MRI scans. An experienced radiologist can mark out a small subset of scans that show tumors and diseases. But manually marking all scans is too time-consuming and expensive task. Nevertheless, a neural network can extract information from a small fraction of the labeled data and improve the accuracy of predictions compared to a model that learns exclusively from unlabeled data.

A popular training method that requires a small set of tagged data is to use a generative-competitive network or GAN.

Imagine a competition between two neural networks, where each is trying to outwit the other. This is GAN. One of the networks, the generator, is trying to create new data objects that mimic the training set. Another network, the discriminator, evaluates whether this generated data is real or fake. The networks interact and improve cyclically, as the discriminator tries to better separate the fakes from the originals, and the generator tries to create convincing fakes.

3.3 Reinforcement training

Training with reinforcement (reinforcement learning) operates on the principle of the game, upon reaching a specific goal receives a reward. Video Games - A Popular Testing Environment for Research.

AI agents are trying to find the best way to achieve a goal or improve performance for a particular environment. When an agent takes actions that contribute to the goal, he receives a reward. The global goal is to predict the next steps in order to earn the maximum reward in the end.

When making a decision, the agent studies the feedback, new tactics and decisions that can lead to greater gains. This approach uses a long-term strategy - just like in chess: the next best move may not help win in the long run. Therefore, the agent is trying to maximize the total reward.

The more levels with feedback, the better the agent’s strategy becomes. This approach is especially useful for training robots that drive autonomous vehicles or inventory in a warehouse.

3.4 Teacherless Learning

Perfectly marked and clean data is not easy to get. Therefore, sometimes the task of the algorithm is to find unknown answers in advance. This is where training without a teacher is needed.

In unsupervised learning, the model has a data set, and there is no explicit indication of what to do with it. The neural network tries to independently find correlations in the data, extracting useful features and analyzing them.

Depending on the task, the model organizes the data in different ways.

  1. Clustering The most common task for learning without a teacher. The algorithm selects similar data, finding common features, and group them together;
  2. Associations. Some characteristics of the object correlate with other features. By considering a couple of key features of an object, a model can predict others with which there is a connection;
  3. Auto encoders accept input data, encode them, and then try to recreate the initial data from the resulting code. There are not many real situations when using a simple auto encoder. But it’s worth adding layers and the possibilities will expand: using noisy and original versions of images for training, auto-encoders can remove noise from video data, images or medical scans to improve the quality of data..

In teaching without a teacher, it is difficult to calculate the accuracy of the algorithm, since there are no "correct answers" or labels in the data. But tagged data is often unreliable or too expensive to obtain. In such cases, by giving the model freedom of action to search for dependencies, good results can be obtained.

conclusions

The article describes the methods of training neural networks. Each type of training has its own unique features that must be considered when choosing a method for training a neural network to solve the problem.

An analysis of existing types of artificial neural networks was performed. And also the structure of their implementation is given.

List of sources

  1. Уоссермен Ф. Нейрокомпьютерная техника: Теория и практика. 1992 г. 236 стр.
  2. Ежов А. А, Шумский С. А. Нейрокомпьютинг и его применение в экономике и бизнесе. М.: МИФИ, 1998. 224 с..
  3. Laurene Fausett, Laurene V. Fausett. Fundamentals of Neural Networks: Architectures, Algorithms, and Applications Prentice-Hall, 1994. 461 с..
  4. Степанов П. П. Искусственные нейронные сети // Молодой ученый. — 2017. — №4. — С. 185-18
  5. Тадеусевич Р., Боровик Б. Элементарное введение в технологию нейронных сетей с примерами программ. Горячая Линия - Телеком 2111 г. 408с.
  6. Funahashi K. On the Approximate Realization of Continuous Mappings by Neural Networks. Neural Networks, 1989, v. 2, № 3, pp.183-191.
  7. Саттон Р., Барто Э., Обучение с подкреплением. Бином. Лаборатория знаний 2017 г. 400 c.