RU UA EN
 
Resume Abstract

Abstract

When writing this essay, the master's work is not yet completed. Final completion: May 2020. The full text of the work and materials on the topic can be obtained from the author or his leader after the specified date.

Content

Introduction

Currently, the trend towards automation is gaining increasing speed. One of the areas in which automation has great prospects, is video analytics.

Using video analytics in retail can help evaluate the store’s advertising strategy and placement efficiency walkways and products. Thus, it becomes very important to place information about the promotion in the right place and at the right time.

There are various analytic options that can help a retail store get a quick idea of the effectiveness of advertising events, as well as the impact of the dynamics of customer flows on conversion in sales areas.

1. Theme urgency

Today, the main goal of any distribution network is to maximize customer satisfaction, which in turn leads to maximizing the profits of the enterprise. To achieve this goal, first of all, it is necessary to study the behavior model buyers in the trading floor, their habits, needs, perceptions. Modern patterns of customer behavior are based on video analysis, as a change in the psycho-emotional state of a person, and a change in the trajectory of the movement of customers. For further understanding of the study of the subject area, it is necessary to conduct the concept of consumer flow, and give it a definition.

Customer flow is the direction that most shoppers walk in the store.

A lot of people today are faced with the problems of attracting consumer flows in the trading floor, this may be due to unattractive display of goods, or with the wrong work schedule of staff, improper placement of the equipment of the trading floor, which does not allow customers to go to any department, etc.

To the owner of the outlet, it is imperative to have an idea about the attendance of the institution, since this information is will allow you to find peak hours, days, weeks, which in turn will allow you to create a convenient work schedule for working personnel. There is also a number of additional data that can be obtained by analyzing the distribution of customer flows, for example heat maps allow you to define areas of activity (departments that cause the greatest interest among visitors), dead zones (departments that do not attract their goods at all) and bottlenecks, this will help to understand where it is best to place

Thus, the main goal of traffic control is to continuously search for leverage so that this traffic grows or at least does not fall. On Today, customers are increasingly interested in technological tools to attract and retain customers.

An integral part of the structure of the shopping center is a video surveillance system to ensure security. If you use the available hardware of such systems, there is the prospect of converting a conventional surveillance system into an intelligent system with functions statistical analysis.

Thus, at this point in time, the most progressive technology for obtaining statistics on visitor behavior in retail store based on video analysis.

Modern systems for analyzing the distribution of customer flows have several advantages, for example, high processing speed, accuracy counting etc. However, they are still not reliable, since there are no open access methods and algorithms with which the task was implemented, quite expensive and have limited functionality.

The master's work is devoted to the urgent scientific task of analyzing the distribution of customer flows based on video information directed to increase efficiency and further optimize the trading floor. Ads and promotions. Trajectory maps show the most popular routes for visitors moving around the trading floor, etc.

2. Goal and tasks of the research, expected results

The purpose of the research is to increase the efficiency of the trading floor through the use of modern methods and algorithms for detecting and tracking video sequence objects.

  1. He will get acquainted with the influence of the dynamics of customer flows on the conversion in trading floors.
  2. To analyze and compare existing methods and algorithms for detecting, tracking objects in a video sequence.
  3. Perform structural and algorithmic analysis of software tools.
  4. Develop an improved algorithm for determining the video stream objects.
  5. Modify the algorithm for tracking objects of system interest.
  6. Conduct experimental studies of the effectiveness of the developed algorithms.
  7. Develop a computerized system for analyzing the distribution of customer flows based on video information.

The object of research is the process of analyzing the distribution of customer flows in the trading floor.

The subject of research is the combination of methods and algorithms for detecting and tracking video sequence objects.

As part of the master's work, it is planned to obtain relevant scientific results in the following areas:

  1. Development of a modified algorithm for tracking system objects of interest.
  2. Improving the method for detecting objects of a video sequence and evaluating the effectiveness of its application.

3. The practical significance of the results

The developed computerized system will be possible to use in retail centers for distribution analysis flows of customers, in order to increase the efficiency of the trading floors, as well as with some modification can be used in other organizations (restaurants, cinemas, etc.)

Using the developed application will allow:

  1. Ensure accurate counting of visitors at the outlet, evaluate store traffic by identifying peak hours, which in turn will help to effectively distribute the work of staff (to optimize labor costs).
  2. Determine the values of the base indicator of the effectiveness of a sales conversion point of sale, which shows the ratio the number of visitors to the outlet in relation to the number of transactions (purchases).
  3. Identify those places in the store where people are most active.
  4. Analyze the movement of customer flows within the store.
  5. Identify low-visited areas, as a result of which rearrange goods that are not in demand in the so-called hot zones, that is, the most popular places to visit the outlet.

4. Research and development overview

The detection and tracking of video sequence objects has gained considerable interest in the last two decades. Increased interest due to the availability of high-quality inexpensive surveillance cameras and the need for automated video analysis. Action recognition a person in real conditions is used in intelligent video surveillance, analysis of customer behavior, Homeland Security, Crime Prevention and more [1 ].

Along with this, this section will provide an overview of the latest research in the field of computer vision, in particular detection and tracking objects of interest in a video sequence using a variety of techniques and algorithms by both American, European, Chinese scientists, and domestic experts.

4.1 Overview of international sources

At the moment, a huge number of publications and related studies in the field of computer vision are being released. There are many classic methods. classification and detection. Basically, these methods consist in highlighting on the images of certain features (special points) or local regions, which will characterize the picture. I can cope with this task, for example, methods such as SVM , HOG / SIFT .

However, today there are more advanced methods, such as neural networks, which themselves abbr the object and return the prediction, that is, the label the class to which the object belongs.

In this regard, it is worth noting first of all the use of neural networks in computer vision, which was covered in the media back in 1983, by handwriting recognition by Jan LeCun [2].

However, the development did not stand still and in 2012 AlexNet appeared, which participated in the ILSVRC competitions (ImageNet Large-Scale Visual Recognition Challenge). Raghav Prabhu [3] provides an overview of the architectures that were the first in the ILSVRC (ImageNet Large-Scale Visual Recognition Challenge) from 2010 to 2016.

The next step to understanding what is going on inside convolutional neural networks was the article [4], in which the authors proposed ways to visualize that what parts of the picture respond to neurons in different CNN layers. The authors showed that the first layers of the convolutional network respond to Low-level things (edges/corners/lines), and the last layers react already to whole parts of images, that is, they already carry some semantics.

In 2018, a team of scientists Li Liu, Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, Matti Pietikainen from China, Canada and Finland introduced a joint publication aimed at studying the use of deep learning in the field of computer vision, the publication talks about various architectures neural networks, experimental studies are conducted to assess the quality of these networks, problems of detection, datasets are considered where research is being done and more [5].

In addition to the classification of images and object detection on video, special attention should be paid to re-identification . Classical methods are described in [6], the authors introduce the history of re-identification of a person and her the relationship with the classification of images and search for instances, exploring a wide selection of hand-crafted systems and large-scale methods for re-identification based on images and videos, describe future directions in end-to-end re-identification and quick search in large galleries, as well as summarizing some important, but underdeveloped issues. Another job of re-tracking presented by a group of authors at the International Conference on Image Analysis and Recognition (ICIAR 2019) [7]. The authors consider the issue of multi-camera tracking of several people an algorithm for re-identification (re-identification) of a person, which recognizes and stores the identifiers of all discovered unknown people in the entire video stream.

Accompanying people in the video sequence is covered by Nicolai Wojke, Alex Bewley, Dietrich Paulus [8] and Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft [ 9 ]. In the last work, one of the first real-time trackers Simple Online and Realtime Traker (SORT), which is reliable and can cope with difficult situations, was published in 2016. In 2017, a modification of SORT was released in the form of DeepSORT [10]. DeepSORT began to use a neural network to extract visual signs, using them to resolve collisions. Today it is considered one of the best online trackers.

4.2 Overview of national sources

The task of tracking objects in the video stream is actively involved in the Moscow State Technical University Bauman, along with this, the following works in the field of computer vision, in particular object tracking, were highlighted. Publication K.L. Tassova and D.E. Bekasova are devoted to the problem of overlapping objects in the video stream [11]. The article provides a description of the task of tracking objects in a video stream, introduces the basic concepts of the problem area, highlights typical solutions to the problem. In the work [12] A.N. Alfimtsev, N.A. Demin proposed an integral algorithm that uses the capabilities of the Lucas–Canada algorithm and the Viola–Jones algorithm to capture and track a remote object. You can also note the work of the group of ators I.I. Lychkov, A.N. Alfimtsev, V.V. Devyatkova Tracking moving objects to monitor traffic flow [13].

The use of convolutional neural networks for detecting objects in images is described in detail in the works of A.P. Beresnev, I.V. Zoev, N.G. Markova (Tomsk Polytechnic University) [14] and N.S. Artamonov, P.Yu. Yakimov (Samara National Research University named after Academician S.P. Korolev) [15].

At St. Petersburg State University of Telecommunications. prof. M.A. Bonch-Bruyevich was offered an intelligent system for detecting people in a pedestrian zone. A prototype was created on the Raspberry Pi 3 microcomputer platform in order to show that these systems can be deployed not only on powerful computing clusters, but also on end devices [16].

Conclusions

At this stage of the master's work, an analysis was made of the influence of the dynamics of customer flows on conversion in trading floors. Such directions in solving the problem as localization of objects, as well as their further support, are determined. The basic algorithms for detecting and tracking people in a video stream are analyzed.

As a result of comparing video detection methods, the following conclusions were drawn: template search methods are extremely sensitive to input data and their changes. Thus, they can only work effectively. under certain, strictly fixed conditions. Methods based on feature extraction and neural networks can be trained for certain classes of objects, which increases the accuracy of detection and allows you to classify objects according to several classes. As a result of a comparative analysis of various methods of searching for objects to implement the detection task, it was decided to use neural network methods for processing video data. This decision is due to the high accuracy of the selected methods and the possibility of their integration into the studied software package.

Analysis of object tracking methods showed that there is an urgent problem of providing continuous tracking. Most existing tracking systems do not support this functionality, or try to solve the problem by choosing an angle, at which the probability of overlap is minimal.

Among the considered methods of tracking objects of a video sequence, we can distinguish the Kalman filter, which is the most A popular algorithm for tracking and predicting current and future positions. The main feature of this algorithm is that it allows you to track the trajectory of an object even if it was not recognized on several frames in a row for one reason or another.

References

  1. Debaditya Acharya, Kourosh Khoshelham, Stephan Winter Real-time detection and tracking of pedestrians in CCTV images using a deep convolutional neural network/ Conference: Proc. of the 4th Annual Conference of Research@Locate, At Sydney, Australia, Volume: 1913/Infrastructure Engineering, The University of Melbourne/April 2017, 31-36
  2. Y.Le Cunn, B.Boser ,J.S. Denker, D.Henderson, R.E.Howard, W.Hubbard and L.D.Jackel Handwritten Digit Recognition with a Back-Propagation Network / [Электронный ресурс]. – Режим доступа: [Ссылка].
  3. Raghav Prabhu CNN Architectures — LeNet, AlexNet, VGG, GoogLeNet and ResNet [Электронный ресурс]. – Режим доступа: [Ссылка].
  4. Matthew D Zeiler, Rob Fergus Visualizing and Understanding Convolutional Networks / Dept. of Computer Science, Courant Institute, New York University [Ссылка]
  5. Li Liu ,Wanli Ouyang, Xiaogang Wang, Paul Fieguth, Jie Chen, Xinwang Liu, Matti Pietikainen Deep Learning for Generic Object Detection: A Survey
  6. Liang Zheng, Yi Yang, Alexander G. Hauptmann Person Re-identification: Past, Present and Future [Электронный ресурс]. – Режим доступа: [Ссылка].
  7. Mohammadreza Baharani, Shrey Mohan Hamed Tabkhi:Real-Time Person Re-identification at the Edge: A Mixed Precision Approach /International Conference on Image Analysis and Recognition (ICIAR 2019), pp.27-39 (2019)
  8. Nicolai Wojke, Alex Bewley, Dietrich Paulus: Simple Online and Realtime Tracking with a Deep Association Metric / University of Koblenz-Landau, Queensland University of Technology (2017)
  9. Alex Bewley, Zongyuan Ge, Lionel Ott, Fabio Ramos, Ben Upcroft SIMPLE ONLINE AND REALTIME TRACKING/Queensland University of Technology, University of Sydney (2017)
  10. Nicolai Wojke, Alex Bewley, Dietrich Paulus Simple Online and Realtime Tracking with a Deep Association Metric [Электронный ресурс]. – Режим доступа: [Ссылка].
  11. Тассов К.Л., Бекасов Д.Е. Обработка перекрытий в задачах отслеживания объектов в видеопотоке. Инженерный журнал: наука и инновации, 2013, вып. 6. – Режим доступа: [Ссылка].
  12. Алфимцев А.Н., Демин Н.А. Захват и отслеживание удаленных объектов в видеопотоке. Инженерный журнал: наука и инновации, 2013, вып. 11. Режим доступа: [Ссылка].
  13. Девятков В.В., Алфимцев А.Н. Отслеживание движущихся объектов для мониторинга транспортного потока / Труды 34-й Конференции «Информационные технологии и системы» (Геленджик, 2011). Геленджик: ИППИ им. А.А. Харкевича РАН, 2011. С. 31-36.
  14. Берсенев А.П., Зоев И.В., Марков Н.Г. Исследование свёрточных нейронных сетей класса YOLO для мобильных систем детектирования объектов на изображениях. Труды международной конференции по компьютерной графики и зрению "Графикон", вып.28, 2018. - 196-199 с.
  15. Н.С. Артамонов, П.Ю. Якимов Классификация дорожных знаков при помощи YOLO CNN на мобильной платформе NVIDIA Jetson / IV Международная конференция и молодёжная школа «Информационные технологии и нанотехнологии» (ИТНТ-2018) Самара, 24-27 апреля 2018 г. - 2328-2334 с.
  16. Яскеляйнен Я.С., Хакимов А.А., Мутанна М. С.А., Мутханна А.С.А., Киричёк Р.В. Разработка системы распознавания пешеходов в режиме реального времени для ее применения в концепции Умного города // Информационные технологии и телекоммуникации. Санкт-Петербург: СПбГУТ, 2018. Том 6. № 3. С. 123–136.