DonNTU   Masters' portal

Abstract on a subject of a master's thesis
Program complex for analysing video recordings of football matches

Content

Introduction

Computer vision – actual area of scientific researches. The rapid development of computational tools, expansion of their possibilities is a major factor increasing the widespread introduction of computer vision techniques in various areas of scientific and practical activities.

Sport – an important social sphere where computer technologies are actively applied. The actual problem here is to collect and analyze statistical data. This is especially significant for football, which is very popular and rapidly expanding the boundaries of their fans.

Sports – a health source for each person. The adverse demographic situation in Ukraine demands activization in attraction of the population, especially the younger generation, to replace bad habits by sports activities. As the most popular sport in the world, football is excellent means of raising the national level of health. For this purpose it is necessary its mass promoting, and it depends first of all on level of the Ukrainian teams game .

The amount of information increases every year, it becomes critically important the speed of processing. Performance requirements, quality and style of play is constantly growing. To meet the requirements, the team must constantly improve the strategy, tactics and technique. An important tool for this is the video of football matches.

Computer automation of the analysis of video data opens new possibilities for an assessment of teams and players tactical-technical characteristics. Program data collection and computer analysis are essential and very important means to achieve success in football. The increasing dynamics of the game requires a more accurate vision and analysis of the situation on the field.

1. Program complex architecture analysis

In matches analysis systems the following approaches are used [1]:

  1. Statistical analysis system based on the video – the analysis of matches is performed manually by a person viewing the video.
  2. Player tracking system based on the video – the stadium is equipped with a sufficient number of cameras to cover the entire field, objects are automatically recognized on the field from video.
  3. Electronic tracking systems – their action is based on the use of microchips placed on the players/referees/ball and further tracking their movements using radio waves.

Well-known programs for automatic analysis of football matches – ProZone and Amisco Pro, which are used such popular clubs as Chelsea and Liverpool [2, 3].

The principles of a system for tracking players on the basis of the video the following:

  1. The stadium is equipped with video cameras to cover the entire field. Each point of the field should be covered by at least two cameras, which improves the accuracy of object recognition (Fig. 1).
  2. Video recordings are synchronized with each other, and then conducted an automatic analysis of the players movement. As a result sets of objects and coordinate of these objects in the field are defined.
  3. The obtained data are exposed to the analysis and visualization. Updatings of the received data and introduction of new data on a match which can't automatically be distinguished are possible. Results of a stage are statistical reports and screen visualization.
Example of cameras location

Figure 1 – Example of cameras location

Architecture of program complex for analysing video recordings of football matches is shown on Fig. 2 [4]. The program complex is intended for recognition of matches videos, visualization of results and calculation of statistical characteristics about tactical actions and physical preparation of teams and players.

Architecture of program complex for analysing video recordings of football matches

Figure 2 – Architecture of program complex for analysing video recordings of football matches

The program complex provides: record and storage of matches videos; videos synchronization from different cameras; search and tracking objects in videos; the statistical data analysis after searching and tracking; visualization of the results.

2. Input and pre-processing subsystems desription

The input subsystem provides recording and further preservation of matches (Fig. 3).

Football match video frame

Figure 3 – Football match video frame

Central processing unit writes data streams video from cameras in the files with unique identifiers. The identifier includes the camera id, the day of the match and team names. Video recordings are stored in a directory tree with hierarchy a video/match/game day/tournament.

The module of reports generation for storage state forms statistics about number of videos for one match/day/tournament, about the current volume of videos.

The pre-processing subsystem provides video recordings converting to an internal format of processing and the further synchronization of the data. Performed a mathematical pre-processing of images by different filters (Kenny, Sobel, a threshold filtering, etc.) to select the optimal strategy for searching and tracking objects in video recordings in the processing subsystem (Fig. 4). The image undergoes binarization, removal of unnecessary details and noise, the calculation of the gradient image, background subtraction, edge enhancement, etc.[5-7].

Videos pre-processing

Figure 4 – Videos pre-processing

The module of videorecordings processing carries out videoframing and creation of an internal format. This format stores a set of images on a hard disk with unique identifiers. The identifier consists the frame time mark and the camera identifier.

The synchronization module processes the internal data and creates the table which stores time marks and lists of images from cameras.

Visualization module allows you to play video, view the individual frames to expose them to the processing filters.

The module of the report on pre-processing generates statistics about time spent for processing of videorecording /of all videorecordings of a match, the best filters for processing of the image, time spent for synchronization of the data, etc.

3. Processing subsystem desription. Pipeline of objects recognition and tracking in videos

Processing subsystem provides detection and the search for objects in the video.

The module of recognition and tracking of processing subsystem realizes the processing pipeline with the use of OpenTL library (Fig. 5)[8].

Pipeline of objects recognition and tracking in videos (animation: 5 frames, 5 cycles of repeating, 100 kilobytes)

Figure 5 – Pipeline of objects recognition and tracking in videos
(animation: 5 frames, 5 cycles of repeating, 100 kilobytes)

The module of initial data formation for the analysis creates files with trajectories of players moving for each processed videorecording.

Report generation module computes statistics about results of processing: the number of detected objects, the number of lost objects, the number of processed frames, the number of processed video, etc.

At each step the pipeline receives one or several images. Search of the new and lost objects in images is made. The object is represented mathematical model (kinds of degrees of freedom and movement) and a condition (position in the frame, speed and acceleration). Their comparison with a set of existing objects is carried out and the new set is formed. The resulting set of objects is passed to the Bayesian filter[8]. In parallel at a stage of preprocessing from the image the information independent of model and a condition is allocated. This is a set of defined visual characteristics: pixel images in RGB and HSV, sets of points that characterize the properties of the image – corners, edges, contours.

At the stage of processing is the comparison of characteristics obtained from an image, and the predicted characteristics of objects processed with Bayesian filter [8].

Output data for each tracked object are: the vector Z = <h,z,e,R,H>, where h and z – compare features, e – the difference between them, R – measurement noise, H – Jacobian of the measurement, and probability of values of the characteristics of P (z|s), where s – the object's state. To improve the accuracy of tracking at this stage is possible to use static merge of object characteristics, received by various methods, in a general characteristic.

Using Bayesian filter for each object tracking solves two main problems: the prediction of object state using dynamic models and upgrades. Gaussian filter (based on the Kalman filter) and the Monte Carlo filters (based on particle filters) can be used [8].

At the last stage of the pipeline check of tracking accuracy after filtering is made. If accuracy low and the object has been lost, return on a stage of search of the new and lost objects is carried out.

Accelerating work of developed algorithms of recognition and tracking is solved with the use parallel calculations on CUDA graphic processors.

The methods of fragments described in [9] can be applied to objects tracking.

4. Analysis and output subsystems desription

Analysis module of the analysis subsystem provides a calculation of statistical characteristics of the players and teams.

During the analysis technical and tactical actions (TTD) are recorded: short (to 10 m), averages (to 25 m) and long (from above 25 m) passes which are carried out forward, back and across a field; holding, picking and catching the ball; bypassing opponent; head-playing; free and corner kicks; shoots; fouls. Physical characteristics of the player (speed, accelerations, spurts, the passed distances), tactical characteristics (percentage of the location of the player/group of players/teams on the field, team and lines of players formation), important game elements (throw-ins, off-sides, penalties, goals) are estimated.

The analysis module provides receiving characteristics on the basis of three approaches: automatically based on the coordinates of objects (speed characteristics, location on the field), based on computer analysis with the adjustment of the expert (passes, shots, dribbling), manually by expert (penalties, jumps, free kicks, fouls).

The module of analysis reports generation creates reports for output subsystem.

The output subsystem provides a output of complex work results to the screen and reports saving.

The module of visualization provides animation of the loaded model of a match, visualization of objects movings trajectories in the field (fig. 6), display the statistical data received in the analysis subsystem (fig. 7) [10].

Visualization of trajectories

Figure 6 – Visualization of trajectories

Speed ??characteristics of the player

Figure 7 – Speed characteristics of the player

The module of reports saving in a DB brings the generated reports in a database.

A module for working with database provides a view of existing reports, report for the period of time and call visualization module according to statistics from the database.

Conclusion

On the basis of methods of computer vision the architecture of program complex for analysing video recordings of football matches is developed. Functional requirements to complex subsystems are formulated. Implemented a basic version of the pre-processing, analysis and output subsystems.

Additional studies will be conducted to improve the analysis, output, and pre-processing subsystems, implementation of input and processing subsystems, parallelization of labor-intensive computing processes.

Parallelization of computational processes will create a high performance system that performs the search and tracking objects in real time.

The developed methods and algorithms for image processing, retrieval and tracking objects in a sequence of frames, the algorithms calculate the statistical characteristics can be used for solving a wide class of problems associated with the processing of data on sports, video systems in other search and analysis facilities for certain characteristics in the systems managed by video data.

This master's work is not completed yet. Final completion: December 2012. The full text of the work and materials on the topic can be obtained from the author or his head after this date.

References

  1. Handbook of soccer match analysis / Christopher Carling, A. Mark Williams, and Thomas P. Reilly. – London: Rourtledge,2005. – 163 c.
  2. Website AMISCO company [Electronic resource]. Access mode: http://www.sport‐universal.com/.
  3. Website PROZONE company [Electronic resource]. Access mode: http://www.prozonesports.com/index.html.
  4. Кулиш М.Н., Ладыженский Ю.В. Архитектура программного комплекса для обработки и анализа видеозаписей футбольных матчей// Моделирование и компьютерная графика — 2011 / Материалы IV международной научно-технической конференции — 5-8 октября 2011 — Донецк, ДонНТУ — 2011.– c. 166-170.
  5. Форсайт Д., Понс Ж. Компьютерное зрение: Современный подход – М.:Вильямс,2004. – 928 с.
  6. Шапиро Л., Стокман Дж Компьютерное зрение– М.:Бином,2006. – 752 с.
  7. A course of lectures Введение в компьютерное зрение [Electronic resource]. Access mode: http://courses.graphicon.ru/main/vision/2011/lectures.
  8. Panin G.. Model-based visual tracking : the OpenTL framework – Chicester: John Wiley & Sons,2011. – 318c.
  9. Ладиженський Ю.В., Середа А.О. Відстежування об’єктів у відеопотоці на основі відстежування переміщення фрагментів об’єктів //Наукові праці Донецького національного технічного університету. Серія: Обчислювальна техніка та автоматизація. Випуск 17 (148). – Донецьк : ДонНТУ, 2009. – 127-134 сс.
  10. Кулиш М.Н., Середа А.А., Ладыженский Ю.В. Визуализация результатов анализа видеозаписей футбольных матчей// Інформатика та комп'ютерні технології — 2010 / Матеріали VI науково-технічної конференції молодих учених та студентів — 23-25 листопада 2010 — Донецьк, ДонНТУ — 2010. – c. 110-115.