photo

Vladislav Luytuy

Faculty of Computer Science and Technology

Department of automated control system

Speciality Information systems and technologies in engineering and business

Deception detection system by facial expression on video

Scientific adviser: Ph.D., associate professor of ACS Alexander Sekirin

   

Abstract

Name Vladislav Luytuy
Date of birth Aug 29, 1996
Place of birth Ukraine, Donetsk
School 2002–2013 — Donetsk school # 14
University Donetsk National Technical University, faculty of computer science and technology:
2014–2018 — DonNTU, FCST. Specialty Information systems and technologies in engineering and business. Academic bachelor.
2018–2020 — DonNTU, FCST. Specialty Information systems and technologies in engineering and business. Master.
Average score 95
Languages Russian (native), Ukrainian (native), English (upper-intermediate)
Personal achievements 2018 — Bachelor's Degree in Information Management Systems and Technologies
Hobbies and interests Tone and color image correction, creating posters and effects in Adobe Photoshop
Create vector art and icons in Adobe Illustrator
Personal qualities Creativity, independence, teamwork, mobility, stress resistance
Professional and computer skills Operating systems: Windows, Ubuntu
Programming languages: PHP, JavaScript, Java, Python
Frameworks: Yii2 (PHP), Laravel (PHP), Spring MVC (Java)
DBMS: MySQL, MongoDB
Modeling languages: UML
Version Control Systems: Git
DevOps stack: Apache, Nginx, Docker, Cron, Jenkins; Backend stack: Memcached, RabbitMQ, MVC, OOP, Patterns, SOLID, DRY, GRASP
Frontend stack: HTML 5, CSS 3, Pug/Twig, LESS/SASS/Stylus, PostCSS, Gulp, Webpack
Graphics editors: Adobe Photoshop, Adobe Illustrator, Figma
Additional courses Online Network Administration Courses — CCNA Routing and Switching
Future plans Getting better in deep learning and big data
Contact information e-mail: vlad112263@gmail.com

Abstract

Content

  1. Introduction
  2. Relevance of the topic
  3. The purpose and objectives of the study, expected results
  4. The subject matter of the study
  5. Features of visual facial recognition methods
  6. Intensity of facial muscle movement
  7. Analysis of methods for detecting faces in the image
  8. Conclusions
  9. References

Introduction

Each of us has encountered a phenomenon called lies. A lie is called the deliberate transmission of untruthful information in order to arouse in another person a conviction that the transmitter himself considers to be untrue. Deceptive can be various kinds of facts and information. Lying takes on special significance when it comes to politics, the media, the legal process, medicine, the work process and other areas of everyday life of a modern person. The eternal problem of human sincerity, deception and lies has repeatedly become the subject of discussion in fiction, philosophy, sociology and psychology. Deception plays an important role in our lives, but it is necessary to distinguish between lies for salvation or lies for lies.

Relevance of the topic

In 2006, the Institute of Statistics in Oklahoma analyzed the accuracy of judgments on the recognition of false information among 6,651 messages in 206 documents of various presentations of information (video, text, audio) from 24,483 people, including 2,842 experts in the field of psychoanalysis. As a result, they obtained the following results: under the created conditions, people made an average of 54% of correct judgments, correctly classifying 47% of lies as deceiving and 61% of truthful ones as deceiving.

To conduct analysis of human facial expressions to assess the reliability of information, experts use the rules and facial cards obtained using psychophysiological studies in the field of human psychology and forensics, which is a disadvantage of using it in different fields of activity, because there are not so many specialized experts in this field.

The first technical competitor was the well-known polygraph. The use of a polygraph has a rather long history. The founder of the polygraph is the United States, where considerable attention was paid to the problem of its application at various levels (federal, regional) and various instances, up to the US Congress, the president, the US Supreme Court, and others. The statistics of the polygraph’s performance is astonishing: the actual performance rating (95%) exceeds the declared theoretical (80%). That is, only 5-7 people out of a hundred are able to go through the polygraph so that the result will be incomprehensible.

However, for all the advantages of using a polygraph, it has a number of drawbacks and the most significant of them are expensive equipment (about 200 thousand rubles), the inability to use it without informing the interrogation subject, a certified expert is required who analyzes the testimony of the device. In the end, we can say that on a par with the development of computer technology, the growth of computing power and the teaching methods of artificial intelligence, polygraph cannot be called technology of the 21st century.

Based on the foregoing, it can be concluded that the best and most promising option is to use a modern mathematical apparatus and a technological process for machine learning computer vision systems for the analysis of arbitrary video sequences for use in the tasks of this study for a future master's thesis.

The purpose and objectives of the study, expected results

The aim of the study is to increase the effectiveness of computer vision systems for recognizing facial expressions in order to identify signs of fraud by video frames. This study will allow in the future to expand the scope of artificial intelligence in the field of the judiciary, political science, sociology and other areas where the human factor, namely the lie, can cause damage to processes or others.

To achieve this goal it is necessary to solve the following tasks:

The subject matter of the study

In the course of the analysis of the subject area, the object of modeling and further algorithms was identified — facial expressions. The term facial expression refers to the movement of muscles in coordinated complexes, reflecting the various mental states of a person. The difficulty of recognizing facial expressions as an object was identified due to the following factors:

The problem of achieving the fastest and most reliable results of the research goal is primarily in:

Features of visual facial recognition methods

Of the features of visual recognition methods, facial expressions distinguish the following: the complexity of visual techniques requires significant time-consuming training. An expert, as a rule, chooses to study only one technique. The user cannot independently make the classification of the veracity of information on the assessment of facial expressions. A comparative analysis of facial recognition methods is presented in table 1.

Method Action type Intensity of action Action time
Ekman & Friesen (1976, 1978) Measurement of all muscle movements; 44 action units Four actions of three control points of intensity Start-stop (start, maximum, shift)
Frois-Wittmann (1930) 28 descriptions N/A N/A
Fulcher (1942) Absence / presence of 16 muscle movements Rating by volume of movements in each area of ​​the face N/A
Ermiane & Gergerian (1978 Measurement of all visible movements; 27 muscle movements Each action is rated on a three-point scale N/A
Landis (1924) 22 descriptions Each action is measured on a four-point scale N/A

Intensity of facial muscle movement

The intensity of muscle movement is determined as follows: letters A to E are added to the unit number from SKLD depending on the intensity of movement (from minimum to maximum).

Values:

  1. A — poorly distinguishable;
  2. B — insignificant;
  3. C — noticeable or pronounced;
  4. D — sharp or extremely noticeable;
  5. E — expressed in the highest degree.

Motor units relative to the imaginary vertical axis of the face can be: bilateral, symmetrical, one-sided, left, right.

Analysis of methods for detecting faces in the image

Face detection algorithms in the image can be divided into four categories:

The empirical approach is based on top-down knowledge and involves the implementation of an algorithm with rules that correspond to the image fragment on which the human face is found. A set of rules is a formalization of empirical knowledge about the representation of a person in an image and the signs that a person is guided by when making a decision: does he see the person or not. Rules:

The method of reducing the image to eliminate possible interference, as well as to reduce computational operations, preliminarily exposes the image to a significant change in size. On such an image, it is necessary to determine the area of ​​the uniform distribution of brightness (the estimated area of ​​the face), and then check for the presence of sharply different brightness areas inside: these are the areas with a different percentage of probability attributed to the face.

The histogram method for determining areas of an image with a “face” builds vertical and horizontal histograms. In "suspicious" areas, a search for facial features occurs. This approach was used during the development of the machine since assumed small requirements for the processing power of the processor for image processing.

The method for detecting complex faces is based on the search for the correct geometrically located facial features. To do this, use a Gaussian filter with many different scales and orientations. After that, a search is made for matches of the found traits and their location by enumeration.

The feature grouping method involves the use of a second derivative of a Gaussian filter to search for areas of interest in an image. After that, the edges are grouped around each such zone using a threshold filter. Next, the Bayesian network assessment is applied to combine the identified features and a sample of facial features is determined.

The control point distribution method is a statistical model representing incidents whose shape can be deformed. A big advantage of this method is the allocation of variable objects as part of a training set with a small number of parameters. This approach is used in the classification systems of features.

The recognition method using templates is easy to implement and effective when working with images with a simple background. The disadvantage of this method is the calibration of the template near the face image.

The neural network method is the most popular way to solve pattern recognition problems. When solving problems, the support vector method is used, which is necessary to reduce the dimension of the feature space. Moreover, the support vector method does not lead to a loss of information content of the selected training objects, and also allows you to go to the basis of space, where the dispersion will be directed along the main axes of the basis.

The subspace spanned by the main axes thus obtained is optimal among all spaces in the sense that it best describes the training set.

These algorithms are similar to learning with the teacher-to-teacher type and are used for classification and regression analysis tasks. The support vector method is based on the fact that a linear separation of classes is sought.

The main goal of training many classifiers is to minimize classification errors on the training set (or empirical risk).

Conclusions

At this stage of the master's work, an analysis of the psychological phenomenon deception and its relationship with micro-expressions and emotions was carried out. In addition, the basic algorithms for recognizing and classifying parts of the face for finding key points were analyzed, and an algorithm for recognizing emotional micro-expressions using a neural network was also presented.

References

  1. Система кодирования лицевых движений [Электронный ресурс]// основные мимические выражения/ URL: http://dic.academic.ru/dic.nsf/ruwiki/1628394 (дата обращения 18.05.2020)
  2. Эмоциональная напряженность методика определения [Электронный ресурс]/URL: http://pandia.ru/text/80/079/8545.php (дата обращения 18.05.2020)
  3. Исследование рынка систем распознавания эмоций сетям [Электронный ресурс]/ URL: https://habrahabr.ru/post/133686/ (дата обращения 19.05.2020)
  4. Иванов А.И. Нейросетевые алгоритмы биометрической идентификации / Изд-во: Радиотехника. 2006. 144 с.
  5. Гундырев К.В. Искусственные нейронные сети в задачах диагностирования рельсовых цепей// Науч.-иссл.лабор. «Компьютерные системы автоматики». 2005.
  6. Круг П.Г. Нейронные сети и нейрокомпьютеры: Учебное пособие по курсу «Микропроцессоры» / Издательство МЭИ, 2009.176 с.
  7. Применение нейронных сетей для задач классификации [Электронный ресурс]/URL: http://www.basegroup.ru/ (дата обращения 18.05.2020)
  8. Шамис А.Л. Поведение, восприятие, мышление: проблемы создания искусственного интеллекта / Изд-во: Наука. 2005. 224 с.
  9. Кривонос Ю.Г. Моделювання та аналіз мімічних проявів емоцій / Ю.Г. Кривонос, Ю.В. Крак, О.В. Бармак, Г.М. Єфімов // Доповіді НАНУ. – 2008. – № 12 – С. 51-55.
  10. Chien, B. C. Learning discriminant functions with fuzzy attributes for classification using genetic programming / B. C. Chien, J. Y. Lin, T. P. Hong //Expert Systems with Applications. – 2002. – Т. 23. – №. 1. – С. 31-37.
  11. Fernandez, A. Revisiting evolutionary fuzzy systems: Taxonomy, applications, new trends and challenges / A. Fernandez, V. Lopez, M. J. del Jesus, F. Herrera //Knowledge-Based Systems. – 2015. – Т. 80. – С. 109-121.
  12. Ekman, P., & Friesen, W. V. (1978). Manual for the facial action coding system. Consulting Psychologists Press.
  13. Рутковская Д., Пилиньский, М., Нейронные сети, генетические алгоритмы и нечеткие системы / М., 2004. 452 с.