Українська   Русский
DonNTU   Masters' portal

Abstract

Contents

Introduction

Methods and algorithms for analyzing and synthesizing the emotional state of a person's face is an integral part of artificial intelligence systems and tools that are aimed at researching, creating and implementing algorithmic and software-hardware systems and complexes with artificial intelligence elements based on modeling human intellectual activity. Modeling and recognizing emotions as one of the channels of non-verbal signal and regulative information transfer, reproduces the dynamics of the inner experiences of a person, is an actual and important direction of research with the goal of creating systems for computer recognition and synthesis of visual images. Non-verbal, facial transmission of information by a person has become the subject of intensive research. The conducted researches made it possible to identify some approaches to the formalization of emotions, in particular: models of emotions in psychology, Darwin's evolutionary theory of emotions, Wundt's "associative" theory, James-Lange peripheral theory, Cannon-Bard theory, psychoanalytic theory of emotions, Weinbaum and its modification, Anokhin's biological theory of emotions, frustration theories of emotions, cognitivistic theory of emotions, Simonov's information theory of emotions, Izard's theory of differential emotions, the system of dirovaniya states face or RAS8 system proposed by Ekman and others. As a result, otriani important data for rigorous experimental studies show expression and differentiation of perception problems posed facial expressions. If the mimiogenes of the zone are relatively independent of each other, do they all contribute equally to the overall effect of perception of expression? How are their connections and relationships reproduced? Or invariant zones of facial expression regarding the modality of emotion?

1. General Information

The problem of simulation of mimic expressions of emotions according to the proposed formal description of basic emotions is considered. To search for the space of characteristic features, to construct the basis of this space, to reproduce the derivatives of emotional states with the subsequent application of a convex combination, the following is proposed:

  1. - Creation of a set of photographic images of facial expressions on the face that correspond to situations in which basic emotions arise, a description of facial expressions peculiar to these emotions;
  2. - Analysis of the obtained set in order to identify areas that contain characteristic features of emotions and their description;
  3. - Creation in space of characteristic features of the basis for the subsequent decomposition of arbitrary vectors of mimic manifestation of emotional states (as a convex combination of basic emotional states).

2. Image contouring technologies

There are many technologies for obtaining on the image of the point curves that correspond to the contours of the eyebrows, eyes and mouth. Basically, they are based on obtaining the image contour as a sharp boundary between the image elements (using convolutions, color analysis, etc.) followed by skeletonization (obtaining a single-thickness contour).

In this paper, it is proposed to apply an imitation of the operation of the visual receptors of the human eye to image contouring. It is known that the eyeball is in a continuous microcircuit. The question of these micro-motions has an ambiguous interpretation. It can be assumed that these micromovements are a necessary condition for the functioning of the device for extracting the outlines of the image. To check this, we force the retina receptors of the artificial eye to fix the proposed image, then in an insignificant way (for example, to 1 point) of the image to the side and again let the receptors of the eye fix it. At this point, the relative changes in the signal will appear at the outputs of the receptors. We take the value of the changes in the receptors and bring them to the corresponding points on the image - we get the outline of the image.

Simulation of receptors on the retina is as follows. There are scenes and the direction of micromovement (for example, diagonally on M points). First, a particular receptor "sees" a point with coordinates (x, y), and after micro-motion - with coordinates (xM, yM). The difference in the color planes between the input point and the point that was in its place as a result of micro-motion is the relative change in the input stimulus signal (for a specific receptor) The contours obtained in this way must be reduced to a "skeletal" form. That is, we need to select some middle line that correctly reflects the contour structure. For this we apply the well-known algorithm of Zong-Sun.

3. Research results

In the transition from a phenomenological description of emotions to situations in which emotions arise, many photographic images of basic emotions were created. Further from the images obtained, the contours (eyebrows, eyes, lips, etc.) necessary for further processing were distinguished by the methods described above (point 2). Fig. 1 contains the contours of the right eyebrow for emotions: joy, sorrow, hope, fear, pleasure.

Fig. 1 - Contours of the right eyebrow

In Fig. 4 contours of the right eyebrow are presented in the form of a dotted curve.

Fig. 2 - Diagram of point curvatures of the right eyebrow

The graph shows that the position of the obtained contours of the right eyebrow will describe the mimicry of emotions. That is, it is clear that for the emotions of pleasure and joy, there is no facial expression, for emotions of grief and hope - the inner corners are raised, and for the emotion of fear - the eyebrow is raised and erected.

The resulting contours were applied to transformations to obtain sets of control points of NURBS-curves. Fig. 3 contains the contour plot and the NURBS curve corresponding to this contour for the position of the right eyebrow at the emotion of joy.

Fig. 3 - Contour plot

Fig. 4 - Representation of basic emotions in a contour form using NURBS-curves

Of the eight obtained sets of reference points of NURBS-curves (templates) and the vector of characteristic facial features for wrinkles, the basis (8) of emotional states (B) of a particular person was constructed. Similarly, from a photographic image of an arbitrary emotion, reproduces the situation in which a sense of guilt arises, the corresponding vector b (9) was constructed. Using the transformation (10-12), the schedule of the obtained vector b was calculated for the basis B

Finding control points on the face

Conclusion

The proposed mathematical model and integrated information technology for the automatic determination of the arbitrary emotional state of a particular person as a convex combination of some basic states. To do this, using the mathematical model and the original software creates a basic space of emotional states of a particular person. In the future, the arbitrary emotional manifestation of this person is decomposed as a convex combination of emotional states in this space. To build a basis for the space of emotional states, flexible patterns of the contours of the main zones of the face are used. Flexible patterns are described using NURBS curves. The adjustment of the pattern to the point contour of a particular image is carried out using the B-spline approximation, by solving the redefined inhomogeneous system of linear equations. The proposed technology is of practical value in systems of visual control of operators of complex industries (nuclear power, etc.) for automatic control over their emotional state.

References

  1. Леонтьев В.О. Классификация эмоций. – Одесса: Инновационно-ипотечный центр, 2002. – 84 с.
  2. Крак Ю.В., Бармак О.В., Єфімов Г.М. Синтез мімічних виразів емоцій на основі формальної моделі, Штучний інтелект. – 2007. – № 2. – С. 22-31.
  3. Крак Ю.В., Бармак О.В., Єфімов Г.М. Використання контурних моделей для побудови базису простору мімічних виразів емоцій, Штучний інтелект. – 2007. – № 4. – С. 288-296.
  4. Piegl, Les and Tiller, Wayne. The NURBS Book, 2nd Edition. – Berlin, Germany: Springer-Verlag, 1996.
  5. C. deBoor. A Practical Guide to Splines. – New York: Springer-Verlag, 1978.
  6. Демидов В.Е. Как мы видим то, что видим. – М.: Знание, 1987. – 240 с.
  7. Свободная энциклопедия «Википедия» – Нейронная сеть [Электронный ресурс]. – Режим доступа: http://ru.wikipedia.org/wiki/Нейронная_сеть
  8. Свободная энциклопедия «Википедия» – Искусственная нейронная сеть [Электронный ресурс]. – Режим доступа: https://ru.wikipedia.org/wiki/Искусственная_нейронная_сеть
  9. 7. Zhang T.Y., Suen C.Y. A fast parallel algorithm for thinning digital patterns, Commun. ACM. – 1984. – Vol. 27, № 3. – Р. 236-239.
  10. Чубукова И. Data Mining[Электронный ресурс]. – Режим доступа: http://www.intuit.ru/studies/courses/6/6/lecture/178?page=5.