URL: http://cnit.ssau.ru/do/index.htm

Cogninive aspects of multimedia in electronic studying support
A.V.Solovov
Samara State Aerospace University

publications of International conference "IEEE International Conference on Advanced Learning Technologies". Kazan: KSTU, 2002. P. 74-78

Translated by Malykhina Darya

Annotation
Two kinds of thinking are examined here - logical and intuitive. They're mapped with corresponding illustrative and cognitive multimedia functions in computer-aided studying systems. Role of cognitive multimedia function is especially distinguished in generation of integral representation of objects or processes being studied, and in development of the professional-oriented intuition. Examples of cognitive multimedia objects in studying multimedia complex of CADIS system are given here.

1. Introduction
Evolution of electronic multimedia facilities opens fundamentally new didactic abilities for studying domain. Systems of interactive graphics and animation for image analysis allow to manipulate with content, shapes, dimensions, coloring and other parameters for achieving the best obviousness. These and others features are still badly realized by teachers as well as by developers of electronic studying technologies. This fact does not allow to use studying potential of multimedia in full measure. Usage of multimedia in electronic studying not only increases the speed of information transfer to students and raises the level of understanding, but also assists in evolution of such important states for a specialist in any domain as intuition, professional "feeling", iconic thinking. Impact of interactive computer graphics on intuitive, iconic thinking has lead to beginnings of the new direction in the problematic of artificial intelligence, named in [1] as cognitive (i.e. conducive to cognition) computer graphics. In our works [2, 3] cognitive acpects of computer graphics were explored in conformity with computer-aided studying systems. In this article, the given approach is generalized to more wide spectrum of multimedia information.

2. Dualism of thinking
Human consciousness uses two mechanisms of thinking [4]. One of them allows to deal with abstract chains of symbols, texts etc. This mechanism of thinking is usually called symbolic, algebraic or logical. The second mechanism of thinking provides working with sensible images and representations of these images. It is called as iconic, geometric, intuitive. Physiologically, logical thinking is connected with the left cerebral hemisphere, and iconic thinking is connected with the right one.

Each of human's cerebral hemispheres is independend system for perception of the outer world, processing of information about it, and planning of behavior in this world. Left hemisphere acts like a big and powerful computer which deals with signs and procedures of their processing. Speech, verbal thinking, rationally-logical procedures of information processing etc. - all these things are implemented in the left hemisphere. Right hemisphere implements thinking on the level of sensible images: aesthetic perception of the world, music, painting, associative recognition, generation of fundamentally new ideas and discoveries etc. All that complex mechanism of iconic thinking which is often determined by single term "intuition" is the area of right cerebral hemisphere activity.

Often right-hemisphere thinking is being connected with art activity. Sometimes this thinking is even called "artistic". However, even more formalized kinds of activity use this intuitive mechanism of thinking in an essential measure.

Difference between two mechanisms of thinking can be illustrated with principles of construction of the connected text from separate elements of information: left-hemisphere thinking builds single-meaning context from these elements, i.e. actively chooses only items which are the most essential for the exact task from the wide set of all the links between objects and events [5].

Right-hemisphere thinking creates polysemantic context by simultaneous inclusion of practically all the features and links of one or several events. I.e. logical-symbolic thinking carries some affectation in the world's picture, and iconic thinking provides natural spontaneity of world's perception, as it is.

This fundamental difference between left- and right- hemisphere strategies of information processing directly concerns to forming of various talents. So, for the scientific work, i.e. for negotiation of traditional representations, it is necessary to percept the world fully, which assumes an evolution of the talents for organization of polysemantic context (iconic thinking). There are a lot of observations that for the people who retain abilities of iconic thinking, creative work is less tiring than routine and monotonous work. The people who have not made the abilities of iconic thinking, are often prefer to fulfil mechanical work, and they do not find it boring, because they are "enslave" by their own formal-logical thinking. From this it follows how important to correcly make education and training since early times, to let both kinds of thinking which are necessary for a human develop harmonious, to do not have iconic thinking be locked with the logic, to have develop of human's creative potential.

3. Impact of computer vocational training on thinking
Wide usage of computer electronic technics practically in all domains of human activity makes additional demands to vocational training, such as acquirement of new informational, formalized in considerable proportions technologies. At the same time, the main point of professional quailification remains former and consists not only in knowledge of formalized methods, but in developed intuition, so-called professional sense, based on the knowledge of fundamental properties of objects and processes in the data domain, and on the ability of deep analysis of these properties. Such professional knowledges were always worthly in the specialists, and for the today their role has been increased a lot because of wide application of informational technologies. To be able to build adequate simulators, it is important to have comprehensive understanding of physical nature of objects being simulated. To take technically competent decisions while working with CALS systems and others human-computer complexes, it is necessary to be able to correctly interpret and comprehend the results of the calculations, to take into account hard-to-formalize factors, which are always present in any professional activity.

However, the "left hemisphere turn" usually takes place in the development of computer-aided systems for automation of the professional activity, including training systems. It is not so harmless. The danger of excessive computerization is seen not in that so computers will force a human out of domain of the intellectual activity, but in that so a human, being more and more involved into the interaction with computer, will start thinking like a machine [6].

Analysis of negative impact of computerization of the vocational training is explained in many respects as weak influence of computer system being used over the intuitive, iconic mechanism of thinking [7]. According to this fact, a clear separation of implicit, subconscious components of knowledge allows to neatly issue the challenge of their mastering, to formulate appropriate demands to methods and means of the studying, including multimedia technologies.

4. Illustrative and cognitive multimedia functions
Interpreting given above differences between left- and right-hemisphere mechanisms of thinking as applied to cognitive activity of the students, it is possible to conclude that logical thinking picking out only some the most essential elements of knowledge and organizes them into monosemantic conception of objects and processes being studied; at the same time, the subconsciousness provides integral perception of the world in all its multiformity.

Starting from this difference, it is possible to separate two multimedia functions - illustrative and cognitive.

Illustrative function provides support of logical thinking. In this case, multimedia object confirms and illustrates some strongly pronounced idea, property of object or process being studied, i.e. that was formed already, for example, by the teacher developer.

Cognitive function lies in opportunity of obtaining new, i.e. non-existing even in the mind of the specialist, knowledge, with the help of some multimedia object, or at least in assistance to intellectual process of getting this knowledge.

This fundamental idea of differences between illustrative and cognitive functions of multimedia blends well with the classification of knowledges and computer-aided training systems [2] proposed before. Illustrative multimedia function is being realized in the training systems of declarative type during transfer to the students of articulating part of a knowledge which is represented as prepared information with graphics, animations, audio and video illustarations. Cognitive multimedia function becomes apparent in the systems of procedure type, when the students obtaining knowledges using simulators of objects being studied, and, as long as this process of knowledge forming is based on intuitive, right-hemisphere mechanism of thinking, these knowledges themselves have personal nature in the essential measure. Every human being forms his own methods of subconscious mental activity. Contemporary psychological science does not provide strict-grounded methods of forming human's creative potential, even the professional one. One of known heuristic approach to development of intuitive professional-oriented thinking is solving of the exploration tasks. Usage of computer-aided training systems of procedure type allows to intensify this process in the essential measure, by getting rid of routine operations, and to make possible any test operations with simulators.

It is hard to overestimate the role of computer graphics in these educational investigations. Graphic images of process and results of experiments on simulators allow each student to form his shape of the object or process being studied in all the integrity and multiformity of the links. It is also undoubtedly that computer images first of all perform cognitive, but not illustrative, function, because during studying work with computer systems of procedure type students are forming doubly personal components of knowledge, which do not exist for nobody in this form.

However, cognitive multimedia function can be implemented in studying systems of declarative type. For example, at the first familiarity with some technical object, it is possible with the help of nature movies create its integral shape in the students' mind.

Differences between illustrative and cognitive multimedia functions are relative enough. Often an ordinal graphical illustration, animation or movie can push some students on the new idea, allow to see some new elements of the knowledge, which were not put in by a teacher/developer of the studying computer system of declarative type. Thus the function of multimedia object which is illustrative by plan turns to cognitive. At the other hand, cognitive function of computer image during first experiments with studing systems of procedure type can turn into illustrative function in the further experiments, showing explored already and not new property of the object being studied.

Nevertheless, fundamental differences between logical and intuitive mechanisms of human thinking, which follow from these differences of the form of knowledge conception and methods of their mastering, make useful in methodological plan discerning of illustrative and cognitive multimedia functions, and allow to formulate didactic tasks of multimedia objects more clear during development of computer-aided training systems.

5. Interactivity
A computer as a tool for passive visualization of multimedia objects has no newness in didactic sphere. An interactivity is the fundamental new thing for didactic because the students can dynamically control the shape of multimedia objects being studied, content, dimensions and color, look at them from different sides, zoom and unzoom, stop and run from any place, change illumination parameters and do a lot of other actions to achieve the maximum clearness. It might be as well to select a kind of visual representation of the information.

Therefore, interactivity gives possibilities not only for passive perception of the information, but also for active exploration of properties of multimedia models of object or processes being studied. The process of training activity becomes closer to computer systems of procedure type by its didactic conditions. Interactivity impacts cognitive character to multimedia, adds gaming and research components to training, normally induces a student to deep and detailed analysis of properties of objects and processes being studied.

6. Samples
Cognitive images are widely useful in training multimedia complexes (TMC) of the system CADIS (Complex Automated DIdactic meanS) which is being developed in the centre of new information technologies of Samara State Aerospace Univercity [2]. One of CADIS complexes is intended for training of mechanic engineers for construction of power schemes - important but hard-to-formalize stage of designing of mechanical structure. As an example of some standard images used in the intellectual simulator of the complex, let's see the optimal assignment of material (image 1) and internal forces (image 2) in the plate which three sides are rigid, and the point load is applied to the fourth one.


Image 1. Graphic representation of the optimal assignment of material in the plate

Three factors determine cognitive character of these image. First, they displaying fields of phisycal parameters, which were obtained after labour-intensive formal calculations on many-dimensional disctete simulators of finite elements method, clear enought in the form popular and adequate for constcuction mechanics. Second, the integral representation of large data amount of the construction and its stress allowos students to find out the most important regularities, induces them to hypothesize and making explorations. In this task, for example, the correlation of plate dimensions was selected in that way so transmission of external load impacts on the sides as well as on the bottom of this original flat "box", which is being tracked clearly on the images. It is easy to guess and check with the simulator's optimization application software how the plate configuration should be modified in order to get external load balanced only on the bottom or on the sides of the "box". And the third, the most important factor is the cognitive character of given images. Analysis of such images in combination with preceding heuristic designing of the power schemes for different conditions of lean and load of constructions very effective develops professional intuition of the students, their design "sense" of choosing rational trajectories of transmission of internal loads in the construction, as shows long-term experience of exploitation of the simulator.


Image 2. Graphic representation of strain distribution in the plate

Good interactive features provides Flash-technology from Macromedia. Image 3 shows an example of flash-animation with the magnifier from TMC CADIS "Virtual training office for plane design", see. http://cnit.ssau.ru/virt_lab/indeõ.htm. The special technique is used here, allowing to see the plane completely and to study small components by moving the magnifier with the mouse.

VRML (Virtual Reality Modeling Language) opens the door to 3D interactive worlds instead of flat 2D images. Usage of the special VRML players as Plug-ins (the most modern browsers already includes such players as the standard software) allows to control of 3D virtual models.


Image 3. Example of flash-animation with the magnifier


Image 4 shows fragments of interaction with virtual model of TMC CADIS for organic chemistry for a secondary school.

  
Image 4. Fragments of interaction with VRML model of benzol molecule

This complex includes a lot of multimedia objects of different types: static graphics, 2D and 3D animations, interactive 3D VRML models of molecules, flash-animations, movie clips, see http://cnit.ssau.ru/organics/index.htm. When studying organic chemistry, it is very important to use cognitive multimedia functions which allow every student to form his own associations of spatial structures of organic compounds, its integrity and amount of connections. Understanding of peculiarities of the matter structures allows students to forecast (and not memorize!) its chemical and physical properties.

7. Conclusion
Separation of the cognitive multimedia function has a great mean for development of intuitive, iconic thinking, very important for many domains of professional activity. Understanding of this role of multimedia will allow the teachers to more clearly formulate requirements for multimedia objects used in computer-aided training systems, eliminate negative factors which are commom for computerization of studying, and realize the didactic potential of the new information technologies more completely.

Literature
1. A.A.Zenkin. Cognitive computer graphics / Edit. D.A. Pospelov, Science, Moscow, 1991.
2. A.V.Solovov. Designing computer-aided training systems: School-book, SSAU, Samara, 1995.
3. A.V.Solovov. Cognitive computer graphics in engineering trainings, Magazine "High education in Russia ", MGUP, Moscow, 1998, ¹ 2, p. 90-96.
4. D.A.Pospelov. Fantasy or Science. On the way to artificial intelligence, Science, Moscow, 1982.
5. G.V.Schekin. Brain skewness and psychological peculiarities of a human, Your testoteck, Inter-regional correspondence university of personnel management, Kiew, 1992, N2, p. 102-112.
6. N.G.Alexeev, B.G.Yudin. Evolution of the problems of artificial intelligence, Cybernetics: unlimited capabilities and possible limitations. Prospects of evolution, Science, Moscow, 1981, p. 117-124.
7. V.A.Komarov, A.V.Solovov. ÀÎÑ and engineeric intuition, Magazine "High school bulletin", High school, Moscow, 1986, ¹ 2, p. 30-33.