Monitoring and evaluation of the quality of training systemsContentRelevanceRelevanceDevelopment of education has always been important issue in society, and today it has not lost its significance. Today education should solve many problems. Of particular importance among them at this stage is the introduction of modern technologies in the educational process. Today we can watch how quickly increases number of learning systems that are based on recent advances in computer technology. However, not all of them there is an effective educational product. Problems creating quality learning systems (LS) specifically increased in the last 10-15 years, apparently due to the beginning of deep of the general crisis national educational systems. At the core of the crisis is the same "information boom", but its effect was not as significant as after the appearance of large numbers of personal computers, e-mail and the Internet, which greatly increased the amount of information available, the quality and speed of work with it compared to the previous period, when the major data carriers were paper, film, photographic film, etc. [1]. Goals and tasksThe purpose of research is to form a specific set of criteria for assessing the quality of LS based on modern computer technics and computer technology. As well as the development of an expert system based on the obtained set of quality criteria that will evaluate the effectiveness of the learning system. The system purpose - the correct evaluation the quality of the learning system based on a set of created criteria for evaluating the quality of the LS. The main research problems: – formation of a set of quality criteria; – analysis and choice the process metod expert judgment, decision-making; – realization an expert system. Expected scientific noveltyExpected scientific novelty consists in modifying the process of building the knowledge base the specialized expert system (ES), which uses a mathematical tool systemic approach to expert judgments. Planned practical resultsAs a result of this master's work it is expected to perform the following: – formation of the set of quality criteria; – analysis and choice the process metod expert judgment, decision-making; – monitoring of the LS based on the selected method; – development of rules for the expert system based on a set of quality criteria; – realization the expert system; – testing of the system. Analytic hierarchy processThe analytic hierarchy process (AHP) is a structured technique for organizing and analyzing complex decisions, based on mathematics and psychology. It was developed byThomas L. Saaty in the 1970s and has been extensively studied and refined since then. Rather than prescribing a "correct" decision, the AHP helps decision makers find one that best suits their goal and their understanding of the problem. It provides a comprehensive and rational framework for structuring a decision problem, for representing and quantifying its elements, for relating those elements to overall goals, and for evaluating alternative solutions. Users of the AHP first decompose their decision problem into a hierarchy of more easily comprehended sub-problems, each of which can be analyzed independently. The elements of the hierarchy can relate to any aspect of the decision problem-tangible or intangible, carefully measured or roughly estimated, well or poorly understood-anything at all that applies to the decision at hand. Once the hierarchy is built, the decision makers systematically evaluate its various elements by comparing them to one another two at a time, with respect to their impact on an element above them in the hierarchy. In making the comparisons, the decision makers can use concrete data about the elements, but they typically use their judgments about the elements' relative meaning and importance. It is the essence of the AHP that human judgments, and not just the underlying information, can be used in performing the evaluations [2]. The AHP converts these evaluations to numerical values that can be processed and compared over the entire range of the problem. A numerical weight or priority is derived for each element of the hierarchy, allowing diverse and often incommensurable elements to be compared to one another in a rational and consistent way. This capability distinguishes the AHP from other decision making techniques. In the final step of the process, numerical priorities are calculated for each of the decision alternatives. These numbers represent the alternatives' relative ability to achieve the decision goal, so they allow a straightforward consideration of the various courses of action. Model the problem as a hierarchy The first step in the analytic hierarchy process is to model the problem as a hierarchy. In doing this, participants explore the aspects of the problem at levels from general to detailed, then express it in the multileveled way that the AHP requires. As they work to build the hierarchy, they increase their understanding of the problem, of its context, and of each other's thoughts and feelings about both. Hierarchies in the AHP An AHP hierarchy is a structured means of modeling the decision at hand. It consists of an overall goal, a group of options or alternatives for reaching the goal, and a group of factors or criteria that relate the alternatives to the goal. The criteria can be further broken down into subcriteria, sub-subcriteria, and so on, in as many levels as the problem requires. A criterion may not apply uniformly, but may have graded differences like a little sweetness is enjoyable but too much sweetness can be harmful. In that case the criterion is divided into subcriteria indicating different intensities of the criterion, like: little, medium, high and these intensities are prioritized through comparisons under the parent criterion, sweetness. Published descriptions of AHP applications often include diagrams and descriptions of their hierarchies. The design of any AHP hierarchy will depend not only on the nature of the problem at hand, but also on the knowledge, judgments, values, opinions, needs, wants, etc. of the participants in the decision-making process. Constructing a hierarchy typically involves significant discussion, research, and discovery by those involved. Even after its initial construction, it can be changed to accommodate newly-thought-of criteria or criteria not originally considered to be important; alternatives can also be added, deleted, or changed. To better understand AHP hierarchies, consider a decision problem with a goal to be reached, three alternative ways of reaching the goal, and four criteria against which the alternatives need to be measured. Such a hierarchy can be visualized as a diagram like the one immediately below, with the goal at the top, the three alternatives at the bottom, and the four criteria in between. There are useful terms for describing the parts of such diagrams: Each box is called a node. A node that is connected to one or more nodes in a level below it is called a parent node. The nodes to which it is so connected are called its children. Applying these definitions to the diagram below, the goal is the parent of the four criteria, and the four criteria are children of the goal. Each criterion is a parent of the three Alternatives. Note that there are only three Alternatives, but in the diagram, each of them is repeated under each of its parents. Figure 1 – A simple hierarchy AHP Once the hierarchy has been constructed, the participants analyze it through a series of pairwise comparisons that derive numerical scales of measurement for the nodes. The criteria are pairwise compared against the goal for importance. The alternatives are pairwise compared against each of the criteria for preference. The comparisons are processed mathematically, and priorities are derived for each node. By definition, the priority of the Goal is 1.000. The priorities of the alternatives always add up to 1.000. Things can become complicated with multiple levels of Criteria, but if there is only one level, their priorities also add to 1.000. All this is illustrated by the priorities in the example below [13].
Figure 2 – Simple AHP hierarchy with associated default priorities Based on the of this method will be created knowledge base and the expert system for evaluating the quality learning systems. The creation and use the expert system shown in Figure 3.
Figure 3 – The process of creating and using ES ConclusionIn the master's work being upgraded the approach to forming the specialized a knowledge base for the expert system that is based on the method of analysis of hierarchies. The expert system proposed in this master's work, allow users to rate the quality of the LS, which will improve the quality of education. Likewise, the expert system will greatly simplify and speed up the process of estimation of quality the LS. References
| |
Elena Cherepkova
Faculty of computer science and technology (CST)
Computer monitoring systems
Speciality "Computer-aided Monitoring Systems in Ecology and Economy"
Monitoring and evaluation of the quality of training systems
Scientific adviser: Gubenko Natalia
