DonNTU   Masters' portal

Abstract

Contents

Introduction

Massive integration of information technologies into all aspects of modern life caused demand for processing vehicles as conceptual resources in information systems. Because a standalone information system without any data has no sense, there was also a need to transform information about vehicles between the reality and information systems [1]. It should be noted that the camera operates on the principle of online transmission of uncompressed data stream recorded by the camera in a circular buffer memory. Cameras also record the streams of uncompressed data and transmit them to the target device on a hard drive or RAM. Thus, the computer receives a very large flow of data to be processed quickly and write in the memory [2]. All these procedures take time. These operations must be optimized to increase the efficiency of the system.

This can be achieved by a human agent, or by special intelligent equipment which is be able to recognize vehicles by their number plates in a real environment and reflect it into conceptual resources. Because of this, various recognition techniques have been developed and number plate recognition systems are today used in various traffic and security applications.

Automatic number plate recognition systems can be used in access control. For example, this technology is used in many companies to grant access only to vehicles of authorized personnel. In some countries, such systems installed on country borders automatically detect and monitor border crossings. Each vehicle can be registered in a central database and compared to a black list of stolen vehicles. In traffic control, vehicles can be directed to different lanes for a better congestion control in busy urban communications during the rush hours.

1. Theme urgency

The urgency of developing a system of vehicle license plate recognition is that the intensity of vehicles has increased in recent years and now it’s needed to improve the automated control at checkpoints, as it is required to identify the offending vehicle in motion and control traffic on the road. The developed system will automatically recognize the vehicle license plate number, will write it to the database, or check it with the already stored data in the database, allowing you to make an automatic pass to the territory. Such monitoring will help to increase the effectiveness of the checkpoint, as it will not depend on the human factor.

This problem refers to the problem of efficient algorithms and recognition software of graphic images in real time. Development of a system that has the same functions, but has a more favorable economic decision in the subject matter of implementation, processing and performance – the current scientific and technical problem.

2. Goal and tasks of the research

Recognition of vehicle license plate includes the following steps: determining the position of the plate, its segmentation, extraction of the individual characters and their normalization, character recognition and parsing the resulting plate.

The goal is to study and develop methods and algorithms for character recognition, numbers and letters in the Russian/Ukrainian alphabet, providing analysis and processing the information from the surveillance cameras for further discovery, definition and identification of the vehicle license plate. The goal has identified the need to solve the following objectives:

  1. Analysis of the localization of license plate and its’ segmentation.
  2. Development of the structure and functions of the system for license plate characters processing.
  3. Research and development character recognition algorithms.
  4. Selection of a method for detecting and evaluating its effectiveness.
  5. Analysis of equipment that gets correct data from the surveillance cameras.
  6. Development of systems and application software for image processing from surveillance cameras to detect the vehicle license plate.

The completion of a workable plan means that the computer information management system was designed, and it’s ready for implementation in the control system of area checkpoints.

3. The main stages in the vehicle license plate recognition (LPR).

Automatic vehicle identification is an essential stage in intelligent traffic systems [3]. Nowadays vehicles play a very big role in transportation. Also the use of vehicles has been increasing because of population growth and human needs in recent years. Therefore, control of vehicles is becoming a big problem and much more difficult to solve. Automatic vehicle identification systems are used for the purpose of effective control. License plate recognition (LPR) is a form of automatic vehicle identification. It is an image processing technology used to identify vehicles by only their license plates. Real time LPR plays a major role in automatic monitoring of traffic rules and maintaining law enforcement on public roads [4].

Lotufo, Morgan и Johnson [5] proposed automatic number plate recognition using optical character recognition techniques. Fahmy [6] proposed bidirectional associative memories (BAM) neural network for number plate reading. It’s appropriate for small numbers of patterns. Nijhuis, Ter Brugge, Helmholf J.P.W. Pluim, L. Spaanenburg, R.S. Venema и M.A.Westenberg [7] proposed fuzzy logic and neural networks for car LPR. This method used fuzzy logic for segmentation and discrete-time cellular neural networks (DTCNN’S) for feature extraction. Choi [8] и Kim [9] proposed the method based on vertical edge using Hough transform (HT) for extracting the license plate. E.R. Lee, P.K. Kim и H.J. Kim [10] used neural network for color extraction and a template matching to recognize characters. S.K. Kim, D.W. Kim и H.J. Kim [11] used a genetic algorithm based segmentation to extract the plate region. Hontani [12] proposed a method for extracting characters without prior knowledge of their position and size in the image. Park et. al. [13] devised a method to extract Korean license plate depending on the color of the plate. H.J. Kim, D.W. Kim, S.K. Kim, J.V. Lee, J.K.Lee [14] proposed a method of extracting plate region based on color image segmentation by distributed genetic.

Structure of the LPR system

Input of the system is the image of a vehicle captured by a camera. The captured image taken from 4-5 meters away is processed through the license plate extractor with giving its output to segmentation part [3]. Segmentation part separates the characters individually. And finally recognition part recognizes the characters giving the result as the plate number.

Plate region extraction

Plate region extraction is the first stage in this algorithm. Image captured from the camera is first converted to the binary image consisting of only 1’s and 0’s (only black and white).By thresholding the pixel values of 0 (black) for all pixels in the input image with luminance less than threshold value and 1 (white) for all other pixels. Captured image (original image) and binarized image are shown in Figure 1. The binarized image is then processed using some methods. To find the plate region, firstly smearing algorithm is used. Smearing is a method for the extraction of text areas on a mixed image. With the smearing algorithm, the image is processed along vertical and horizontal runs (scan-lines) [3]. If the number of white pixels is less than a desired threshold or greater than any other desired threshold, white pixels are converted to black. After smearing, a morphological operation, dilation, is applied to the image for specifying the plate location. However, there may be more than one candidate region for plate location. To find the exact region and eliminate the other regions, some criteria tests are applied to the image by smearing and filtering operation. The processed image after this stage is as shown in Figure 1.

Segmentation

In the segmentation of plate characters, license plate is segmented into its constituent parts obtaining the characters individually. Firstly, image is filtered for enhancing the image and removing the noises and unwanted spots. Then dilation operation is applied to the image for separating the characters from each other if the characters are close to each other. After this operation, horizontal and vertical smearing is applied for finding the character regions [3]. The next step is to cut the plate characters. It is done by finding starting and end points of characters in horizontal direction. The individual characters cut from the plate are as follows in Figure 1.

Visual representation of vehicle license plate character selection process for further recognition

Figure 1 – Visual representation of vehicle license plate character selection process for further recognition (animation: 7 frames, 8 loops, 83 kilobytes)

Character recognition

Before recognition algorithm, the characters are normalized. Normalization is to refine the characters into a block containing no extra white spaces (pixels) in all the four sides of the characters. Then each character is fit to equal size as shown in Figure 1. Fitting approach is necessary for template matching. For matching the characters with the database, input images must be equal-sized with the database characters. The next step is template matching. Template matching is an effective algorithm for recognition of characters [3]. The character image is compared with the ones in the database and the best similarity is measured.

Example of template symbols is shown in Figure 2.

Template symbols

Figure 2 – Template symbols

Moving window with template matching method

Moving window using the template matching method (sum of squared differences) is a common and practical technique utilized in many pattern recognition applications [15, 16]. The template matching method gives high recognition accuracy and reduces the processing time compared to other methods such as cross-correlation. The applied method computes the sum of squared differences in each position while the word image we want to recognize moves over the background template. The point where the sum of squared difference is less than a preset threshold will be considered as the point of matching. First a window containing an object with size smaller than that of the main image is defined. Only a portion of the image is visible through this window. The template matching function is performed between the object in the window and the corresponding area of the image. Then the window is shifted and the template matching function is carried out between the object in the window and the new part of the image visible through the window. Thus, the window is moved left to right and top to bottom in single pixel displacement steps until the entire image is covered and template matching is carried out for all different window positions. Mathematically, distance measure is a measure of the similarities or shared properties between two signals. The distance metric commonly used is the Minkowski metric d(x,y) [17]:

Minkowski metric

Рисунок 3 – Minkowski metric

where X, Y are two N dimensional feature vectors, and r is a Minkowski factor. And when r is 2, it is actually Euclidean distance.

Moving window and template matching method

Рисунок 3 – Moving window and template matching method

Multi-layer neural network for license plate character recognition.

Multi-layer neural network can simulate the function of virtually of any complexity, the number of layers and the number of elements in each layer determines the complexity of the function [18]. Determination of the number of intermediate layers and the number of elements in them is an important issue in the design. Multi–layer neural networks can be divided into four most significant and important class of neural networks:

Good recognition results give neural back-propagation network and Kohonen networks. Back-propagation neural network structure similar to the Kohonen network, but operate and learn differently. The signal from the output neurons and hidden layer neurons is partially transmitted back to the inputs of neurons in the input layer (feedback) [19].

Among the benefits we can highlight that back-propagation – an effective and popular learning algorithm of multilayer neural networks, with the help of numerous practical problems are solved.

Feed-forward network

Рисунок 4 – Feed-forward network

Conclusion

The research goal was the studying of the algorithmic and mathematical aspects of the vehicle license plate recognition. The computer vision problems, pattern symbols and recognition, the usage of neural networks and other methods were mentioned. An effective camera image processing system must not only have the necessary equipment, but it also must have the appropriate software, which will properly and effectively conduct operations of controlling the territory, which is intended for the vehicle passage.

Research and choose of the most appropriate methods for image selection areas, processing methods of segmentation and binarization will reduce the requirements for the recognition. It is very important in the vehicle license plate recognition problem.

In the trials carried out:

  1. The structure of a software system and function components has been developed.
  2. The main algorithms that can be used in the design were selected for developing of automated system. This conclusion is based on the analysis of the literature highlights.
  3. The possibilities of integrated automation image processing system from surveillance cameras, requirements for the software were estimated, similar functional software were found and it was analyzed for the advantages and disadvantages.

Further studies focused on the following aspects:

  1. The effectiveness of character recognition methods gives a possibility of using a back-propagation neural network, Kohonen network, or other image patterns methods in the recognition of plate segmented characters.
  2. The known processing and image enhancement methods will be adapted. It will be useful in maximizing the quality of the plate recognition.
  3. The developing of functional automated image processing system from surveillance cameras will realize the proposed determining method of the vehicle license plate.

At the moment master's work is not complete yet. Final completion date: December 2012. The full text of the work and its materials can be obtained from the author or his scientific adviser after that date.

References

  1. Martinsky Ondrej – Algorithmic and mathematical principles of automatic number plate recognition systems [Текст] / B.SC. Thesis. — Brno University of technology, faculty of information technology, department of intelligent systems, 2007. — 76 с.
  2. FastVideo. Скоростная съемка с камер видеонаблюдения [Электронный ресурс]. – Режим доступа: Ссылка
  3. Serkan Ozbay, and Ergun Ercelebi – Automatic Vehicle Identification by Plate Recognition, World Academy of Science, Engineering and Technology 9, pp.222-225, 2005.
  4. Bailey D.G., Irecki D., Lim B.K. and Yang L., Test bed for number plate recognition applications, Proceedings of First IEEE International Workshop on Electronic Design, Test and Applications ( DELTA’02 ), IEEE Computer Society, page 501, 2002.
  5. Lotufo R.A., Morgan A.D., and Johnson AS., 1990, Automatic Number-Plate Recognition, Proceedings of the IEE Colloquium on Image analysis for Transport Applications, V01.035, pp.6/1-6/6, February 16, 1990.
  6. Fahmy M.M.M., 1994, Automatic Number-plate Recognition : Neural Network Approach, Proceedings of VNIS’94 Vehicle Navigation and Information System Conference, 3 1 Aug-2 Sept, pp.291-296, 1994
  7. Nijhuis J.A.G. , Brugge Ter M.H., Helmholt K.A., Pluim J.P.W., Spaanenburg L., Venema L., Westenberg M.A., 1995, Car License PlateRecognition with Neural Networks and Fuzzy Logic, IEEE International Conference on Neural Networks, pp.2232-2236, 1995
  8. Choi H.J., 1987, A Study on the Extraction and Recognition of a Car Number Plate by Image Processing, Journal of the Korea Institute of Telematics and Electronics, Vo1.24, pp. 309-315,1987.
  9. Kim H.S., et al., 1991, Recognition of a Car Number Plate by a Neural Network, Proceedings of the Korea Information Science Society Fall Conference, Vol. 18, pp. 259-262, 1991
  10. Lee E.R., P.K. Kim, and H.J. Kim, 1994, Automatic Recognition of a Car License Plate Using Color Image Processing, Proceedings of the International Conference on Image Processing, Lecture Notes in Computer Science, pp.307-314, 2005
  11. Kim S.K., Kim D.W., and Kim H.J., 1996, A Recognition of Vehicle License Plate Using a Genetic Algorithm Based Segmentation, Proceedings of 3rd IEEE International Conference on Image Processing, V01.2., pp.661-664, 1996
  12. Hontani H., and Koga T., (2001), Character extraction method without prior knowledge on size and information, Proceedings of the IEEE International Vehicle Electronics Conference (IVEC’01), pp. 67-72.
  13. Park, S. H., Kim, K. I., Jung, K., and Kim, H. J., (1999), Locating car license plates using neural network, IEE Electronics Letters, vol.35, no. 17, pp. 1475-1477.
  14. Kim H.J., Kim D.W., Kim S.K., Lee J.V., Lee J.K., 1997, Automatic Recognition of Car License Plates Using Color Image Processing, Engineering Design & Automation, 3(2), pp.215-225, 1997
  15. Fairhurst M.C and Hoque M.S, Moving Window Classifier: a new approach to off-line image recognition. Electronics letters, 36(7), pp.628-630, 2000.
  16. Hoque M.S and Fairhurst M.C, Face recognition using the moving window classifier. In proc. Of 11th British Machine Vision Conference (BMVC2000), Bristol, UK, pp. 312-321, 2000.
  17. B.Li, Chang E., and Wu Y., Discovery of a Perceptual Distance Function for Measuring Image Similarity, ACM Multimedia Journal Special Issue on Content based, pp.512-522, Volume 8, Number 6, p.512-522, 2003.
  18. Портал искусственного интеллекта. [Электронный ресурс]. – Режим доступа: Ссылка
  19. Головко В.А. Нейронные сети: обучение, организация и применение: Учеб. пособие по направлению подгот. бакалавров и магистров "Прикладная математика и физика"; Под общ. ред. А.И. Галушкина.— Москва: Редакция журнала "Радиотехника", 2001 .— 256 с.