Variety of Methods Based on Discrete Wavelet Transform
Effective Implementation of Multispectral Image Fusion
Remote sensing is the process of gathering information about an object, area or phenomenon without direct contact with it. Remote sensing techniques are based on recording in analog or digital form reflected or natural electromagnetic radiation from surface areas in a broad spectral range [1].
After intensive development, remote sensing has provided scientists and dedicated specialists new ways of studying the Earth's surface, ensuring the availability of a substantial amount of diverse data which have sufficiently high quality.
The product of remote sensing refers to digital images, which pixel values correspond to the values of some characteristics (reflectivity, temperature, etc.) of the relevant portion of the surface area. The size of this area depends on the resolution of the resulting image. In general, a set of products can be viewed as a multidimensional matrix of characteristics for a given area, defining the vector of characteristics for each segment of the area.
Because the data obtained from sensors mounted on satellites have different spatial, spectral, temporal and other resolutions [2], the question of image fusion becomes relevant in order to integrate the characteristics for their subsequent analysis. The information contained in the result of image fusion procedure is more complete and accurate [2], which leads to a clearer understanding of the properties of the object.
Image fusion problem has arisen at dawn of the development of remote sensing but remains relevant nowadays considering the continuing trend to higher requirements for resolution and quality of the material. In turn, these requirements give rise to new, consisting of the need for better performance of implementations of image fusion. In the context of this problem, in order to provide maximum computing power, it is reasonable enough to consider the possibility of modification (adaptation) of algorithms of these methods for their efficient implementation on parallel computer systems, in particular on the video cards, taking into account that modern graphics cards have reached unprecedented computing power, ensuring a high degree of parallelism at multiple levels (task level, command level, data level) [6]. It should also be noted that there are very few works (in the open access) on the effective implementation of image fusion techniques for graphics adapters. Moreover, problem of obtaining performance gains with the use of the cluster, each node of which is equipped with the aforementioned devices, is very interesting. Therefore, the present state of development of remote sensing technology, bringing increased demands for quality and performance of image fusion, makes the relevance of research on the above topic.
There are a variety of applications in the problem of image fusion, including:
Next will be considered fusion of images with different spectral resolution (multispectral image fusion).
Fusion of multispectral images (multispectral image fusion, pan sharpening) refers to the process, that lets to obtain single image from panchromatic (panchromatic, Pan) and multispectral (multispectral, MS) bands. Panchromatic channel typically has a higher spatial resolution than multispectral one (e. g., for the QuickBird satellite: Pan 0,61-0,72 m, MS 2,44-2,88 m [3]). At the same time, for example, in geographic information systems, it is necessary to obtain an image with both high spectral and spatial resolution. Fig. 1 can serve as an illustration of this process.
![Fig. 1 - Multispectral fusion of images obtained by IKONOS satellite. Left (Pan) and central (MS) - original image, right - the result of fusion [4]](abstract1.jpg)
Fig. 1 – Multispectral fusion of images obtained by IKONOS satellite. Left (Pan) and central (MS) – original image, right – the result of fusion [4]
Variety of methods were developed to solve the problem of improving the spatial resolution (pan sharpening, multispectral image fusion), including:
The above methods are representatives of different generations, starting with the simplest method – weighted averaging (eighties), and ending with methods based on discrete wavelet transform (our time) [5]. Next, concepts of the above methods are considered.
Weighted averaging method (one of the easiest methods) is a per-pixel superposition of images using weight coefficients [6]. This process is described by
I(x,y) = w1I1(x,y) + w2I2(x,y)
where Ik(x,y) – pixel that corresponds to the k-th source image, wk – weighting factor which determines the fraction of the participation of the original image in the resulting product.
Despite the fact that this fusion method is the simplest of the considered methods, and do not have the advantage of providing a higher quality of results, its simplicity leads to the fact that it requires virtually no optimizations, which in turn leads to some degree of the objectivity of the comparison of implementations for different platforms.
IHS method involves the transformation of the color model of multispectral images – the transition from the RGB (red, green, blue) color space to the IHS (intensity, hue, saturation), after which I channel is replaced with panchromatic channel. After that inverse transformation of color space is performed [5]. Schematic diagram of the IHS method is shown in Fig. 2.
Fig. 2 – Scheme of IHS method
PCA method is a transformation, in which mutually correlated bands of multispectral image (MS) is converted into a new set of uncorrelated components. Due to the similarity of the first component of this set with a panchromatic band, first component is replaced by the Pan, having a high resolution. After that inverse PCA transformation (IPCA, inverse PCA) [5] is applied. Schematic diagram of the PCA method is shown in Fig. 3.
Fig. 3 – Scheme of PCA method
One of the most modern techniques are ones that are based on discrete wavelet transform [7]. The main feature of these methods is applying of discrete wavelet transform (DWT, discrete wavelet transform) to the source images, a further merging of the obtained coefficients and obtaining result image using inverse transform (IDWT, inverse DWT). Generalized schematic diagram of the method is shown in Fig. 4.
Fig. 4 – A generalized scheme for the DWT-techniques (animation: 5 frames, 9 seconds, 25 KB)
Almost all of the existing methods of multispectral image fusion have their drawbacks, in particular color distortions, that make it impossible to select one of them as universal [5].
An effective implementation will further be understood as the implementation, providing faster processing of data while maintaining the quality level provided by basic implementation.
Majority of the image fusion methods uses a variety of computationally complex operations, which adversely affects the performance of their implementations, taking into account the dimension of remote sensing products (e. g., a product with a resolution of 15 m/px, covering the territory of a square with sides of 170 km, will have a dimension of 11 333 elements horizontally and vertically respectively).
In order to improve the performance of the aforementioned methods it can be considered to adapt their algorithms for implementation on parallel computer systems, in particular, on the modern graphics cards, that are able for general-purpose computing.
It should be noted that, of course, such attempts have been already made (for example, [6]). Papers devoted to the topic discuss the use of programmable pipelines of modern graphics cards by implementing the methods using shader programs (shaders) and multi-pass rendering. This approach pays off, allowing to achieve performance gains over single-threaded implementation [6] (see Table 1.).
| Image resolution | Processor | Weighted averaging (s) | DWT-based method (s) |
|---|---|---|---|
| 512x512 | CPU | 0,016 | 0,364 |
| 512x512 | GPU | 0,012 | 0,251 |
| 2048x2048 | CPU | 0,109 | 5,840 |
| 2048x2048 | GPU | 0,033 | 2,528 |
However, in most sources the problem of the use of multiple devices simultaneously have insufficient consideration, as well as the possibility of organizing the simultaneous work of a CPU (central processing unit) and GPU (graphics processing unit) on a single task.
The task of merging of multispectral images in remote sensing is quite important – the result gives a more complete picture of the object of research and is widely used, particularly in geographic information systems. There are quite a number of methods to solve this problem, each of which has its advantages and disadvantages.
At the same time question of effective implementation of these techniques to achieve high performance is quite important due to the large resolutions of the data that are obtained from sensors (this is caused by the constant evolution of technology).
Having reviewed the available publications on the implementation of the above methods on parallel computing systems, we can conclude that the issue of using graphical cluster (here the graphics cluster means a cluster, each node of which has a graphics card that enables general purpose calculations) is insufficiently studied. In particular, a sufficiently small number of publications devoted to the use of technology CUDA (Compute Unified Device Architecture, hardware/software architecture that allows to perform calculations using a NVIDIA's general purpose computations capable GPUs) in the context of solving the problem of multispectral image fusion in remote sensing. At the same time, the use of this technology can ensure success in solving the problem of finding efficient methods for implementing fusion schemes.
Considering aforementioned it is planned to investigate the possibility of adapting some of the algorithms for multispectral image fusion methods for their implementation on parallel computer systems with the aim of achieving better performance in processing of the data with large resolution.
At the time of writing of this abstract master's work is not complete. Expected completion date: December 1, 2011. As a consequence, the full text, as well as other materials can be obtained from the author or his supervisor after that date.