Adaptive perceptual color-texture image segmentation

Junqing Chen, Thrasyvoulos Pappas

Sourse of information: http://spie.org/x8899.xml?ArticleID=x8899

By combining knowledge of human perception with an understanding of signal characteristics, natural scenes can be segmented into perceptually/semantically uniform regions.

Image segmentation has been one of the fundamental problems since the birth of computer vision. It provides the foundation for object-based image/video processing. Of particular interest when we developed our method is the content-based image retrieval (CBIR) system. Image segmentation remains an open research topic, especially the segmentation of natural scenes. Much of the difficulty in segmenting these types of images is due to the intrinsic complexities and uniformities in terms of low-level features such as texture and color (see, for example, Deng and Manjulath1). Another challenging aspect of image segmentation is the extraction of information that's perceptually relevant to humans (the ultimate users of many of these applications). This requires the extraction of (low-level) image features that can be correlated with high-level image semantics.

We proposed an image segmentation algorithm that is based on spatially-adaptive texture features. To this end, we developed two types of features: one describes the local color composition; the other, the spatial characteristics of the texture's grayscale component. The algorithm first develops these features independently and then separates the image into smooth and textured areas using the spatial texture features. It then combines the color composition and spatial texture features to consolidate textured areas into regions.

The color composition features consist of the dominant colors and associated percentages in the vicinity of each pixel. They are based on the estimation of spatially-adaptive dominant colors. This takes into account the fact that the human visual system cannot simultaneously perceive a large number of colors, and that region colors are spatially varying. This technique is superior to previous approaches based on the concept of extracting the dominant colors in the image by addressing the issue of spatial variations, one of the most common characteristics for images of natural scenes. Spatially-adaptive dominant colors can be obtained using the local intensity functions of the adaptive clustering algorithm (ACA). The perceptual similarity of the color composition features is determined by a modified optimal-color-composition-distance (OCCD) metric.

The spatial-texture features are based on a multiscale frequency decomposition with four orientation subbands (horizontal, vertical, and ±45°). We use the local energy of the subband coefficients as a simple but effective characterization of spatial texture. An important novelty of the proposed approach is that a median filter operation is used to distinguish the energy due to region boundaries from the energy of the textures themselves. We also show that, while the proposed approach depends on the structure of the frequency decomposition, it is relatively independent of the detailed filter characteristics.

The proposed segmentation algorithm combines the color composition and spatial texture features to obtain segments of uniform texture. This is done in two steps. The first relies on a multi-grid region growing algorithm to obtain a crude segmentation, Figure 1(a) The segmentation is crude due to the fact that the estimation of the spatial and color composition texture features requires a finite window. The second step uses an elaborate border-refinement procedure to obtain accurate and precise border localization by appropriately combining the texture features with the underlying ACA segmentation. This is shown in Figure 1(b).

Figure 1. Color and texture image segmentation

Several critical parameters of the texture features and segmentation algorithm can be determined by subjective tests. These include thresholds for the smooth/non-smooth classification, for determining the dominant orientation, and for the color composition feature similarity. The goal of the tests is to relate human perception of isolated (context-free) texture patches to the statistics of natural textures. Experimental results demonstrate that this perceptual tuning leads to significant improvements in segmentation performance.(For details of the algorithm please refer to Chen et al.

We are currently in the process of identifying category descriptors for regions that are obtained through the segmentation algorithm. We will then interpret the overall scene based on region descriptors. Once perfected, this technique will impact many applications, including CBIR systems.

Authors

Junqing Chen
Unilever Research and Development,
Trumbull, CT

Thrasyvoulos Pappas
Department of Electrical Engineering and Computer Science
Northwestern University
Evanston, IL

References:

1. Y. Deng, B. S. Manjunath, Unsupervised segmentation of color-texture regions in images and video, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol: 23, pp. 800-810, 2001.
2. W. Y. Ma et al, Tools for texture/color based search of images, Proc. SPIE, Vol: 3016, pp. 496-507, 1997.
3. T. N. Pappas, An adaptive clustering algorithm for image segmentation, IEEE Trans. Signal Processing SP, Vol: 40, pp. 901-914, 1992.
4. A. Mojsilović, Extraction of perceptually important colors and similarity measurement for image matching, retrieval, and analysis, IEEE Trans. Image Processing, Vol: 11, pp. 1238-1248, 2002.
5. http://www.eecs.northwestern.edu/
6. J. Chen, Adaptive perceptual color-texture image segmentation, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol: 14, pp. 1524-1536, 2005.