ABSTRACT:
Despite geospatial information systems are widely used in many different fields as a powerful tool for spatial analysis and decision- making, their capabilities to handle realistic 3-D urban environment are very limited. The objective of this work is to integrate the recent developments in 3-D modeling and visualization into GIS to enhance its 3-D capabilities. To achieve a photorealistic view, building models are collected from a pair of aerial stereo images. Roof and wall textures are respectively obtained from ortho- rectified aerial image and ground photography. This study is implemented by using ArcGIS as the work platform and ArcObjects and Visual Basic as development tools. Presented in this paper are 3-D geometric modeling and its data structure, texture creation and its association with the geometric model. As the results, photorealistic views of Purdue University campus are created and rendered with ArcScene.
1. INTRODUCTION
Three-dimensional (3-D) photorealistic visualization is one of the most important functionalities in geospatial information system (GIS). Such 3-D photorealistic visualization is able to provide geospatial information for easy human perception. It is helpful in geospatial information interpretation and the decision making process (El-Hakim, 1998). Through the past years, many successful models are developed to represent typical geospatial features such as terrain, road, water body, vegetation and many others. However, these are primarily 2- D or 2.5-D in nature. 3-D features like buildings may not be represented well by existing 2-D data structures and tools (Braun, et, al, 1995). An effective geometric modeling approach is needed to model the spatial location and the topology of buildings. In addition, this model should also be extended to include other information for buildings, such as attributes and image textures for query and visualization purpose. Using image as texture will not only increase the realism of the model but also enable the viewing of details. It improves the interpretability; in other words, the relationship between objects can be perceived easily and reliably when photorealistic models are used in visualization (Varshosaz, 2003). Although there are a lot of advanced studies in computer graphics and commercial products in computer industry, they do not typically handle geospatial information and can not be easily integrated into existing GIS packages to enhance its visualization capability, which is becoming a demanding functionality in processing 3-D geospatial information. On the other hand, GIS is a popular and standard system with powerful 2-D spatial analysis capabilities. However, its 3-D functionality is very limited where photorealistic visualization is certainly a very useful addition.
The objective of this study is therefore to develop a 3-D data structure, incorporate it into GIS and implement it for 3-D photorealistic visualization to enhance current GIS capabilities. We first discuss the suitability of the geometric building models to be used for 3-D modeling and visualization. Then, a 3-D geometric approach for building modeling is proposed. Under this model, a building is composed of two parts; roof and (vertical) walls. To achieve photorealistic visualization, these 3- D geometric models are extended through texture mapping with aerial photos for roof and ground photos for walls. In addition, the algorithm associating geometric models with corresponding texture images is discussed. After photorealistic building model and appearance are achieved through modeling and texturing, we discuss how to integrate 3D building model with other geospatial data to generate the entire urban model. These developments are integrated into ArcGIS package to enhance its 3-D visualization capabilities. Precise texture generation and automatic texture mapping are discussed as an important procedure to achieve high quality visualization effect. To demonstrate and evaluate our developments, photorealistic virtual campus models over Purdue University are presented.
2. GEOMETRIC BUILDING MODELS
Most GIS packages are designed for modeling terrain and other 2-D features. It is therefore not suitable for building modelling. In such data models, the 3-rd dimension usually is obtained by vertically extruding the 2-D footprint. In this approach, only horizontally flat roofs or piecewise horizontally flat roofs can be modeled. Hence we need a true 3-D model and incorporate it into GIS.
There are many methods to model 3-D objects, such as wire- frame model, surface model, and solid model. Usually GIS packages use surface model and solid model is widely used in the field of CAD (Computer Aided Design) and computer graphics. Recent trend is to adopt the solid model to represent buildings under CAD environment. The solid model such as CSG (Constructive Solid Geometry) and B-rep (Boundary representation) are very suitable to represent objects with complex geometry and their physical characteristics. However, it requires large memory and is relatively slow in displaying because of the complex data structures and evaluation of the data during rendering. Figure 1 shows the wire frame model and the surface model for the same group of buildings. In the wire frame model, edges are formed from vertices and no explicit information exists about faces of the object. The dot symbol in the wire frame model stands for the closing vertex of a polygon, based on which the faces for rendering are generated. The surface model is an advanced wire frame model. The object is represented not only by edges but also by surface components called patches, each of which may consist of several polygons. These polygons can be simply represented by vertices and their topology if necessary. The data structure is simple so it can ensure fast rendering. However, as a trade-off, the polygon- based surface modeling may not be adequate to show the curve objects with high fidelity because the number of segments required to compose the curve is increased proportional to the radius of curvature (Foley, et, al, 1993). After comparing both modeling methods, we decide to apply the surface model in this paper. The reasons are its simplicity and good supportability by GIS tools. Also, attribute data can be directly associated to the building object because it consist of polygons. In addition, buildings in GIS are usually not considered as solid objects and are mostly composed of straight edges or boundaries. Therefore, the advantage of the solid model and disadvantage of the surface model are marginal in case of modeling the buildings for visualization applications.
(a)
(b)
Once the modeling method is selected, we consider its association with the corresponding image textures. In general, 3-D graphic software use the directional projection method for texture mapping so we need to avoid that the patches are overlapped when they are projected on the plane. For this reason, the geometric building model is separated into roof and wall two parts. In our approach the shapefile format of ESRI (Environmental System and Research Institute) is adopted and enhanced for 3-D building modelling and visualization. This file structure consists of a main file, an index file, and a dBASE table. The main file is a direct access, variable-record-length file in which record describes the shape with a list of its vertices. The index file contains the offset of the corresponding main file record and the dBASE table contains the feature attributes. (ESRI White Paper, 1998) The main file supports shape types in 3-D space such as PointZ, MultiPointZ, PolyLineZ, PolygonZ and MultiPatch. Usually the MultiPatch shape is good and suitable for 3-D drawing. However, it is difficult to include various topology and spatial analysis functionalities provided by GIS and to directly link the attribute data to geometric object. Therefore, we choose PolygonZ type and modify it to fit to our objective for building modeling and visualization. This will be addressed in the next sections.
The shapefile containing polygons with z-value is created when digitizing a pair of aerial stereo images using the Stereo Analyst of Erdas IMAGINE. The initial data from the Stereo Analyst for the building models contains the 3-D coordinates of roof vertices and z-values of building footprint at the ground level. The coordinates of walls can be automatically generated using such information in shapefile based on the assumption that walls are vertical to ground. Hence, if some parts of building have no vertical relationship with ground foot, they are considered as a roof in terms of model structure even they are walls in real. These geospatial data of vertical walls is added into the roof shapefile. The main file structure of the modified shapefile for building is shown below in Figure 2. In this way, we successfully integrate all building components in to one compact data model, which will benefit the texture mapping and model rendering process in the subsequent steps.
Figure 2. The Data Structure of Main File for Building Model
3. TEXTURE CREATION
For the photorealistic building modeling, each building facade (a planar face, either vertical or not vertical) requires an association to a realistic texture that is possibly composed of a colour image with three bands, or the RGB colour intensities in the simplest case (Gulch, E, 1997). Two different ways are used to acquire and associate the images to building objects, respectively for roof and walls. As mentioned earlier, a building is separated to roof and walls for geometric model and texturing. For roof texture, the ortho-rectified aerial image, created using the building models, is used; while ground images collected from handheld cameras are used for wall textures.
3.1 Roof Texture
The scanned aerial photo is used to create roof texture. To avoid many small image files, all building roof objects in the study area are associated to their texture images at one time based on the photogrammetric principle. The collinearity equations are involved in this calculation for image data retrieval. However, it is not efficient to associate the texture image to the roof object with rigid photogrammetric model every time when we display the building model. To solve the problem, we generated and used the ortho-rectified image in which the image values have the same coordinates system as building roof model. After that, we can define the relationship between images and 3-D building objects directly for texture mapping. The location of the image segment for a building roof texture in an aerial image and orientation are most important parameters to map the texture. In this paper, an ortho-rectified image, which has contents only inside each building boundary and contains null outside, is generated as building roof texture. This is accomplished by enhancing current ArcGIS with advanced photogrammetric capabilities to handle model based true ortho rectification. Figure 3 presents an example of such generated ortho image for roof texture.
(a)
(b)
3.2 Wall Texture
The building wall pictures were taken on the ground to create the wall textures with a digital camera and a 35mm roll film camera. In case of roll film camera, we got the image after developing and scanning using a desktop scanner. For the wall texturing, we use a different way with roof texturing. Each wall is related to one individual image file. In addition, the wall texture is directly corresponding with view plane without 3-D building model. The detail methodology about the correspondence between the wall objects and image files and the association with view plane will be discussed in next section.
4. PHOTOREALISTIC MODELING
Once the geometric models and texture images are prepared, the photorealistic building model can be generated using the ArcObjects programmed and embedded in ArcGIS for this study. ArcObjects are a set of computer objects specifically designed for programming with ArcGIS Desktop applications base on the object-oriented programming (Zeiler, 2001). We use such development to automatically select wall texture images and associate them to the geometric building model. Figure 4 illustrates the general procedure for generating a photorealistic model by integrating all necessary data.
(b)
After generating the 3-D geometric building model, the building roof can be textured using the ortho-rectified image segment that is generated with aerial photo image. The assignment of texture image to the right building objects is accomplished by using the geometric relations between 2-D coordinates of texel and 3-D coordinate of building model through the orthographic projection equation. Twelve parameters are required to represent the 3-D object on the 2-D view plane as shown the following homogeneous transform matrix (Hearn, et, al, 1994). In this matrix, u and v are pixel coordinates in screen, X, Y, and Z object coordinates, R rotation matrix with element Rij, t shift, a,m and v intrinsic parameters and s and T perspective parameters.
We can display the 3-D building object with texture conveniently using perspective projection based on viewing angles and scale after the relationship between building geometric model and ortho-rectified image is prepared by the method mentioned before. The rendering result is shown in Figure 5a, where the ortho-rectified roof image is used as texture and overlaid atop building model in ArcScene.
However, this data structure is not adequate to represent the texture for both roof and walls at the same time in ArcScene for display. ArcScene doesn’t support the PolygonZ format that we chose for storing geometry model of the buildings to display if it has several patches belong to the same object. Therefore, we cannot use the rendering class of ArcScene for wall texturing although the shapefile contains the geospatial information of vertical walls. They need to be treated separately.
(b)
To display wall textures, we add a graphic layer to represent the texture of vertical wall. That layer is generated only for visualization and has no concern about data structure or query because it doesn’t have any attribute data and geospatial data. All information about the vertical walls is contained in the same shapefile that includes the polygon data for the roof. Using the spatial data about building wall in building layer, we create the wall polygon temporarily and associate the texture with that polygon. After that, we get the parameters that relate the texture image with the view plane using the same way as roof texturing and then we change the polygons with texture into graphics and save them. This process is repeated until every wall texture of entire building is generated. The next step is then to link the image file to the appropriate wall. Taking pictures of all building wall in the study area is almost impossible and unnecessary practically. Therefore, we categorized the building walls according to the material, color, and the height-width ratio. Three types of walls are used in this study: dark red brick, light red brick, and gray concrete with various sizes. ArcObjects for 3-D photorealistic building modeling can select and assign the wall texture images, which are grouped already, to the temporary wall object automatically. That program calculates the ratio of height and width of image and then compares it with ratio of facade for vertical walls. After it finds out the facades that have most similar ratio, the image is textured on it. Figure 5b is the result with both roof and wall textures being rendered.
Once the 3-D model is built, we can load this model into ArcScene or other commercial GIS software that use the shapefile format. The advantage of this is that we are able to use the whole functions which are already existed in the commercial GIS software such as changing view point, shadowing, adding and combining data and so on. The building model can be added to city model and become more realistic using those functions.
The entire procedure mentioned above is applied to create a photorealistic virtual model for the Purdue campus in West Lafayette. During this process, the textures for the walls are selected automatically according to the conditions mentioned earlier. After that, the terrain model is generated with aerial picture and DEM (Digital Elevation Model). Finally, the virtual model for Purdue campus shows up through integration of 3D building model and the terrain model. Figure 6 shows the textured buildings in a closer view. This picture shows the image textures fit into the building models well both in the corner area and in the central area of the entire campus model. Figure 7 represents the campus model at the ground level view, while Figure 8 is the panorama of the campus model. The result is more realistic if appropriate texture were selected through manual editing and more texture groups were used according to color and material of walls. In case of landmark such as a tower and a statue and environmental objects such as tree and rock, it might be hard to model and texture them with current data model and texture mapping methods. They need to be modeled separately and will be considered in future study.
(a)
(b)
Figure 7. Ground View of the 3-D Photorealistic Model for Purdue Campus
Figure 8. Bird View of 3-D Photorealistic Model for Purdue Campus
5. CONCLUDING REMARKS
This study has successfully integrated photorealistic visualization capabilities into a 3-D GIS. Some useful experience is gained through this process. First, a 3-D geospatial geometric model needs to be carefully selected based on the multiple considerations, such as the definition of 3-D geometry and topology, support of geospatial query, and effective texture mapping. It is shown that PolygonZ structure is an appropriate choice that can accommodate all the considerations with certain amount development and enhancement. Next, texture mapping is a crucial step in photorealistic visualization. In our study, a model based ortho rectification is developed to produce true ortho image based on building models. In this way, all building roofs can be easily mapped to the texture to achieve an effective 3-D rendering. Third, we successfully achieved an automatic association of building walls to their corresponding textures. This significantly reduces the amount of time for texture mapping, which is usually carried out manually.
Future efforts will be made to improve our work. Other possibilities in effective 3-D data model will be explored to further simplify the texture mapping and rendering operations. Another problem is the consistency between roof texture and vertical wall texture in terms of color, resolution, contrast and spatial accuracy. In addition, we will make development to include more complex geospatial features and other geospatial information.
REFERENCES
- Braun C., Kolbe T.H., Lang F., Shickler W., Steinhage V., Cremers A.B., Forstner W., Plumer L. (1995). “Models for Photogrammetric Building Reconstruction.” Computer & Graphics, Vol. 19, No. 1, pp. 109-118.
- Bruke R.(2003) “Getting to know ArcObjects” ESRI Press
- El-Hakim S., Brenner C., Roth G.(1998). “An approach to creating virtual environments using range and texture.” International Archives of Photogrammetry and Remote Sensing, Vol. 32, pp. 331-338, Hakodate
- Environmental System and Research Institute (1998) “ESRI Shapefile Technical Description” ESRI Press
- Foley J. D., Dam A. V., Feiner S. K., Hughes J. F, Phillips R. L.(1993), “Introduction to Computer Graphics”, Addison- Wesley
- Gulch E.(1997) “Application of Semi-Automatic Building Acquisition”, Automatic Extraction of Man-Made Objects from Aerial and Space image, Birkhauser Verlag, Vol. 2, pp129-138
- Hearn D., Baker M. P.(1994) ” Computer Graphics” ,2nd Ed., Prentice-Hall
- Varshosaz M. (2003) “True Realistic 3-D Models of Building in Urban Areas”. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XXXIV-5/W10
- Zeiler M.(2001) “Exploring ArcObjects” ESRI Press.