Content
- Introduction
- 1. Areas of application for generating image depth maps
- 2. Overview of generating a depth map from a stereo pair
- 3. Directions, goals and objectives of further research
- Conclusions
- List of sources
Introduction
The modern creation and distribution of multimedia content is moving towards the presentation of increasingly “realistic” scenarios. The transition from 2-dimensional to 3-dimensional representation was the main driving force in this direction. Recently, a large number of approaches have been proposed for creating 3D images/videos, most of which are based on creating depth maps, the number of multimedia data created and distributed around the world has grown significantly. This required active research in the field of image and video processing, while constantly developing applications and solutions for areas such as surveillance and security, healthcare, computer vision, and many others. The advent of digital cameras has revolutionized the way photography is taken. Compared to their counterparts, such digital cameras (including those present on mobile devices) provide quick and easy shooting, storage and image retrieval. However, most popular digital cameras are capable of capturing a 2-dimensional projection of the scene, while the components corresponding to the depth are lost (not recorded). The three-dimensional image appeared as an improvement to traditional 2D technology with additional depth information. 3D cameras have begun to appear on the market, but they are too expensive. For an ordinary user with a two-dimensional digital camera, three-dimensional images can be created by extracting depth information from two-dimensional images using various methods proposed previously. Among these methods, the most popular is creating a depth map from a stereo pair of images. It finds many uses in three-dimensional visualization, decoding of images in bright fields and other areas.
1. Areas of application for generating image depth maps
There are a large number of methods for generating image depth maps, given the wide scope of this technology and the large number of various modifications. Application areas include:
• Imitation of the effect of uniformly dense translucent media in the scene - such as fog, smoke or large volumes of water.
• Simulation of shallow depth of field - when some parts of the scene are out of focus.
• Z-buffering and z-cropping, methods that can be used to increase the rendering efficiency of 3D scenes.
• Shadow mapping - part of the process used to create shadows cast by lighting in three-dimensional computer graphics.
• Providing distance information needed to create and generate auto-stereograms and other related applications designed to create the illusion of three-dimensional viewing using stereoscopy.
• Subsurface scattering - can be used as part of the process of adding realism by modeling the translucent properties of translucent materials such as human skin.
• In computer vision, for the modeling of three-dimensional figures or their reconstruction, depth maps of images of one or more images or other types of images are used.
• In machine and computer vision, the processing of three-dimensional images using two-dimensional image tools.
2. Overview of generating a depth map from a stereo pair
The depth map is an image on which for each pixel, instead of color, its distance to the camera is stored. The depth map can be obtained using a special depth camera (for example, the Kinect sensor is a kind of such a camera), and can also be built on a stereo pair of images.
The idea behind building a depth map using a stereo pair is very simple. For each point in one image, it searches for a pair of points in another image. And by a pair of corresponding points, you can perform triangulation and determine the coordinates of their inverse image in three-dimensional space. Knowing the three-dimensional coordinates of the prototype, the depth is calculated as the distance to the plane of the camera.
The paired point must be sought on the epipolar line. Accordingly, to simplify the search, the images are aligned so that all the epipolar lines are parallel to the sides of the image (usually horizontal). Moreover, the images are aligned so that for the point with coordinates (x0, y0) the corresponding epipolar line is given by the equation x = x0, then for each point the corresponding pair point must be searched in the same line on the image from the second camera. This image alignment process is called rectification. Typically, rectification is accomplished by re-imageting and combining it with getting rid of distortion.
After the images are rectified, it's time for searching for the corresponding pairs of points. For each pixel of the left picture with coordinates (x0, y0), a pixel search is performed on the right picture. It is assumed that the pixel in the right picture should have coordinates (x0 - d, y0), where d is the value called mismatch/offset. The search for the corresponding pixel is performed by calculating the maximum of the response function, which, for example, can be the correlation of pixel neighborhoods. The result is a displacement map.
Actually, the depth values ??are inversely proportional to the amount of pixel displacement. The resolution of stereo vision systems that operate on the basis of this method is better at short distances, and worse at far distances.
3. Directions, goals and objectives of further research
As a method of generating a map of the depth of the image, the method of generating a depth map from a stereo pair will be used. This method is the most popular and has its own modifications. Despite its relative ease of implementation, this method is the most optimal.
The purpose and objectives of further research is a more detailed analysis of the functioning of the method of generating a depth map from a stereo pair and its optimization by speed and quality of work. The output product should be a modified system. Considering that one of the drawbacks of the method is the relatively low speed of image processing, a parallel implementation of depth map generation will be performed. The processor of the PC video card will be used as a parallel computing system for this system, and OpenACC will be the software standard for parallel programming.
Conclusions
In the course of the work, an analysis was made of existing methods for generating image depth maps and some of them were examined, as well as the areas of application of depth maps and the main limitations that arise when working with them. A depth map generation on a stereo pair was reviewed and this method was determined to be the most optimal, and a preliminary analysis of the possibility of parallelizing the system was performed. It is planned to use a parallel processor of a PC video card as a parallel computing system and organize work with it using OpenACC technology.
List of sources
- Depth map generation by image classification [Electronic resource] – Access mode: https://www.researchgate.net/
- Depth-Map Generation using Pixel Matching in Stereoscopic Pair of Images [Electronic resource] – Access mode: https://arxiv.org/
- Îñíîâû ñòåðåîçðåíèÿ [Electronic resource] – Access mode: https://habr.com/
- An overview of free viewpoint Depth-Image-Based Rendering (DIBR) [Electronic resource] – Access mode: http://www.apsipa.org/
- ÂÛ×ÈÑËÅÍÈÅ ÊÀÐÒÛ ÃËÓÁÈÍÛ ÑÒÅÐÅÎÈÇÎÁÐÀÆÅÍÈß ÍÀ ÃÐÀÔÈ×ÅÑÊÎÌ ÏÐÎÖÅÑÑÎÐÅ Â ÐÅÀËÜÍÎÌ ÂÐÅÌÅÍÈ [Electronic resource] – Access mode: https://www.fundamental-research.ru/
- 3D stereoscopic image pairs by depth-map generation [Electronic resource] – Access mode: https://www.researchgate.net/
- Depth map – Âèêèïåäèÿ [Electronic resource] – Access mode: https://en.wikipedia.org/
- DeepStereoBrush: Interactive Depth Map Creation [Electronic resource] – Access mode: https://www.researchgate.net/
- Disparity and depth map generation Monoscopic video to 3D [Electronic resource] – Access mode: https://www.compression.ru/
- 3D Stereoscopic Photography [Electronic resource] – Access mode: http://3dstereophoto.blogspot.com/
- Innovative 3D Depth Map Generation From A Holoscopic 3D Image Based on Graph Cut Technique [Electronic resource] – Access mode: https://arxiv.org/
- Ñòåðåîïàðà – Âèêèïåäèÿ [Electronic resource] – Access mode: https://ru.wikipedia.org/