[back]

3D Reconstruction of Specular Surfaces
Silvio Savarese, Min Chen and Pietro Perona

Abstract. Specular reflections carry valuable information on surface shapes. A curved mirror surface produces "distorted" images of the surrounding world. For example a straight line reflected by a curved mirror is in general a curve. It is clear that such distortions are systematically related to the shape of the surface. Our goal is to explore the geometry linking the shape of a curved mirror surface to the distortions produced on a scene it reflects. To this effect, we assume a simple known (calibrated) scene composed of lines passing through a point. We demonstrate that local shape geometry of the surface may be recovered from local deformation of the reflected images of at least three intersecting lines.

Figure 1. M.C. Escher (1935). Still Life with Spherical Mirror

Introduction and motivation. One of the main tasks of a visual system is computing the shape of the objects. A number of cues, notably stereoscopic disparity, texture gradient, motion parallax, contours and shading, have been shown to carry valuable information on surface shape, and have been studied extensively both computationally and psychophysically. The study of shading has been mostly restricted to the Lambertian case. Specularities have been mostly ignored with the significant exception of the analysis of specular highlights. We are interested in exploring the possibility of recovering information on the shape of a surface from the specular component of its reflectance function. Since we wish to ignore the contributions of shading, we will study surfaces that are perfect mirrors. A curved mirror surface produces `distorted' images of the surrounding world (see Fig.1). For example, the image of a straight line reflected by a curved mirror is, in general, a curve. It is clear that such distortions are systematically related to the shape of the surface. Is it possible to invert this map, and recover the shape of the mirror from the images it reflects? Our goal is to explore the geometry linking the shape of a curved mirror surface to the distortions produced on a scene it reflects. To this effect, we assume a simple calibrated scene composed of lines passing through a point. We demonstrate that local information about the geometry of the surface may be recovered from local deformation of the reflected images of at least three intersecting lines. We view our analysis as a promising start in the quest of computing the global shape of specular surfaces under fairly general conditions. The case of an uncalibrated world appears much more challenging and will require most certainly the integration of additional cues and some form of prior knowledge on the likely statistics of the scene geometry.

Method. We assume a known (calibrated) scene composed of one point p and at least 3 lines through it (see Fig. 2). The point and the lines are reflected off the unknown mirror surface into the camera image plane of a known (calibrated) camera, yielding a deformed version of scene around the intersecting point q. First, we have studied this mapping and found an analytical expression which describes the deforming local behavior of the mirror surface acting on the scene around p. Then, we have studied the inverse problem and derived local surface shape as function of the measurements. By using the orientations of at least 3 reflected curves in the image plane (see for instance red dashed lines in Fig. 2) as a measurement, we have showed that it is possible to recover i) mirror surface position, i.e. the distance of the reflected point r from the camera center c, ii) surface orientation, i.e. the normal of the surface at r, iii) surface shape up to second order (up to one unknown scale parameter for generic surfaces).

Figure 2. Reconstruction of a planar mirror.

Experimental Results. We validated our theoretical results with a set of experiments with real mirror surfaces. In our experiments, a Canon G1 digital camera, with image resolution of 2048x1536 pixels, was used. The surface was typically placed at a distance of 30-50 cm from the camera. The pattern -- a set of planar triplets of intersecting lines -- is formed by a tessellation of black and white equilateral triangles. For instance, 3 white dashed edges as in Fig. 3 form a triplet of lines. The camera and the ground plane (i.e. the plane where the pattern lies) were calibrated by means of standard calibration techniques. We validated the method with four mirror surfaces: a plane (Fig. 3), a sphere (Fig. 4), a cylinder (Fig. 5) and a sauce pan's lid (Fig. 6). Where we had a ground truth to compare with, we qualitatively tested the reconstruction results. As for the plane, depth and normal reconstruction errors are about 0.2% and less than 0.1% respectively. As for the sphere, the curvature reconstruction error is about 2%.

Figure 3. Reconstruction of a planar mirror. Upper left panel: a planar mirror placed orthogonal with respect to the ground plane. A triplet of pattern lines and the corresponding reflected triplet are highlighted with dashed lines. We calculated the ground truth on the position and orientation of the mirror by attaching a calibrated pattern to its surface. We then reconstructed 15 surface points and normals. The resulting mean position error (computed as average distance from the reconstructed points to the ground truth plane) is 0.048cm with a standard deviation of 0.115 cm. The mean normal error (computed as the angle between ground truth plane normal and estimated normal) is 1.5x10-4 rad with a standard deviation of 6.5x10-4 rad. The reconstructed region is located at about 50 cm to the camera. Upper right: 3/4 view of the reconstruction. For each reconstructed point, the normal and the tangent plane are also plotted. Lower left: top view. Lower right: side view.

Figure 4. Reconstruction of the sphere. Left panel: a spherical mirror with radius r = 6.5 cm, placed on the ground plane. We reconstructed the surface at the points highlighted with white circles. For each surface surface point we estimated the sphere's radius. The mean reconstructed radius is 6.83 cm and the standard deviation is 0.7 cm. The reconstructed region is located at a distance about 30 cm to the camera. Right panel: top view of the reconstruction.

Figure 5. Reconstruction of the cylinder. Left panel: a cylinder placed with the main axis almost orthogonal to the ground plane. We reconstructed the surface at the points highlighted with white circles. Right: top view of the reconstruction.

Figure 6. Reconstruction of the sauce pan's lid. Left panel: a sauce pan's lid placed with the handle touching the ground plane. We reconstructed the surface at the points highlighted with white circles. Notice that one point belongs to the handle of the lid. Right panel: side view of the reconstruction. Notice how the reconstructed point on the handle sticks out from the body of the lid.

Publications.
[1] S. Savarese and P. Perona, "Local Analysis for 3D Reconstruction of Specular Surfaces", in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, Kawa'i, USA, December 2001.

[2] S. Savarese and P. Perona, "Local Analysis for 3D Reconstruction of Specular Surfaces -- part II", in Proc. of 7th European Conference of Computer Vision, Denmark, May 2002.
[3] S. Savarese, M. Chen and P. Perona, "Second Order Local Analysis for 3D Reconstruction of Specular Surfaces", in Proc of 1st International Symposium on 3D Data Processing, Visualization and Transmission, IEEE Press, 2002