Scanner to camera calibration
Calibration
A nodding laser scanner (Hokuyo URG-30LX) produces a 3D point cloud of a scene that is also imaged by a camera. The camera is fixed rigidly with respect to the nodding mechanism. A calibration of the system would produce the 3D offset between the camera and the laser scan, and a concomitant projection matrix for 3D points of the scanner onto the camera image.
Currently the STAIR group has a system that extracts planes and points from the 3D cloud, finds correspondences to edges and points in the camera image (with some user input), and then solves the PnP problem to produce a calibration. Errors are typically about 7 pixels. A further stretch-fit of the laser scan using MRFs can achieve about 5 pixels error.
Theory of Operation
The Hokuyo URG-30LX can output intensity images in the near-IR range. We take advantage of this, and use a planar checkerboard target to find reliable points in both camera and scanner images. The left image below is from a wide-angle camera at 640x480, the right an IR image from the nodding Hokuyo, reprocessed by Hai Nguyen to perspective projection on a 640x480 image plane. The scanner data is slightly less dense than the image, so Hai does very nice interpolation derived from color interpolators, preserving dominant edges.
- Camera image:
- Nodding scanner image, 140 degrees, hemispherical projection:
- Nodding scanner image, ~100 degrees, planar projection:
Using just the standard stereo tools, I ran stereo calibration on the images, and it worked with no tweaking. The relevant parameters are:
Left Intrinsics:
fx: 5.193494 - horiz and vert focal lengths, in mm (5.2 um pixels)
fy: 5.197921
cx: 317.396940 - center of projection on the image (pixels)
cy: 235.300751
k1: -0.323922 - radial distortion parameters (just two terms)
k2: 0.143630
k3: 0.000000
t1: 0.000000 - tangential distortion params, not used
t2: 0.000000
Right Intrinsics:
fx: 3.309819
fy: 3.400269
cx: 319.327720
cy: 234.540393
k1: 0.000783
k2: 0.002958
k3: 0.000000
t1: 0.000000
t2: 0.000000
Externals:
Tx: -199.362598 - translation from left to right camera (mm)
Ty: 48.659224
Tz: 20.591577
Rx: -0.026108 - rotation from left to right camera (rads)
Ry: 0.018103
Rz: -0.013716
Note the very different focal lengths (the right image is the Hokuyo, which has a very wide FOV, of which we're just using about 140 degrees). The distortion parameters are fine (I used a 2-parameter radial distortion model), they show very low distortion on the Hokuyo, as expected, and high distortion on the wide-angle camera. Theoretically there should be no distortion at all on the Hokuyo, and we could constrain the optimization to recognize this, but I think it would make little difference.
The externals seem correct. The baseline between the devices is about 20 cm, and there is significant Y and Z offset too.
Now for the kicker: RMS error from epipolar lines is less than a pixel, 0.79 pixels to be precise. This is scattered over the whole image set, where the target is placed differently in different images, and with taking 9 different Hokuyo scans, so really the calibration is very stable (thanks to Jeremy for getting the Hokuyo readings placed accurately). Perhaps the points are a little weighted towards the center of the image, so I'll look into eliminating some of them for error calculations, and the error may go up slightly. But... less than a pixel is just great, the targets and intensity data are a big win.
- Left and right rectified images (epipolar lines are horizontal)
It's somewhat interesting that the Hokuyo range info was only used for the projection operation - it was not used in the actual calibration, where it could constrain the external parameters of the target, and perhaps give a better calibration - though I'm not sure we can do much better than this.
Data
The tar file contains the camera images, scanner point clouds as ascii files, and scanner intensity images.