Home | Contact ST  

Feature Article

3D Mapping of the Seafloor in Color Using a Single Camera
Benthic Mapping Based on Video Recordings and Laser Profiling To Generate Colored 3D Reconstructions of the Seafloor

AUTHORS:
Adrian Bodenmann
Project Researcher
Blair Thornton
Research Associate
Tamaki Ura
Professor
Underwater Technology Research Center
Institute of Industrial Science
The University of Tokyo
Accurate, high-resolution maps of the seafloor are useful in many areas of underwater research, including benthic habitat mapping and mineral resource surveys. Furthermore, they can be used as a reference for illustrating sensor measurements, thereby visualizing the environment where data has been recorded to form a geographic information system. Bathymetry generation and photo mosaicking are now common mapping techniques that provide either shape or color information. 3D maps in color would enhance representation and have numerous advantages for seafloor classification.

One way to generate 3D color maps is by overlaying a photo mosaic on bathymetry data. As photo mosaics typically assume a flat terrain within each photo, pronounced relief may lead to geometric distortions due to perspective effects, producing artifacts in the mosaics that result in inherently inaccurate 3D reconstructions. Places of interest underwater often include areas with pronounced relief, such as areas rich in mineral resources, hydrothermal vents, man-made underwater installations, shipwrecks, coral reefs and archaeological sites. In such cases a method that handles the images in a fully 3D way—from input to output—is required to produce accurate results. Approaches respecting this requirement include stereo vision and structure from motion.

In this article, a new, simple method, first deployed in March, is introduced that uses laser profiling and video images of the seafloor to generate 3D maps by assigning RGB (red, green, blue) values on a pixel-based level. Since the proposed method uses laser profiling, the depth resolution is high and the algorithm is easy to implement because no feature matching needs to be performed. Simple software and hardware requirements make the system ideal for use even with small-size underwater vehicles, making it useful in a wide field of applications.


Data Acquisition
The proposed algorithm uses image and navigation data collected at low altitudes by an autonomous underwater vehicle (AUV) or a remotely operated vehicle (ROV). Seafloor topography is measured by laser profiling using a sheet laser mounted on the front of the vehicle; the sheet laser points downward to project a line on the seafloor that is perpendicular to the vehicle’s heading. A color camera is mounted a certain distance away from the sheet laser, usually at an angle off-vertical, so that its field of view extends to the projection of the sheet laser while also covering the area vertically beneath the camera. The laser line projection follows the shape of the terrain, which allows a bathymetric model to be generated based on triangulation after identification of the laser line in the images taken by the camera.

A light is used to illuminate only part of the camera’s field of view so that the seafloor directly underneath the camera is lit. A shade is used to ensure that the area surrounding the laser line projection remains dark, guaranteeing that the laser line projection appears with a strong contrast in the video, which makes its extraction more accurate. Color information is retrieved only from the area directly below the camera, allowing lighting to remain uniform in the final 3D map. Backscatter from floating particles is reduced by placing the light source so as to minimize overlap of its light cone with the camera’s field of view.

The baseline between the sheet laser and camera, its mounting angle, the height off the seafloor and the vehicle speed influence the resolution of the generated map, which is, in general, different in the vehicle’s longitudinal (x-axis), transverse (y-axis) and vertical (z-axis) directions. The baseline and the camera mounting angle can be changed depending on the mission and the physical constraints of the vehicle in order to optimize image quality and the overall resolution of the map.

The altitude of the vehicle off the seafloor is normally kept between one and three meters, depending on the nature of the terrain and the turbidity of the water; a short distance between the camera and the target is desirable to limit the effects of scattering and nonuniform absorption of different wavelengths of light.

The vehicle’s navigation sensors provide the necessary position and orientation information. For most vehicles, this consists of a Doppler velocity log (DVL), depth sensor and a three-axis magnetic compass, gyro and acceleration sensor or, if available, a fiber optic or ring laser gyro-based inertial navigation system (INS). These sensors provide the x, y, z coordinates and the roll, pitch and yaw orientations of the vehicle with reference to a global frame.

In a typical mission, the underwater vehicle cruises above the seafloor and scans it with the sheet laser, light and camera while also performing measurements in parallel with other payloads. As the vehicle advances, a point on the seafloor swept by the laser line is illuminated by the light after the vehicle has advanced a few centimeters. The 3D mapping algorithm uses the video frame showing the point being swept by the laser line to calculate its position in space. To retrieve color, the algorithm uses the navigation information to search for the video frame showing the same spot when the camera was vertically above it and computes the coordinates of the corresponding pixel.


Hardware
For projecting the line onto the seafloor, a green 532-nanometer 50-milliwatt sheet laser with an opening angle of 64° in water is used. A wide opening angle is desirable in order to maximize the swath of the system.

To achieve a high resolution in the forward-moving direction, it is important that images are recorded at a high rate. For this, video is more suitable than still photos. An NTSC-format video camera recording 29.97 frames per second is used due to its ease of implementation on existing vehicles. While higher resolution cameras are widely available, unless a faster frame rate can be achieved, the resolution in the direction of motion (x-axis), which is currently lower than the transverse resolution (y-axis), would not be improved.

To illuminate the seafloor, a light-emitting diode (LED) panel is used because it is both compact and energy efficient. A polyvinyl chloride board is used as shade to prevent casting light on the seafloor where the laser line is projected.


Algorithm
The algorithm requires a set of video images corrected for lens distortion, along with navigation data that gives the position and orientation of the vehicle when each frame was taken. Apart from that, the position and mounting angle of the video camera and sheet laser, as well as a parameter indicating the focal length of the camera lens, also need to be provided.

In the first step, the laser line is detected in each video frame. Together with the position and orientation data, this allows calculation of the bathymetry of the seafloor as a list of 3D points. In the next step, color is added to each bathymetry point. For each set of 3D points, the video frame taken when the camera was closest to being vertically above is determined using navigation data. In this frame, the area of seafloor mapped by these 3D points is illuminated by the LED panel, revealing its color. For each 3D point, the pixel that shows the same spot on the seafloor is determined, which allows the RGB values to be assigned.

After iterating through all 3D points, a list of georeferenced points in 3D and known color is obtained. In the final step, Delaunay triangulation is performed to transform these points into a mesh of polygons that, once stored in Polygon File Format, can be opened in a 3D model viewer.

The 3D map can then be viewed from any angle, and it is possible to zoom in to show details of the map, as well as directly measure dimensions within the map. Although precise navigation data is important for accurate reconstruction, since the time between the measurement of bathymetry and color is typically only a few seconds, sensor drift does not have any significant adverse affect on the color matching; a standard DVL and compass-based INS have been found sufficiently accurate to generate consistent 3D maps.


Deployment Results
So far the system has been deployed at sea a number of times using two different platforms, one being the 240-kilogram AUV Tuna-Sand, developed at the University of Tokyo’s Underwater Robotics and Application laboratory, and the other being the 3.8-ton work-class ROV Hyper-Dolphin from the Japan Agency for Marine-Earth Science and Technology (JAMSTEC).

In one example, data were recorded at depths of up to 3,000 meters using Hyper-Dolphin to survey manganese crusts at the #5 Takuyo Seamount during the NT10-11 cruise of JAMSTEC’s research vessel R/V Natsushima. The sheet laser, LED panel, camera and DVL were mounted on a jig attached to the ROV, which was operated about one meter above the seafloor at a speed of 30 centimeters per second, mostly in a straight line. With this setup, each video frame covered an area of approximately 1.17 by 0.94 meters. Video and DVL data were recorded together with acoustic sub-bottom measurements, and the surveyed area could be reconstructed in post-processing. The availability of both shape and color information makes it possible to distinguish between manganese crust, nodules and sand, which is otherwise difficult when purely based on either bathymetry or images. Dimensions of the crust patches, as well as of other topographic features, could be measured in the 3D map. It was also used as basis for visualizing acoustic crust thickness measurements, allowing even more accurate estimations of the actual deposit amount.


Conclusions
A new method of 3D seafloor mapping in actual color suitable for application on an AUV or ROV has been developed and implemented. Several deployments at sea have shown its usability and versatile applications. Since the required hardware is compact, it can be easily implemented on almost any AUV or ROV.

Though the method can be used for missions where mapping is the only aim, it serves as a more powerful tool when used together with data from other sensors, as it allows direct visualization and georeferencing of the terrain where the data was collected to assist its interpretation.


Acknowledgments
The authors would like to thank their colleagues—Mehul Sangekar, Takeshi Nakatani and Takashi Sakamaki—and JAMSTEC for support while collecting data and during data processing.


References
For a full list of references, contact Adrian Bodenmann at adrian@iis.u-tokyo.ac.jp.



Adrian Bodenmann is a project researcher at the Underwater Technology Research Center of the University of Tokyo. He received a M.Sc. in microengineering from the École Polytechnique Fédérale de Lausanne in Switzerland. Currently he works on the seafloor mapping project and the development of a deep-sea-going AUV.

Blair Thornton is a research associate at the Underwater Technology Research Center of the University of Tokyo. He received a Ph.D. in underwater robotics from the University of Southampton. His research interests involve the development of
in-situ sensors for integrated acoustic, visual and chemical marine surveys.

Tamaki Ura is the director and founder of the Underwater Technology Research Center and a professor at the University of Tokyo. He also founded the Underwater Robotics and Application laboratory and has developed various innovative ocean-going AUVs, which have performed successful dives to hydrothermal fields and gas hydrate fields.




-back to top-

-back to to Features Index-

Sea Technology is read worldwide in more than 110 countries by management, engineers, scientists and technical personnel working in industry, government and educational research institutions. Readers are involved with oceanographic research, fisheries management, offshore oil and gas exploration and production, undersea defense including antisubmarine warfare, ocean mining and commercial diving.