Machine Vision for the big drill

The University of Leoben in Austria has developed a robust image recording system to analyse the rock and monitor the construction progress as well as for geological documentation during tunnelling, using machine vision components from STEMMER IMAGING.

With drill head diameters of up to 15 metres and several metres in length, modern tunnel boring machines (TBM) are some of the largest tools in the world. These gigantic machines are used for excavating tunnels such as the Brenner Base Tunnel and help improve the infrastructure of railway and car traffic.

Reliable geological mapping plays an important role in tunnel engineering projects. It is used to identify the characteristics of the rock to be excavated in the tunnel cross-section as completely as possible. This way, information on work progress can be documented and, if necessary, reasons for possible delays and the resulting cost increase can be verified. For example, rock formations that are harder than expected can cause the advance rate to fall behind schedule and increase wear on the so-called disks, i.e. the actual excavation tools.

Difficult access

In traditional tunnelling, geological mapping is usually done by geologists who manually record their observations such as the soil and rock composition at regular intervals with hand sketches or pictures. Using TBMs, this cannot be carried out so easily because cutter heads have only a few openings to the working face, thus considerably limiting their visibility.

"The driving speeds of TBMs can be significantly higher than those of conventional tunnelling machines, but the necessary geological documentation of the working face clearly lags behind with these modern machines due to poor accessibility," explains Robert Wenighofer from the Chair of Subsurface Engineering at the University of Leoben in Austria.




In order to improve this situation, the University of Leoben and Geodata, a company based in Leoben and specialised in tunnel surveying, have developed a new approach: Instead of manual evaluation, which is difficult in TBMs due to lack of space, camera-based recording systems mounted in a drill head ensure the acquisition of image data which serve as a basis for geological mapping.

Of course, this is not possible in real drilling operations, says Wenighofer: "During drilling, neither the mechanical forces nor the dust development allow the recording of the necessary image material. However, the systems require regular maintenance and are repeatedly shut down, e.g. to extend the conveyor belts for removing the excavated rock. Pauses like these allow the installation of the system that captures the images of the working face.”

Up to three cameras

As a preparatory measure, the drill head is first moved back by a few centimetres and the excavated rock and dust are removed from the gap between the drill head and the working face to give a clear view to the cameras. Then, up to three cameras in robust protective housings are mounted at different positions and distances from the drill head's centre in the existing disc boxes and prepared for recording with the corresponding control unit. "The integration into disc boxes enables TBMs to be equipped with the camera system without the need for additional openings," emphasises Wenighofer.




For image acquisition, the cutting head is manually rotated – slightly more than 360° – without contact forces. The cameras used capture over 1,000 images, which are transferred to and stored in the control unit. Based on these images, the 3D reconstruction of the geological conditions found at this location is carried out later using stitching or photogrammetry methods. "Initially, we only captured the images from a single opening while the cutter head rotated," says Wenighofer, describing the first steps with this technology.




However, using this method, we could only record narrow, ring-shaped strips of the working face, which did not provide sufficient information on the overall condition and required the cameras to be repositioned at different distances from the centre. The new solution allows the recording system to be adapted to TBMs with large diameters. With a suitable design of the optical properties and the flexibility to retract the drill head, a three-camera system is usually sufficient to record the entire working face in one rotation, even for large drill head diameters.

Solution for a demanding environment

The environmental conditions in a tunnel construction site are incredibly harsh: temperatures of up to 40 °C, high humidity and a lot of dust impose major challenges for image acquisition. During the first attempts of putting together a suitable system, Wenighofer came across STEMMER IMAGING in 2014:r im Jahr 2014 auf STEMMER IMAGING:

The experts provided us with excellent support in selecting the right components and quickly gained an insight into the unfamiliar terrain of tunnel construction and its special circumstances.

Among other things, the partners were able to find a joint solution to the problem of high temperatures and humidity in the drill head after driving, which leads to condensation of the cooler camera unit and thus delays the recording. As a workaround, the camera system can now be powered even before installation, thus reaching a suitable operating temperature and adapting faster to the conditions in the cutter head.

The entire system consists of a flexible camera unit in a robust protective housing, incorporating an Allied Vision 2 megapixel industrial camera, a 5 mm fixed focal length lens and a customised LED ring for illumination.

The aperture is preset to a depth of field ranging from approximately 40 to 200 cm enabling the camera to detect even deep rock outcrops. The LED ring is designed with a luminous intensity of over 10,000 lumens. The strobe interval of 4 ms and the brightness ensure optimum image quality for the subsequent photogrammetric processing.

The strobe mode also avoids motion blur that could result from drill head rotation. The camera is set to continuous auto exposure, which updates the exposure time for each image. This prevents overexposure or underexposure of the working face sections, which is another crucial factor for reliable results using photogrammetric imaging.

A sophisticated system

The second part of the system is the control unit that remains in one position in a double-disk box or manhole for the recording period. It contains a PC, a power supply, a mono-axial tilt sensor and is equipped with interfaces allowing the parallel operation of multiple cameras.

To obtain sufficient overlap with the images, the control triggers the connected cameras at a constant rate of 2 images per second. Its configuration ensures the automatic start of image acquisition after booting the PC. There is no need for direct input into the system or any interaction with the camera control under the harsh conditions at the cutting head. As a special feature, the PC can be remotely controlled with an Android device such as a smartphone which allows quick evaluation of the image quality.



High resolution orthophotos are created using stitching methods, serving as a basis for photogrammetric evaluation.

The control unit's tilt sensor is responsible for creating a common coordinate system for all images recorded during a measurement run. Each image is defined by an angle value retrieved from the tilt sensor when a camera captures an image. Based on both the angle and the known relative position of the disk box, it is possible to determine the absolute camera position in the 3D reference coordinate system of the cutting head, which is essential for subsequent photogrammetric processing.



Colour-coded relief images show the position and depth of excavations after photogrammetric evaluation.

What is interesting for the evaluation, is that the system uses software for processing aerial image data. This enables matching the large number of images with the orientation data in order to calculate a point cloud of the entire working face. From this data, it is possible to develop a orthophoto and the interpolation of a digital elevation model, which can be used to realise the required geological mapping and evaluation.

A successful patent

The system for image acquisition and evaluation has proven itself in various tunnel construction projects and was successfully patented at the end of 2019.

STEMMER IMAGING has been an excellent partner on this journey and has provided us with decisive support in further developments such as the simultaneous control of multiple cameras via software triggers,says Wenighofer, clearly pleased with the progress made.

In tunnel construction projects, the required evaluations are thus easier and faster. For example, thanks to the large-area images of the working face, it is now possible to determine the position and the quantity of the excavated rock. The more we know about the surface roughness of the working face, the more precisely we can analyse the wear of the disks and asymmetrical loads on cutting heads in order to provide information on the service life and material fatigue of the cutting tools.

It is also very important that the digital working face recordings allow a more objective geological mapping, which is an essential basis of geological assessment for both the client and the contractor. The close cooperation with STEMMER IMAGING during the system’s development was extremely helpful for us.

www.stemmer-imaging.com

  Ask For More Information…