Originally, cameras capture a circular image

Light field technology in industrial image processing: New opportunities through special camera arrays

Stereoscopic vision is standard in industrial image processing - but that is not the only way to capture images. The light field technology offers a further approach which, thanks to extensive information, delivers a more precise result.


One or the other may still remember; From 2014 to 2016 a so-called "light field camera" could be acquired on the end customer market. The highlight: Recordings with this digital camera made it possible to subsequently set the focus point on photos in software. “Computational Photography” was born. The recorded scene was always absolutely sharp - and even a 3D simulation was possible. Unfortunately, the company behind it later decided to concentrate fully on the industrial application of VR. The product was pulped.

But the name of the camera said it was program: like all cameras in this class, it took that entire light field of a scene on. But what is hidden behind a light field? Why is it so special and why is it a real godsend for industrial image processing? We want to pursue these questions today.


Light field and plenoptic function

The light field is a physical concept and describes the Totality of all light-related information in a scene. This includes:

  • A light beam (straight line) from the recorded scene, determined by two angles θ and ζ
  • a place of observation of the light beam (Vx, Vy, Vz)

The variables of the function can be adjusted to the wavelength if necessary λ of light for color photographs as well as time t can be expanded for a video recording.

The actual one results from these individual aspects plenoptic function P.(θ, ζ, λ, t, Vx, Vy, Vz). All further calculations for the use of light field technology can be carried out on this basis.


Light field technology - or the attempt to capture all light information as an image

With the help of the plenoptic function, it is possible to determine exactly from which direction the light came in the photographed scene.

Recording a scene with a downstream camera array that generates numerous micro images. These are then digitally merged into one image.

The light field technology is used to calculate the camera position and thus also the associated focus point Camera arrays with multiple lenses. This grid breaks up the respective light rays again into a cone - modern algorithms are able to calculate an exact entry angle and thus the position of the originally recorded point. This puts the position and focus of the image into perspective. Appropriate software can now move both at will - the image is sharp everywhere and can be adjusted as desired. The viewing angle can also be varied afterwards.


What for comprehensive image information?

In industrial image processing, this information provides additional potential for difficult tasks. This is how light field cameras capture complex materials problem-free, even if this, for example shine or shimmer. In the case of a stereoscopic recording, the highlight would instead often be incorrectly interpreted as a surface point. This leads to errors in the depth estimation. Overall, the surface must be assumed to be Lambertian for stereo approaches - i.e. with absolutely matt reflection properties, which is never achieved in reality. The light field technology also has an advantageous effect in factories where no active lighting is desired, as the light field is very robust. With the camera arrays used, that's enough already regular ambient light for a good shot.

Also important: the light field can be used to calculate exact positions in the recordings. The step to detailed point cloud, to quantified surface properties or for 3D simulation is only very small. This in turn can then be used for various industrial applications such as Object recognition, bin picking or quality inspection - are used.