Sep 05, 2021

Public workspaceStructure-from-motion multi-view photogrammetry applied to linear-scan sediment cores images V.2

This protocol is a draft, published without a DOI.
  • 1Université Savoie Mont Blanc;
  • 2Université Grenoble Alpes
Kevin Jacq
Icon indicating open access to content
QR code linking to this content
Protocol Citation: Kevin Jacq, William Rapuc, Claire Blanchet, Estelle Ployon, Cecile Pignol, Didier Coquin, Bernard Fanget 2021. Structure-from-motion multi-view photogrammetry applied to linear-scan sediment cores images. protocols.io https://protocols.io/view/structure-from-motion-multi-view-photogrammetry-ap-bxzfpp3nVersion created by Kevin Jacq
Manuscript citation:
Jacq, K.; Ployon, E.; Rapuc, W.; Blanchet, C.; Pignol, C.; Coquin, D.; Fanget, B. Structure-from-motion, multi-view stereo photogrammetry applied to line-scan sediment core images. J. Paleolimnol. 2021, 66, 249–260, doi:10.1007/s10933-021-00204-x.
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License,  which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it’s working
Created: September 05, 2021
Last Modified: September 05, 2021
Protocol Integer ID: 52999
Keywords: Structure from motion, Photogrammetry, Sediment core, Automatic image assembly, Orthorectification, Agisoft Metashape, Agisoft Photoscan,
Abstract
Image acquisition is the first step to save primary color information for environmental samples before disturbing them with other, mainly destructive analyses. The improvement of RGB cameras and image processing algorithms allows to obtain a metrically calibrated image at high resolution called ortho-image. The way to obtain this image requires the processing of several raw images acquired along the sample. We propose a semi-automatic method that uses metrically calibrated targets to create the ortho-image with Agisoft Photoscan or Metashape software.
HAL Id : hal-03257581, version 1
DOI : 10.1007/s10933-021-00204-x
General description of the acquisition bench
General description of the acquisition bench
Schematic description of the sediment core image acquisition tray by linear scanning

Image obtained with the acquisition bench. Sedimentary core surrounded by two rulers and markers

Camera acquisition parameters and calibration
Camera acquisition parameters and calibration

Safety information
It is important that the parameters related to the camera (shutter speed, aperture, etc.) are fixed throughout the acquisition. The autofocus must also be disabled.

With the chosen acquisition parameters, the camera must be calibrated in two steps:
The first is to perform a white balance, in order to obtain relevant colors according to the lighting, this is called radiometric correction.
The second consists in taking pictures of a checkerboard from several angles and over the entire surface acquired by the camera. This allows to correct the images for geometric distortions induced by the lens that imply that the pixels are not square.

Images of a checkerboard from several angles to estimate and correct geometric distortions
Schematic representation of geometric distortions
The Agisoft Lens software uses these checkerboard images to estimate the corrections to be applied to the images.

Importing images
Estimation of distortions and corrections
Estimation of distortions and corrections
Example of estimated parameters

Recommendations on coded markers and their georeferencing
Recommendations on coded markers and their georeferencing
The method described in this protocol relies on markers placed on rulers along the sample to create a planar spatial reference (z=0).
The Agisoft Photoscan or Metashape software proposes a list of markers all different which will be automatically recognized by the software.
These markers must be placed in a regular way so as to be present on three successive images to allow a good registration of the images and at a size not exceeding 30 pixels for the central circle. In our case, the markers are 10 cm apart and have a diameter of 1 cm.

Safety information
Markers should be printed on a material that does not get damaged and remains clean from sediment and water. They should also not be plastified as this will reflect light, and the algorithm will be distorted. For these reasons, we decided to use a mat adhesive paper.

Safety information
These targets should be cut in squares and not in circles, keeping in mind that the targets are circles on a white background.

Safety information
If a marker is damaged, it will probably be necessary to replace it with a new one that must be different from the others.

Each marker, defined by the software, is numbered and must be georeferenced to constrain the models with this landmark created with the rules and markers.
The georeferencing is performed by measuring the position of the marker's center relative to a point defined as the origin. In our case, the origin was chosen as one of the corners of the sedimentary core frame by the rulers.
The marker numbers and their positions are recorded in a csv file which will be used by the software.
Example of markers

Image acquisition
Image acquisition
The acquisition of the images must be carried out to have sufficient overlap between the successive images.
In our case, we have chosen an overlap of 75% so that each point of the sample is present on 4 images.
Creation of the orthoimage
Creation of the orthoimage
1) Importing images
Select the images with the "Add Photos" icon in the Workspace tab or in Workflow > Add Photos...
2) Corrections with camera settings
Now that the images have been loaded, they need to be corrected with the camera settings. You have to import them by going to Tools > Camera Calibration...

Loading camera distortion correction parameters
Then load the parameters with the "load" icon. Set the calibration to be applied to all images by checking the box.

Loading camera distortion correction parameters

3) Marker detection
Two methods can be used to detect and georeference the markers. One automatic and one semi-automatic.
Automatic method:
A Python script has been developed to detect the markers and retrieve their positions in the csv file previously created in step 6.It can be found on github.

Safety information
It is possible that this script must be corrected, because in the case of a change of version of the Agisoft software, the command tags may be different.

Load the script that will automatically detect the markers and associate them with their coordinates (x, y, z). To do this go to Tools > Run Script...

Opening the Python script
Opening the Python script

Semi-automatic method:
Markers can be detected in Tools > Markers > Detect markers...

Marker detection
Then define the markers to be detected according to those that have been printed and placed on the rulers.

Markers to be detected
After the markers have been detected, it is necessary to manually enter their positions in x,y, and z in meters.

Manual georefencement of each marker with z=0

Now that everything has been calibrated and the markers defined, it is necessary to follow the order of the Workflow for the creation of the orthoimage.

Workflow for the creation of the orthoimage
The different stages of the model are visible in the Workspace tab (bottom left) by unrolling Chunk:

Each step of the modeling

4) Registration and assembly
The first step is the alignment of the images which estimates the analog pixels between them, the positions of the cameras and creates tie points that correspond to the connection points between the images.
Go to Workflow > Align Photos... then select settings.

First step of the alignment
This step is the most important. It is important to check that, following these calculations, the reconstruction is horizontal. If a curved shape appear, it is a well-known “dome or arc effect” due to the miss-estimations of pixel positions in x, y, and z typical of linear image acquisition.
To do this, use the radius of the sphere to rotate the model, by staying pressed with the left click, you can rotate it. There may be some points that are not correct, they will not prevent the continuation of the steps. The main thing is that the model is horizontal.
Correct reconstruction
Poor reconstruction
The reconstruction error of the marker positions is also estimated, allowing to estimate areas where new control points have to be added manually to optimize the model. To add control points, select locations on images with right click > Create Marker and right click > Place Marker for the following images to associate a marker to multiple images. It may be necessary to de-align the images, before aligning them with the new control points.

Reconstruction errors

The second step is the dense cloud that estimates a reconstruction of the depth of each photo and determines its coordinates with the multiview stereo correspondence and the previous estimate of the image geometry.
To calculate the dense cloud, go to Workflow > Build Dense Cloud.

Dense cloud step

5) Mesh and orthorectification:

The algorithm then uses the points of the dense cloud to generate a closed surface with triangular surfaces, this step is called meshing.
It can be made with Workflow > Build Mesh...

Mesh step
Look at the result of the 3D model. In the case of a very dark image it means that the model is upside down, otherwise the model is in the right direction. This is to be taken into account in the next step.
Model in the right direction
Model in the wrong direction

Finally, the orthorectified image is estimated from the projection of the colored mesh in a 2D space to correct perspective image and projection distortions.
To perform this step, go to Workflow > Build Orthomosaic...
If the model is in the right direction then select the default settings, otherwise change Top XY to Bottom XY.

Orthoimage creation step
Othoimage according to the direction of modelling
Finally it is possible to export the orthophoto with File > Export Orthomosaic > Export JPEG/TIFF/png...