Aug 18, 2020
UAV DSLR photogrammetry with PPK processing
- 1Applied Remote Sensing Lab, Department of Geography, McGill University
- Canadian Airborne Biodiversity ObservatoryTech. support email: jocelyne.ayotte@umontreal.ca
Protocol Citation: Oliver Lucanus, Margaret Kalacska 2020. UAV DSLR photogrammetry with PPK processing. protocols.io https://dx.doi.org/10.17504/protocols.io.bjm2kk8e
License: This is an open access protocol distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Protocol status: Working
We use this protocol and it's working
Created: August 12, 2020
Last Modified: August 18, 2020
Protocol Integer ID: 40346
Abstract
The following protocol details the steps needed to acquire UAV based photographs with a DSLR and generate geotags through a PPK workflow. The protocol requires the following equipment:
- DJI Matrice 600 Pro
- Ronin MX gimbal
- Canon 5D Mark III
- Emlid RS+
- Emlid M+/M2 with LoRa adaptor
- Pocket Wizard Multimax II intervalometer
- MiFi portable Wifi
In addition, a subscription to an NTRIP service (e.g. Smartnet NA https://www.smartnetna.com/coverage_network.cfm) and a portable wifi hub with a cellular plan are required.
Mission Planning
Mission Planning
Steps for GS Pro 2.0 (v 2.0.12 at time of writing). This section is usually done days in advance of the actual flight, but can be carried out in the field as well.
On iPad launch GS PRO
Click on "My Missions"and then the large + icon to create a new pre-planned mission
Select "3D Map Area"
Select "Tap" and follow instructions on screen to draw a polygon encompassing the area to be mapped
Define and edit area by moving corner points
Name mission "Mission_name- horizontal" where Mission_name is specific to the area of interest, for example MerBleue-horizontal
In the camera model menu scroll down to "New Custom Camera". Enter a name for the camera (e.g. Canon 5D Mark III 24). Input sensor width and height (in mm), sensor resolution width and height (# of pixels), focal length (in mm), and minimum and maximum shutter intervals (in sec).
For the 5D Mark III with the 24 mm lens set the resolution to 5760 px (W) x 3840 px (H), 36 mm (W) x 24 mm (H) and both shutter intervals at 2 seconds. Do not turn on the 35 mm equivalent toggle. Click Update to save and ensure it is the camera model selected.
Select shooting angle "Course aligned"
Select capture mode "Equal distance interval"
Select flight course mode "Inside mode"
Flight speed and shutter interval are automatically set
Set flight altitude (in meters AGL). With the Canon 5D Mark III and a 24 mm lens, 50 m altitude (and following the camera settings and processing instructions in the sections below results in 1.3-1.6 cm GSD and a ground footprint size of 75 m x 50 m). This link provides a GSD calculator from Pix4D: https://support.pix4d.com/hc/en-us/articles/202560249-TOOLS-GSD-calculator
GSD Calculator from Pix4D
Click the "Advanced" toggle
Set front and side overlaps to 80-90% such a high overlap is recommended because the camera is not triggered by the UAV and therefore, the position of the photographs is not as precise
Select a course angle that is parallel to one of the axes of the rectangular polygon denoting the area to be mapped
Set gimbal pitch to -90 degrees
Set End of Mission Action to "Hover"
In the information window under the mission name, swipe left to the second screen and note flight time and battery requirements. For large areas more than one set of batteries may be required.
Click the computer disk icon on top left of screen to save the mission
Initial course heading parallel to the long axis of the rectangle
Click left arrow "<" in the upper let corner of the screen to return to list of missions
Find the mission that was just created "Mission_name- horizontal", swipe left and select "Copy". Highlight the copied mission and click Edit
Rename this copied missing as "Mission_name- vertical"
Under the Advanced toggle rotate the course angle by 90 degrees.
Course angle has been rotated by 90 degrees
Click the computer disk icon to save the mission
UAV Preparation
UAV Preparation
The following steps apply to most UAV systems, but this protocol is written specifically for the use of a DJI Matrice 600 Pro. Use of a D-RTK system, while optional, greatly improves the UAV's accuracy and precision in following the pre-planned mission.
Ensure UAV batteries (multiple sets if necessary), flight controller, gimbal battery (or multiple batteries as needed) and iPad are fully charged. Charge overnight if necessary.
Ensure aircraft payload rating is capable of lifting the gimbal and camera weight if using a system other than the M600 Pro with a Ronin MX gimbal and Canon 5D Mark III with 24 mm lens.
Check weather and any new airspace restrictions that may be in place.
Onsite, prepare and mark home point (takeoff/landing site)
OPTIONAL: Set up D-RTK ground station on a tripod with a clear view of the sky and no nearby tall objects. Turn it on and wait for the LED on
Insert batteries into M600 Pro, turn on flight controller (make sure iPad is connected via USB cable to the controller) and then turn on the the UAV
Open DJI GO. Tap the System Status bar and select Calibrate. Follow the on-screen instructions to
calibrate the compass. Then after successful compass calibration, turn the UAV off.
Mount the gimbal onto M600 Pro making sure to connect the CAN cable between the gimbal and the UAV and insert a gimbal battery.
Ground control point set up
Ground control point set up
While the use of GCPs and checkpoints is recommended whenever possible, there are circumstances where it may be logistically imporactical or unsafe to do so. See https://www.mdpi.com/2504-446X/4/2/13/htm for a discussion on the implications of not including GCPs.
Consult: https://www.pix4d.com/blog/why-ground-control-points-important for details about GCP placement and number of GCPs to set out. Even if GCPs will not be used, aim to set out targets if possible to use as checkpoints to verify the accuracy of the final products.
If using GCPs and/or checkpoints set out targets in area to be mapped. Their location can be measured now in the workflow or after Section 8. Follow these steps to measure their position. The same Emlid RS+ unit that is being used as the base station (section 4) can be used, if not other unit is available.
Turn on MiFi, wait for MiFi to initialize before proceeding
Mount RS+ to a survey pole (2 m or taller)
Turn on RS+ and wait for all three lights to come on in sequence -> orange, blue, green. The blue light will start flashing rapidly as it joins the MiFi network. Once it has joined the network it will continue to flash slowly. This is what we want – we do NOT want the base to make its own wifi network (solid blue light).
Connect cell phone to MiFi wifi network
Connect to RS+ via the Emlid Reach app on cell phone
Go to RTK tab and set mode to Static and GPS AR mode to Fix and Hold. Set GLONASS AR to ON. If it is not possible to get a good solution for the position by the end of this section, try setting the mode to Kinematic.
Leave default elevation (15 degrees) and SNR (35). These can be changed if it is not possible to get a good solution for the position by the end of this section
Set GNSS to GPS + GLONASS + SBAS @ 5 Hz
Go to Correction Input tab and ensure it is set to NTRIP and your credentials are entered. Click apply if you changed anything. Status should say connected. **This is why the base must be connected to the MiFi, rather than making its own network.
Go to Position Output tab and ensure it is turned OFF
Go to Base Mode tab and ensure it is turned OFF
Go to Bluetooth tab, ensure it is turned OFF
Go to the Status page – you should see mostly green bars for satellites in view with a corresponding grey bar for each (incoming NTRIP correction). If not, check the settings again.
Go to the Survey Tab and create a new project by clicking New project. Fill in Project name, Author and optionally, add comments.
Do not enable auto save rules, Click Done to processed to next step.
Press the "2 m" button (lower left of map) to change the default 2 m value to the height of the pole, press Save
Press the large + icon (lower right of screen) to move to next step
OPTIONAL: change default point name
Press Collect to start point collecting process. When a Fix is achieved, click Accept to save the location.
Move to next point and repeat 3.20. Continue until location of all points have been measured. Click the Back button to exist the collection window.
In the survey tab, under the project's name click the Export button and select Shp file. Email the downloaded file to yourself.
Turn off the RS+ via the software. The MiFi can be left on if proceeding to Section 4, otherwise turn off.
Emlid Base set up
Emlid Base set up
Mount Emlid RS+ on a tripod in a location with a clear view of the hemisphere (as much as possible). Tall trees or other objects nearby will interfere with reception. Electronic devices may also affect the reception of the GNSS signal. Keep all electronics (except the MiFi) as far as possible from RS+ during data collection.
Attach the LoRa antenna
Turn on MiFi, wait for MiFi to initialize before proceeding
Turn on RS+ and wait for all three lights to come on in sequence -> orange, blue, green. The blue light will start flashing rapidly as it joins the MiFi network. Once it has joined the network it will continue to flash slowly. This is what we want – we do NOT want the base to make its own wifi network (solid blue light).
Connect cell phone to MiFi wifi network
Connect to RS+ via the Emlid Reach app on cell phone
Go to RTK tab and set mode to Static and GPS AR mode to Fix and Hold. Set GLONASS AR to ON
Leave default elevation (15 degrees) and SNR (35). If at the end of this section you notice very low SNR because of your location you may need to reduce the SNR or elevation mask
Set GNSS to GPS + GLONASS + SBAS @ 5 Hz
Go to Correction Input tab and ensure it is set to NTRIP and your credentials are entered. Click apply if you changed anything. Status should say connected. **This is why the base must be connected to the MiFi, rather than making its own network.
Go to Position Output tab and ensure it is turned OFF
Go to Base Mode tab and ensure it is turned ON and set to LoRa. Set the base coordinates mode to Average Fix. Choose a reasonable time period ~15min should be sufficient. The longer you set this time the longer you need to wait before proceeding to the next section (M+/M2 set up).
Set output power to 20 dB and air rate to 9.11 kb/s (recommended). Select a frequency (e.g. 901 MHz) Record these values because the M+/M2 must be set to listen on these same settings.
Set RTCM messages as follows:
a. 1002 @ 1Hz
b. 1006 @ 0.1 Hz
c. 1010 @ 1 Hz
You can optionally turn on 1107 @ 1Hz and/or 1097 @ 1 Hz but it is usually not needed. RTK will not function if 1002 and 1006 are not turned on.
Go to Logging tab and turn all three logs ON. Failure to do so will result in having to collect the data over again. Check the logging period – by default new logs are started every 4 hrs.
Go to Bluetooth tab, ensure it is turned off
Go to the Status page – you should see mostly green bars for satellites in view with a corresponding grey bar for each (incoming NTRIP correction). If not, check the settings again.
Exit the Emlid Reach app on the phone and disconnect from the MiFi
Place MiFi on the ground nearby with screen facing DOWN, is a shaded location where it will not overheat.
Emlid M+/M2 set up
Emlid M+/M2 set up
Ensure the M+/M2 will be powered independently from the gimbal during set up. Turning the gimbal on/off without properly shutting down the M2/M+ corrupts the settings and can corrupt its software if it loses power during its boot up/loading stage (i.e. before green light is solid). **Only after the camera is balanced and facing nadir should the USB cable be connected to the gimbal for power during flight (see steps below). The M600P should be outside with a clear view of the GPS antenna to the sky for this part.
Connect antenna cable to M+/M2
Connect LoRa transmitter and the M+/M2 is connected via the S2 port
Turn on the M+/M2 by plugging in the USB cable (powered by an external USB bank)
Orange, blue and green lights should come on in sequence. Solid blue means M+/M2 has made its own wifi network. It is important the unit creates its own wifi network rather than connecting to a pre-existing one.
Wait for a solid green light – this loads the software, does time sync, etc. Do not proceed until there is a solid green light on the unit.
Connect to its wifi network using phone with the Emlid Reach app installed. Default password is emlidreach DO NOT change this password
Go to the Correction Input tab and make sure the frequency, air rate and power settings match what was configured on the base unit
Go to RTK tab, make sure mode is set to Kinematic. Leave SNR mask (35) and elevation mask (15 degrees) as default.
For the M2 enable all GNSS at 5 Hz.
For the M+ enable ONLY GPS + GALILEO at 5 Hz
Go to Base mode tab, ensure it is OFF
Go to Position Output tab, ensure it is OFF
Go to Logging, ensure all three logs are turned ON. Failure to do so will mean having to do the data collection over.
Go to Status, you should see green and orange bars with grey ones next to them (incoming LoRa correction)
Turn off M2/M+ via the software
Camera and intervalometer set up
Camera and intervalometer set up
Ensure Canon 5D Mark III camera has a full battery
Ensure camera has a formatted SD card. Use minimum 1000x speed card, 2000x is better (minimum 32 GB). Slower cards will not work.
Remove lens cap and any filter that may have been installed on the lens. Set lens to 24 mm.
Turn camera on and Set to P mode
Set camera to acquire Raw only (not Raw + jpg), then turn the camera off.
Connect 3-pin Pocket Wizard remote control cable to camera
Mount camera on Ronin-MX gimbal
Connect other end of the remote control cable to the P2 port on the Multimax II intervalometer. DO NOT turn on the intervalometer yet
Balance the camera in a horizontal position (ensure it is balanced by turning on the gimbal and adjusting the motor stiffness as necessary). Then turn off the gimbal.
Turn on the M600P, then turn on the gimbal and using the M600P’s remote control, turn the camera to face nadir. Leave the gimbal on from this step onwards.
Move the M600P to its takeoff location (should be turned off if it needs to be moved but the gimbal can remain on). GPS antennas need a clear view of the sky for all subsequent steps.
Connect the USB cable from the M+/M2 to the gimbal’s USB port. As long as the gimbal is on, this step will turn the M+/M2 back on.
Connect the Emlid hotshoe adapter to the camera – pins down. The other end should be plugged into the C1 port of the M2/M+
Connect phone to the M2/M+ via its wifi (all three lights should be on and solid by now, if not, wait until they are to proceed)
Check that there are both green and/or orange satellite signal bars AND grey ones (incoming correction) on the status page of the Emlid Reach app. If not follow steps above to check the settings.
Go to the Logging tab and make sure all logs are ON **If the logs are not on data collection will need to be repeated**
Exit the Emlid app on the phone
Turn on the camera
Turn on the intervalometer (Tx mode)
Press TEST on the intervalometer to start triggering the camera. If it does not trigger the camera follow steps below to make sure it is set to infinite intervalometer mode with a 2 sec delay.
OPTIONAL STEP:
Only follow this step if Multimax II is not set up correctly to trigger the camera.
*/MENU A B Activates camera shutter release at set interval for set number of activations. Set interval (amount of time between activations) in Hours:Minutes:Seconds to 2 sec, then set count to 0 for infinite count.
Photograph acquisition
Photograph acquisition
If it is not already on, turn on flight controllef (ensure iPad is connected), then turn on the UAV
Launch the DJI GO app, wait for system to initialize, ensure compass is callibrated and top status bar is green "Ready to Go"
Click battery symbol, check each battery for equal cell voltage(within 2% capacity) and that all batteries are equally full. DO NOT continue if the batteries are not equally full.
Launch GS Pro, enter list of saved missions and select ""Mission_name- horizontal"
Click blue aircraft button on the top right corner of the screen
If pre-flight checks pass click "Start to fly". At this point the waypoints will be uploaded. Once that is complete the UAV will start the mapping mission. Monitor its progress and battery status during flight.
Once the mission is complete, take over manual control and land safely.
OPTIONAL: If the area to mapped is large and one battery set is not enough to complete the entire Mission_name- horizontal. Click the "Pause" button in GSPro during flight when the batteries are low (~30% capacity left). Take over manual control and land. Upon landing when it is safe to approach the UAV, hold the * key on the intervalometer to stop triggering the camera. Do not turn off the gimbal. Turn off the UAV and change the batteries. Turn the UAV back on. Press the TEST button on the intervalometer to resume triggering the camera. Follow steps 6.2, 6.3 and 6.4. Click the "Resume" button in GS Pro and finish acquiring the photographs.
When it is safe to approach the UAV hold the * key on the intervalometer to stop triggering the camera. Do not turn off the gimbal. Turn off the UAV and change the batteries. Turn the UAV back on. Press the TEST button on the intervalometer to resume triggering the camera. Follow steps 6.2, 6.3 and 6.4, except this time select "Mission_name- horizontal" and follow steps 6.5, 6.6 and 6.7.
Data retrieval
Data retrieval
Follow these steps after landing – BEFORE turning anything off
Hold the * key on the Multimax II to cancel intervalometer
Turn off the Multimax II and the camera
Log into the M2/M+ via phone and the Emlid Reach app (connect to its wifi first). Go to the Logging tab and turn all logs off
Wait for logs to finish writing
Export each log one at a time and email them to yourself
Turn M2/M+ off via the app, then turn the gimbal and M600P off
Go to the RS+ base unit, connect to the MiFi and then log into the unit via the Emlid Reach app
Go to the Logging tab and turn all logs off. Wait for all the logs to finish processing, if the unit has been running for a long time it may take a few second to finish writing the logs.
Export all three logs and email them to yourself. Then turn RS+ off via the app and then turn MiFi off.
Remove SD card from camera and download photos. Make sure to download and keep ALL photos, even the ones of the ground before takeoff and after landing otherwise matching the event tags to the correct photo in post processing will be very difficult and inaccurate.
If not already done, measure the location of the GCPs and/or checkpoints as described in Section 3.
Geotag generation
Geotag generation
These steps are nomally carried out in the laboratory, but could also be done in the field if there is access to a laptop with sufficient processing power. Check current hardware recommendations: https://support.pix4d.com/hc/en-us/articles/202557289-System-requirements-Minimum-and-recommended-computer-specifications
Software required:
• Adobe Lightroom (or other utility to convert .CR2 to large JPG with minimal compression)
• RTKLib (most recent version downloaded from the Emlid website)
• Pix4D Mapper
Optional software for troubleshooting:
• Exiftool
Additional data required:
• Commercial and/or NRCAN Active Control System base station data from a station near study site. Data are needed for the entire period of the data acquisition. Download in Rinex 3 format.
• Broadcast Ephemerides and Clock Corrections (from NRCAN ACS)
Convert ALL Canon .CR2 photographs to large JPGS in Adobe Lightroom. Ensure all metadata such as camera and lens info are retained.
Organize Emlid base station (RS+) and rover (M2/M+) files into separate folders
Organize the Commercial/NRCAN base station Rinex files, Broadcast Ephemerides and Clock Corrections files in a separate folder
Unzip the Commercial or NRCAN ACS base station data into a separate folder. When using SmartNet NA data each hour is provided as a separate zip file, unzip all files from all hours into the same folder.
Base station files from Smartnet NA should be unzipped to a single folder
Launch rtkconv_qt.exe and select the Emlid base .UBX file for conversion
In the Options menu turn off the frequencies and GNSS systems not used
The RS+ is a single frequency (L1) system. Click Convert
Repeat the conversion process for the Emlid rover UBX file. Note the M+ is a single frequency (L1) system but the M2 is a dual frequency (L1 and L2) system.
Launch rtkpost_qt.exe. Select the files as shown below:
Note the input (i.e. Rover OBS) file is the output of the Emlid base UBX conversion. The Base Station file is the ‘o’ file from the unzipped SmartNetNA Rinex files. Replace the hour letter designation with a *. Repeat for the ‘n’ and ‘g’ files. Also select the clock and broadcast ephemerides files.
In the options tab, adjust the various options as shown below:
When processing the Emlid base station positio, the RTKLib Base Station should be set to RINEX Header Position if using Rinex files from NRCAN or Smartnet NA
Click Execute and wait for processing to finish
Click Plot and a window with the processed positions of the base station opens:
Display the .pos file
The large majority of positions should be quality level 1 (green) with a small spread.
Open the .pos file in Excel (space delimited)
Sort based on the “Q” quality field and delete all non Q=1 rows
Calculate the average lat, lon and height
Save/write down the average base coordinates (keep all decimal places)
In rtkpost_qt.exe set up the files as shown below this time to process the Emlid M+/M2 files (extracted from the UBX file)
Note the clock and broadcast ephemerides files remain the same, but the Rover OBS file is that of the Emlid M+/M2 and the Base Station files (.obs and .nav) are the one created in the previous step.
Click on Options and set the options as shown below:
Note positioning mode is kinematic for the +/M2
Enter the average Emlid base position calculated in Excel
Click Execute and wait for processing to finish
When processing is finished, click on Plot. The flight path of the UAV will be shown as logged by the M+/M2.
In the Plot window click on File-> Open Solution 2 and select the “events” file
In the Plot window turn off Solution 1, keeping only Solution 2 which show the geotags for each photograph
Most of the positions should be quality level 1
Open the events.pos file in a text file viewer such as Notepad++, each row is the position information for a photograph. When using SmartNet NA as the original base station and as NTRIP provider during acquisition the coordinates are in NAD83CSRS datum and CGVD2013 ellipsoidal height
Verify that the number of geotags in the events.pos file matches the number of photographs
OPTIONAL STEP: If the number of geotags do not match the number of photographs. Extract photograph exif information with Exiftool to try to figure out which photographs or event tags don't belong (or do not have a match).
Open a command line window.
Navigate to the folder that contains the Exiftool executable.
The following is an example:
exiftool -csv D:\MB\2017\geotest\June3\X5S > D:\MB\2017\geotest\June3\X5S\exifinfoX5.csv
The path is the directory where the jpgs are located.
Delete photographs at the beginning and end of the sequence that were captured on the ground. Delete the corresponding geotags from the events.pos file. Delete the header info and all columns except lat, lon and height. Add a column called “Station” and number consecutively the rows starting from 1. Columns 2-4 should have the following headers: latitude longitude height. Subtract the vertical displacement between the GPS antenna and the camera film plane (~50 cm on the M600P). Save as a .csv file.
Open GPS-H in a web browser (https://webapp.geod.nrcan.gc.ca/geod/tools-outils/gpsh.php )and set up batch processing mode:
Set the Epoch based on the province the data were collected in. If using Smartnet NA consult https://support.smartnetna.com/hc/en-us/articles/228080208-SmartNet-Coordinate-Information for broadcast epoch. Note this may differ from "Provincially adopted Epochs"
Select the .csv file created above and click Send File
GPS will return the processed file with the following columns:
The Ho column represents orthometric height
Create a .csv file with the following columns from the GPSH output: Station, latitude, longitude, Ho. Rename the Ho column to height.
On the TRX webpage (https://webapp.geod.nrcan.gc.ca/geod/tools-outils/trx.php ) set up Batch processing mode as:
Make sure to select the correct UTM zone. and Epoch. If using Smartnet NA consult https://support.smartnetna.com/hc/en-us/articles/228080208-SmartNet-Coordinate-Information for broadcast epoch. Note this may differ from "Provincially adopted Epochs"
The output from TRX will look like:
Retain the easting, northing and height columns for the generation of the Geotag file for Pix4D (saved as .csv):
Continue with stardard Pix4D workflow.