For several years now, spruce bark beetles have been causing high levels of damage to timber in Bavarian forests. In order to help identify and contain the spread of bark beetle infestations as early as possible, various research approaches are being pursued, including remote sensing techniques for damage detection. The monitoring systems established in this context aim to detect the infestation as early as possible. Ideally, a largely automatic evaluation procedure would be available that would quickly and reliably provide users with coordinates showing the respective location of infested spruce trees. For this, it is first necessary to analyse the performance of various satellite data.
This is the background against which the LWF launched the IpsSAT research project in 2019. The project uses innovative satellite remote sensing techniques to test whether it will be possible in future to use the data for computer-assisted, automatic detection and monitoring of damaged spruce trees over a large area.
Bark beetle infestation from the remote sensing perspective
The literature on the detection of bark beetle damage by remote sensing often distinguishes between three stages of infestation:
- The green-attack stage: A spruce is infested, but its crown is still green.
- The red-attack stage: The crown of the spruce is turning reddish brown.
- The grey-attack stage: The spruce has lost most of its needles and the crown is now grey.
Although possibilities for the detection of bark beetle infestation at the green-attack stage or for early detection of bark beetle infestation on spruce with remote sensing technology have been studied for many years, to the authors’ knowledge there is no practicable procedure available so far. The evaluations of IpsSAT thus concentrate on the recording of spruce trees that have already turned reddish-brown and grey, i.e. on the recognition of infestation at the red- and grey-attack stage.
Even if early detection is not possible with this, it should still be possible with this application to get an overview of the extent and distribution of damaged areas. In addition, it should also help to localise infestation sites where the damaged trees have not yet been processed and removed, where there may be freshly infested spruce trees close by with crowns that are still green.
Fig. 1: Spruce stand severely damaged by bark beetle infestation in the Frankenwald forest. Photo: Eike Reinosch (LWF)
Varying intensity of damage
Figure 2 shows the geographical location of the IpsSAT study areas. In 2019, the investigations were initially carried out on the “Bad Kohlgrub” (30 km²) and “Bad Kötzting” (134 km²) areas. At the end of 2020, it was decided to add a section of the Frankenwald Forest (160 km²) as an additional area, as massive bark beetle infestations had developed in the spruce forests there. The spruce stands in this area were those most severely damaged of all the selected study areas.
Fig. 2: Geographical location of the selected IpsSAT study areas; in the background a tree species group map is visible, derived by the LWF on the basis of Sentinel-2 satellite data from the European earth observation programme Copernicus.
Focus on optical satellite sensors
The analysed optical satellite systems record the solar radiation reflected from the Earth's surface in different spectral ranges and with different spatial resolutions. To assess the suitability of the satellite systems in IpsSAT, a comparison was made with aerial data from an aircraft. The aerial photographs were taken with special digital measuring cameras with high spatial resolution. The following data sets were tested for damage assessment:
- aerial images (4 spectral bands with 0.2 m ground resolution)
- WorldView-3 (8 spectral bands with 1.2 m ground resolution and one panchromatic band with 0.3 m ground resolution)
- WorldView-2 (8 spectral bands with 1.2 m ground resolution and one panchromatic band with 0.5 m ground resolution)
- SkySat (4 spectral bands with 1.1 m ground resolution and one panchromatic band with 0.8 m ground resolution)
- PlanetScope (4 spectral bands with 3 m ground resolution)
- Sentinel-2 (4 spectral bands with 10 m and 6 spectral bands with 20 m ground resolution)
The spatial resolution of remote sensing data refers to the ground area that a pixel maps on the Earth's surface. It significantly influences the possibilities of detecting and recognising objects in the image, e.g. whether individual tree crowns can be reliably differentiated or not. The highest spatial resolution is provided by aerial image data recorded from an aircraft, with a ground pixel size of 0.2 m × 0.2 m. The images with the lowest resolution are the free Sentinel-2 satellite image data from the European Earth observation programme Copernicus. Depending on the spectral band, Sentinel-2 has a resolution of 10 m × 10 m or 20 m × 20 m. The spectral resolution, i.e. the number of spectral ranges that can be captured with one sensor, also varies. The aerial images available in IpsSAT, SkySAT and PlanetScope data have four spectral bands and can thus cover the visible as well as the near infrared range. In the case of Sentinel-2 there are even ten bands available, as the short-wave infrared is also recorded here, in addition to the visible and near infrared range.
Automated damage detection with machine learning
Various methods from the field of machine learning are being tested in IpsSAT for automatic damage detection. Currently, two classical methods are being used for this at the LWF: Random Forest and Support Vector Machine. Both approaches can be used to classify image content into different categories or thematic classes. In the present case, the forest areas recorded via remote sensing are to be classified into reddish-brown or grey spruce, vital trees, and if necessary other classes, such as ground areas. For the prediction of these target classes based on the image information to be as precise as possible, the machine learning methods must first be trained with manually selected image sections - so-called training data. For this purpose, the algorithm is given examples that are as representative as possible for each class. For the adaptation of the learning algorithm, 70% of this training data was randomly selected, while the remaining 30% served as test data to check the accuracy.
Fig. 3: Top row: Section of the Bad Kötzting study area for several remote sensing data sets, in colour infrared representation; bottom row: associated classification result, modelled with machine learning (here using the Random Forest method).
Figure 3 shows a small section of the Bad Kötzting study area with a size of approximately 150 m × 150 m for several remote sensing data sets tested in IpsSAT. For each data set, a colour-infrared representation is shown together with the corresponding result of the automatic damage classification. It is striking how strongly the spatial resolution of the sensor used influences the detection of individual tree crowns. It is also clear that both the WorldView-3 and SkySAT images were recorded with an oblique view of the sensor. The oblique photographs result in visible tilting of the tree crowns compared to the aerial photograph (Figure 4).
Fig. 4: Tilting and positional offset of trees due to oblique images in World-View-3 and SkySAT satellite images. The positional reference crown on the aerial image is circled in orange, and the projection on the satellite images is circled in green.
In principle, the damaged forest areas with the discoloured tree crowns could be clearly delineated in the remote sensing data (Figure 3). Since individual tree crowns can no longer be visually differentiated in the images from the PlanetScope and Sentinel-2 satellite systems, it was also not possible to divide them into the red-attack and grey-attack stages with this image material. With these data, the evaluation was limited to a general distinction between damaged spruce and vital tree crowns.
As a statistical measure for assessing the agreement between the manually and automatically created classes, the so-called kappa coefficient according to Cohen was calculated. The higher the kappa value, the more accurate the automated image evaluation is. A maximum value of 1 would indicate a perfect match with the manually created reference data. The highest kappa values for the three classes (red-attack, grey-attack and other) could be achieved with the aerial photo dataset - with very good agreement values of 0.89 to 0.97. For the high-resolution satellite data, the kappa values decrease depending on the spatial resolution: 0.71 to 0.89 for WorldView-2 and WorldView-3, and 0.79 to 0.87 for SkySAT. For the medium-resolved sensors, the separation of two classes (damaged spruce and other) was examined. The kappa values were in the range of 0.76 to 0.97 for PlanetScope, and 0.90 to 0.91 for Sentinel-2 - which is considered as acceptable to high agreement here.
A closer examination showed that reddish-brown and grey spruce trees can indeed only be distinguished in aerial photographs. As Figure 5 illustrates, there is an increasing spectral mixing between these two damage categories with all other sensors. Shown are distributions of spectral values for assessing the separability of the red-attack and grey-attack classes for the respective best explanatory variable per sensor. The more the spectral value ranges for red-attack and grey-attack overlap in the representations, the greater the spectral mixing of the two classes. This makes the possibilities of a visual and also an automatic differentiation increasingly difficult.
Fig. 5: Density plots with the distribution of spectral values to assess the separability of the red-attack and grey-attack classes for aerial image data, WorldView-3, SkySAT and PlanetScope using the Bad-Kötzting study area as an example.
Current findings, further research
The accuracy of the detection and delineation of damaged spruce in the remote sensing data depends on several factors. Technical aspects such as the spatial and spectral resolution of the remote sensing sensors are of particular importance.
According to current IpsSAT findings, aerial photography from an aircraft is best suited for manual and also for automated damage assessment. So far, a differentiation of the red-attack and grey-attack infestation stages is only possible in the aerial photographs. With the tested satellite data, it has not been possible to achieve a reliable separation of these damage categories so far. The image data from the satellite can however be used to obtain a general overview of damaged forest areas and thus to speed up the process of identifying focal points for bark beetle control measures. The key advantage of satellite technology in comparison with the use of aircraft is that larger areas can be imaged within a shorter period of time. A prerequisite for the use of optical systems is that no clouds impair the view of the earth's surface. In addition, a new fly-over or higher repetition rate is easier to realise with the satellites than with the aircraft. The Sentinel-2 satellites can for example provide up-to-date images every five days in cloud-free conditions. Because of their good availability, the Sentinel-2 data are used at the LWF to record large-scale changes in the forest. If such analyses were made available on a regular basis, practitioners on the ground and also decision-makers could be given early warning of coming need for action.
Further research in the IpsSAT project is focused on optimising the approaches taken so far. The project partner IAB GmbH is currently testing modern AI procedures and deep learning methods for automated damage detection. Deep learning is a sub-field of machine learning. It is based on deep learning artificial neural networks that are trained with large learning data sets. It remains to be seen whether the detection of damaged spruce trees can be improved with these newer evaluation methods.
Summary
The main objective of the IpsSAT research project is to examine possibilities for automated detection of bark beetle damage using optical satellite data and aerial photographs taken from aircraft. Various methods from the field of machine learning are used for this purpose. Current evaluations by IpsSAT show that aerial photographs from aircraft are best suited for a differentiated damage assessment. The satellite data, on the other hand, provide a general overview of damaged areas in the forest - a differentiation of different damage categories has not been feasible so far. In the further course of the project, the aim is to improve damage detection with modern AI procedures and deep learning methods.
The IpsSAT project is being funded by the Bavarian State Ministry of Food, Agriculture and Forestry (Running time: 01.09.2019 - 30.09.2023) and carried out in cooperation with the company IAB GmbH.
