Using sensor technology for tree recognition

Since the launch of the Climate Adaptation Knowledge Programme by the Directorate-General of Public Works and Water Management in May 2023, significant progress has been made. The Smart Sensor Systems research group at The Hague University of Applied Sciences, together with the Hanze University of Applied Sciences Groningen, has developed groundbreaking technologies to monitor biodiversity. “The Directorate-General of Public Works and Water Management now wants to gain more knowledge about the trees along the main roads/waterways, in order to keep them safe”, explains John Bolte, project leader and professor of Smart Sensor Systems.

The focus of the Directorate-General of Public Works and Water Management has shifted from monitoring plants to trees, also known as standing timber. “We will measure where trees are within four meters of the motorway. “How tall they are, what species they are and what their health status is”, adds Bolte. 

Safety risks

For safety reasons, trees may not be taller than 2.5 metres and four centimetres in diameter. If young trees in the obstacle-free zone are not felled in time, they must be replanted elsewhere due to the replanting obligation under the Environment and Planning Act [Omgevingswet]. In addition, safety risks increase when trees are sick or weakened. Heavy rainfall can cause roots to rot and dry out in extreme drought, resulting in falling branches or trees being blown over during storms. The impact of climate change makes trees more vulnerable than ever.

Innovative technologies

The Smart Sensor Systems Team, part of the  Centre of Expertise Digital Operations & Finance of The Hague University of Applied Sciences, has developed a lidar camera system that can create a detailed 3D representation of the environment. This artificial intelligence-based technology shows promising results and can recognise plants at a speed of 80 kilometres per hour. “Super expensive high-speed cameras can do that too, but the trick is to develop affordable systems for this purpose and we have succeeded”, elaborates Bolte proudly. “We are able to achieve an 85% recognition rate, despite challenges such as motion blur due to the high speed”.

Complex job

For trees, the research project is even more challenging. "We can accurately determine the height of trees with lidar images", Bolte explains. The point cloud generated by lidar allows the distance and height of trees to be measured precisely. However, information on tree species requires camera images. "With approximately 217 different tree species along the Dutch motorways, plus variations, that is a complex job. In addition to camera images, we also use multispectral images to determine health status. These images tell us a lot about the condition of a tree".

Combining data

The project team starts by combining data from different sources. Bolte: “We map out which databases are available, such as the tree register and the Netherlands Elevation Model (Actueel Hoogtebestand Nederland), and assess the quality of that data. Interviews with experts from the Directorate-General of Public Works and Water Management help formulate research needs. Do they just want to know if a tree is healthy or unhealthy? Or also what disease the tree has? Depending on that, we choose the right technology and sensors".

Training the model

Subsequently, the researchers determine which configurations and analysis methods are needed to optimise the measuring system. "We look for hardware and software that work at high speeds", says Bolte. "In pilot fields, we want to test technologies and collect datasets. Much of the success depends on labelled data to train the AI models. To that end, we need thousands of sharp images of different tree species and health levels".

Outlook

The ideal outcome is a system that can identify tree species and health statuses at high speeds. If this is not feasible, we consider alternatives such as using drones or lower speeds. Perhaps after research it will turn out that mapping by drone is the best option.

Challenges and opportunities

The project is facing challenges, such as obtaining enough labelled datasets and dealing with motion blur. Still, the potential is huge. The Directorate-General of Public Works and Water Management can use the results to conduct risk analyses and deploy ecologists and road inspectors to inspect specific trees. Bolte: “This technology does not only enable the Directorate-General of Public Works and Water Management to make roads safer, but also preserve biodiversity”.