Buff-tailed bumblebee (Bombus terrestris) feeding on the nectar of Creeping thistle (Cirsium arvense) flowers © Damien Hicks

---

Authors Damien Hicks and Christoph Kratz introduce their team’s latest research demonstrating the use of machine learning for quadrat surveys to improve accessibility and resource efficiency of current methods for floral vegetation monitoring.

The nectar sugar contained in flowers is a key driver of pollinator abundance and diversity. Reduction of this food resource, primarily through habitat loss and changes in land-use, is indicated as a major cause of insect pollinator decline yet its measurement in the field and laboratory is laborious, and requires specialist skills.

To address this, an international partnership of Natural England, the University of Edinburgh, University of Orléans and Microsoft Corporation have trained a Convolutional Neural Network model to identify and count the nectar-producing flowers in surveyors’ photographs of flower-rich grasslands – which can then be used to calculate the weight of sugar available to pollinators.

Building a deep learning model

One of the nice things about this research was that all of the equipment used was inexpensive. We took over 2,000 top-down images, each of around 1m² of vegetation with no extraneous objects (e.g. quadrat frame, litter, feet!), then used open-source software (Python, R and VoTT) to create a dataset of 25,000 labelled ‘tags’ (i.e. bounding boxes) around the nectar-producing flower species in these images.

With the support of the Microsoft AI for Earth program we accessed a powerful virtual machine to run several Convolutional Neural Network (CNN) models. This method is designed to learn feature hierarchies and local patterns in a stochastic way, typically learning these in small two-dimensional frames of input images.

Assessing model performance

Our final model can be given new, model-unseen quadrat images, from which it outputs a visual display of its proposal regions and taxon classifications together with a spreadsheet of counts of detected taxa converted to nectar sugar mass per m².  On a standard outdated laptop this takes approximately 6 seconds, and for a standard quadrat survey this tool could cut pollinator-plant survey time per stand of vegetation from hours to minutes.

To gauge the performance of the model we gathered another 50 new images from a range of countries, and compared the nectar sugar mass estimates made by the CNN to those of three human surveyors. Happily, these returned similar means and standard deviations and over half of the nectar sugar mass estimates made by the model fell within the absolute range of those of the human surveyors.

Applications of deep learning to ecological management

The project was originally conceived from discussions with the Results-Based Agricultural Payments team of Natural England.

As agri-environment schemes in the UK move from prescriptive to results-based, this approach provides an independent barometer for grassland management which is usable by both landowner and scheme administrator. This work is paralleled by increased management of greenspaces in cities, such as a new set of eight urban pollinator meadows planted this March along Edinburgh’s Firth of Forth shoreline.

The CNN model can be adapted to visual estimations of other ecological resources such as winter bird food, floral pollen volume, insect infestation and tree flowering/fruiting, and by adjustment of classification threshold may show acceptable taxonomic differentiation for presence-absence surveys. The team are investigating further applications of this deep learning approach, including the monitoring of agricultural pests, bioacoustics of bumblebees and detection of artificial drains on peatland.

Read the full article: “Deep learning object detection to estimate the nectar sugar mass of flowering vegetation” in Issue 2:3 of Ecological Solutions and Evidence.

Publication supported by Department for Environment, Food & Rural Affairs (Defra) through their AER membership.