Drones and computer vision Team

Principal Investigator:Professor Ming-Der Yang

—

Summary

This research project aims to establish an intelligent agricultural cultivation support system, by integrating unmanned aerial vehicle (UAV) surveillance and artificial intelligent (AI) analytical techniques. The support system provides a great help in various current agricultural management challenges, such as sparse agricultural data and manpower limitation. Three major tasks are proposed in order to achieve the goal over the four years’ project span, including forming a UAV multi-source image database, developing relevant AI image process technologies, and establishing a UAV image analysis cloud platform. Meanwhile, the UAV flight standard operation procedure (SOP) for rice cultivation surveillance at different stages is established. In the first year, rice, the major staple food crop in Taiwan, is selected and monitored for two whole crop life cycles. More than 10 TB UAV multi-source image data, including visible spectrum, multispectral, thermal, and hyperspectral images, were collected from three study sites, NCHU (National Chung Hsing University) Experimental field, NCHU Agricultural Experimental Station, and TARI (Taiwan Agriculture Research Institute) Experimental Field. We have developed various of models and applications, such as seedling positioning/counting, leaf color analysis, plant height analysis, green coverage ratio analysis, yield prediction, grain moisture content assessment, damage assessment, crop recognition using Convolutional Neural Networks (CNN) and Deep Neural Networks (DNN) with edge computing capabilities. We also developed a cloud platform of Aerial Agriculture Analysis for tasks, such as image mosaicking, image texture analysis, vegetation index analysis, and 3D model construction functions.

Keywords

Unmanned aerial vehicle (UAV), UAV multi-source image database, UAV image cloud platform, edge computing, Convolutional Neural Networks (CNN), Deep Neural Networks (DNN)

Innovations

• A UAV agricultural multi-source image database was established to cover the complete life cycle of rice growth stages. More than 10 TB UAV images, including visible spectrum, multispectral, thermal, and hyperspectral images, were collected.
• A variety of image analyses covering rice growth stages were implemented for seedling positioning/counting, leaf color analysis, plant height analysis, green coverage ratio analysis, yield prediction, grain moisture content assessment, damage assessment, and crop recognition.
• Convolutional Neural Networks (CNN) and Deep Neural Networks (DNN) were used with edge computing capability to develop a real-time crop recognition model for rice.
• A cloud platform of Aerial Agriculture Analysis was established to provide UAV image analysis and expert advice support system.

Benefits

• The project completed 354 UAV aerial imaging tasks and established a UAV agricultural multi-source image database consisting of more than 10 TB images, such as visible light, multi-spectral, and thermal image data.
• Through UAV image correction, 3D modeling, and ortho-photo, and digital terrain model (DSM) generation, the processed images, and original data are stored in a database and are displayed on a website for data sharing.
• Various field surveys were also conducted to establish a link between processed UAV images and rice growth for further applications.
• Many applications have been developed, such as seedling positioning/counting, leaf color analysis, plant height analysis, green coverage ratio analysis, yield prediction, grain moisture content assessment, damage assessment, and crop recognition, which can be applied to other crops in the future.
• A real-time crop recognition model with edge computing capacity was developed based on CNN and DNN for real-time classification and object identification.
• The cloud platform of Aerial Agriculture Analysis (Beta) has been established proving functions of UAV image uploading, automatic orthophoto mosaicking, and Google Map overlapping with GPS information.