Abstract

Accurate citrus fruit yield and estimation is of utmost importance for precise agricultural management. Unmanned aerial vehicle (UAV) remote-sensing systems present a compelling solution to this problem. These systems capture remote-sensing imagery with both high temporal and spatial resolution, thus empowering farmers with valuable insights for better decisionmaking. This research assessed the potential application of UAV imagery combined with the YOLOv7 object detection model for the precise estimation of citrus yield. Images of citrus trees were captured in their natural field setting using a quadcopter-mounted UAV camera. Data augmentation techniques were applied to enhance the dataset diversity; the original YOLOv7 architecture and training parameters were modified to improve the model’s accuracy in detecting citrus fruits. The test results demonstrated commendable performance, with a precision of 96%, a recall of 100%, and an F1-score of 97.95%. The correlation between the fruit numbers recognized by the algorithm and the actual fruit numbers from 20 sample trees provided the coefficient R2 of 0.98. The strong positive correlation confirmed both the accuracy of the algorithm and the validity of the approach in identifying and quantifying citrus fruits on sample trees.

Keywords

Agricultural management, unmanned aerial vehicle (UAV), remote-sensing systems, YOLOv7 object detection model, crop yield estimation

REFERENCES

1. Marani R, Milella A, Petitti A, Reina G. Deep neural networks for grape bunch segmentation in natural images from a consumer-grade camera. Precision Agriculture. 2021;22:387–413. https://doi.org/10.1007/s11119-020-09736-0
2. Gongal A, Amatya S, Karkee M, Zhang Q, Lewis K. Sensors and systems for fruit detection and localization: A review. Computers and Electronics in Agriculture. 2015;116:8–19. https://doi.org/10.1016/j.compag.2015.05.021
3. Sengupta S, Lee WS. Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosystems Engineering. 2014;117:51–61. https://doi.org/10.1016/j.biosystemseng.2013.07.007
4. Maldonado Jr W, Barbosa JC. Automatic green fruit counting in orange trees using digital images. Computers and Electronics in Agriculture. 2016;127:572–581. https://doi.org/10.1016/j.compag.2016.07.023
5. Zhao C, Lee WS, He D. Immature green citrus detection based on colour feature and sum of absolute transformed difference (SATD) using colour images in the citrus grove. Computers and Electronics in Agriculture. 2016;124:243–253. https://doi.org/10.1016/j.compag.2016.04.009
6. Dorj U-O, Lee M, Yun S. An yield estimation in citrus orchards via fruit detection and counting using image processing. Computers and Electronics in Agriculture. 2017;140:103–112. https://doi.org/10.1016/j.compag.2017.05.019
7. Liu T-H, Ehsani R, Toudeshki A, Zou X-J, Wang H-J. Detection of citrus fruit and tree trunks in natural environments using a multi-elliptical boundary model. Computers in Industry. 2018;99:9–16. https://doi.org/10.1016/j.compind.2018.03.007
8. Liu S, Yang C, Hu Y, Huang L, Xiong L. A method for segmentation and recognition of mature citrus and branches-leaves based on regional features. In: Wang Y, Jiang Z, Peng Y, editors. Image and graphics technologies and applications. Singapore: Springer; 2018. pp. 292–301. https://doi.org/10.1007/978-981-13-1702-6_29
9. Xu L, Zhu S, Chen X, Wang Y, Kang Z, Huang P, et al. Citrus recognition in real scenarios based on machine vision. DYNA. Ingenier´ıa e Industria. 2020;95(1):87–93. https://doi.org/10.6036/9363
10. Zhang X, Toudeshki A, Ehsani R, Li H, Zhang W, Ma R. Yield estimation of citrus fruit using rapid image processing in natural background. Smart Agricultural Technology. 2022;2:100027. https://doi.org/10.1016/j.atech.2021.100027
11. Maheswari P, Raja P, Apolo-Apolo OE, Perez-Ruiz M. Intelligent fruit yield estimation for orchards using deep learning based semantic segmentation techniques – A review. Frontiers in Plant Science. 2021;12:684328. https://doi.org/10.3389/fpls.2021.684328
12. Yamamoto K, Guo W, Yoshioka Y, Ninomiya S. On plant detection of intact tomato fruits using image analysis and machine learning methods. Sensors. 2014;14(7):12191–12206. https://doi.org/10.3390/s140712191
13. Loddo A, Loddo M, Di Ruberto C. A novel deep learning based approach for seed image classification and retrieval. Computers and Electronics in Agriculture. 2021;187:106269. https://doi.org/10.1016/j.compag.2021.106269
14. Han B-G, Lee J-G, Lim K-T, Choi D-H. Design of a scalable and fast YOLO for edge-computing devices. Sensors. 2020;20(23):6779. https://doi.org/10.3390/s20236779
15. Sivakumar ANV, Li J, Scott S, Psota E, Jhala AJ, Luck JD, et al. Comparison of object detection and patch-based classification deep learning models on mid-to late-season weed detection in UAV imagery. Remote Sensing. 2020;12(13):2136. https://doi.org/10.3390/rs12132136
16. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, et al. SSD: Single shot multibox detector. In: Leibe B, Matas J, Sebe N, Welling M, editors. Computer Vision – ECCV 2016. Cham: Springer; 2016. pp. 21–37. https://doi.org/10.1007/978-3-319-46448-0_2
17. Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition; 2016; Vegas. IEEE; 2016. pp. 779– 788. https://doi.org/10.1109/CVPR.2016.91
18. Redmon J, Farhadi A. YOLO9000: better, faster, stronger. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition; 2017; Honolulu. IEEE; 2017. pp. 7263–7271. https://doi.org/10.1109/CVPR.2017.690
19. Redmon J, Farhadi A. YOLOv3: An incremental improvement. 2018. https://doi.org/10.48550/arXiv.1804.02767
20. Bochkovskiy A, Wang C-Y, Liao H-YM. YOLOv4: Optimal speed and accuracy of object detection. 2020. https://doi.org/10.48550/arXiv.2004.10934
21. Jocher G, Chaurasia A, Stoken A, Borovec J, Kwon Y, Fang J, et al. ultralytics/yolov5: v6.1 – TensorRT, TensorFlow edge TPU and OpenVINO export and inference. Zenodo. 2022. https://doi.org/10.5281/zenodo.6222936
22. Ge Z, Liu S, Wang F, Li Z, Sun J. YOLOX: Exceeding YOLO series in 2021. 2021. https://doi.org/10.48550/arXiv.2107.08430
23. Wang C-Y, Bochkovskiy A, Liao H-YM. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition; 2023; Vancouver. IEEE; 2023. pp. 7464–7475. https://doi.org/10.1109/CVPR52729.2023.00721
24. Girshick R. Fast r-cnn. Proceedings of the 2015 IEEE International Conference on Computer Vision; 2015; Santiago. IEEE; 2015. pp. 1440–1448. https://doi.org/10.1109/ICCV.2015.169
25. Ren S, He K, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. In: Cortes C, Lawrence N, Lee D, Sugiyama M, Garnett R, editors. Advances in neural information processing systems. Purchase Printed Proceeding; 2015.
26. Lucena F, Breunig FM, Kux H. The combined use of UAV-based RGB and DEM images for the detection and delineation of orange tree crowns with mask R-CNN: An approach of labeling and unified framework. Future Internet. 2022;14(10):275. https://doi.org/10.3390/fi14100275
27. Xu L, Wang Y, Shi X, Tang Z, Chen X, Wang Y, et al. Real-time and accurate detection of citrus in complex scenes based on HPL-YOLOv4. Computers and Electronics in Agriculture. 2023;205:107590. https://doi.org/10.1016/j.compag.2022.107590
28. Yang R, Hu Y, Yao Y, Gao M, Liu R. Fruit target detection based on BCo-YOLOv5 model. Mobile Information Systems. 2022;2022:8457173. https://doi.org/10.1155/2022/8457173
29. Lai Y, Ma R, Chen Y, Wan T, Jiao R, He H. A pineapple target detection method in a field environment based on improved YOLOv7. Applied Sciences. 2023;13(4):2691. https://doi.org/10.3390/app13042691
30. Chen J, Liu H, Zhang Y, Zhang D, Ouyang H, Chen X. A multiscale lightweight and efficient model based on YOLOv7: Applied to citrus orchard. Plants. 2022;11(23):3260. https://doi.org/10.3390/plants11233260
31. Yang H, Liu Y, Wang S, Qu H, Li N, Wu J, et al. Improved apple fruit target recognition method based on YOLOv7 model. Agriculture. 2023;13(7):1278. https://doi.org/10.3390/agriculture13071278
32. Ministry of Agriculture. https://www.agriculture.gov.ma
33. Zheng Z, Wang P, Liu W, Li J, Ye R, Ren D. Distance-IoU loss: Faster and better learning for bounding box regression. AAAI-20 Technical Tracks 7. 2020;34(7):12993–13000. https://doi.org/10.1609/aaai.v34i07.6999