外文摘要：In the agricultural sector worldwide, there is a need for more human labor in fruit harvesting and processing operations, requiring advanced technologies like human-robot collaboration (HRC), where robots collaborate with humans to improve safety, efficiency, and productivity. In this context, we propose a new end-toend vision-based detection, tracking, and classification (DTC) architecture enabling robots to assist human pickers in fruit-picking operations. The proposed solution includes using RGB-D camera perception, customized convolutional and recurrent neural network models, and pose estimation to detect, track, and classify the activities of human pickers from continuous streaming data captured in different seasons. It also provides a module for location estimation of the human pickers, allowing the robot to navigate to the estimated goal and assist them in real-time with logistic tasks during harvesting. The authors contribute a new video dataset of real fruit-picking operations under suboptimal conditions, which is used to evaluate the effectiveness of the proposed solution. Simulation results, obtained from the collected experimental dataset, show that the overall time required to compute insights-to-action is around six seconds for challenging scenarios such as multiple human pickers in the scene. This research aims to establish a baseline, end-to-end AI pipeline of insight predictions to prescriptive robot actions in fruit-picking operations to reduce workers' working time and physical fatigue in logistics operations in the field. The authors expect this study to be used further to improve and develop anticipatory scheduling of robots methodology to save time and energy for pickers in logistics activities. The vision-based DTC architecture presents a reliable and cost-effective solution to address the scarcity of human labor in agriculture by facilitating the collaboration between humans and robots in fruit-picking tasks, as shown by simulations in the ROS-Gazebo framework.
外文关键词：Smart farming;Agricultural automation;sensor fusion;Human-robot collaboration;Human motion prediction
作者：Pal, Abhishesh;Leite, Antonio Candea;Johan, Pal
作者单位：Norwegian Univ Life Sci NMBU
期刊名称：ROBOTICS AND AUTONOMOUS SYSTEMS
期刊影响因子：0.0
出版年份：2024
出版刊次：172
原文传递申请：江苏省科技资源（工程技术文献）统筹服务平台