A study on deep reinforcement learning-based exploration intelligence for occluded object search

초록

This paper proposes a deep reinforcement learning (DRL)-based intelligent action policy for efficiently searching and grasping target objects that are completely occluded in cluttered shelf environments. The framework integrates two key modules: a Column-wise Exploration Selector (CwES), a fully convolutional network (FCN)based module that infers target existence probabilities by fusing visual similarity and geometric occlusion cues from RGB images, and a DRL-based Action Selector that determines optimal grasping and sweeping actions based on the probabilistic guidance provided by the CwES. Comparative experiments were conducted against state-of-the-art heuristic and tree-search methods, as well as human performance, across various randomized scenarios. In a 3 & times; 4 shelf environment, the proposed method achieved a 96% success rate and 3.48 mean steps, outperforming existing methods by over 33% in success rate and 37% in step count while maintaining near-human performance. To evaluate configuration-robust scalability, the framework was extended to a more complex 4 & times; 5 environment via transfer learning by adjusting only the input and output dimensions of the pre-trained model. In this environment, the system maintained a 95% success rate, surpassed human search efficiency by 16.6%, and improved the success rate over existing methods by over 123.5%. Furthermore, robustness evaluations under assumption-violating conditions, including adversarial edge-case placements and non-grid object arrangements, confirmed success rates of 80% and 100% respectively, demonstrating reliable configuration-robust generalization beyond standard grid-based settings.

키워드