FuPaSCo: Long-range and local context fusion for 3D panoptic scene completion

초록

In recent years, scene completion has emerged as a crucial component in the field of computer vision for autonomous vehicles. State-of-the-art methods based on the U-Net convolutional neural-network (CNN) architecture have made significant advancements in this area. U-Net employs an encoder-decoder architecture with skip connections to combine contextual features in the encoder to complete missing spatial details in the decoder. However, the architecture's reliance on convolutional layers can limit its ability to capture long-range contexts, which are often crucial for understanding complex scenes. To address these challenges, we leverage the capabilities of the transformer paradigm, which is designed to more effectively sense long-range context. Therefore, we propose a novel approach that includes (1) a dual-branch architecture with a full-range completion (FuraCo) module that completes the geometry by combining a CNN branch, which exploits the local context, and a transformer branch, which extracts the long-range context, and (2) an orthogonal-alignment (OrAli) generative decoder that minimizes the aggregation differences to enhance generative decoding features. Our experiments conducted on the panoptic scene-completion challenge of the SemanticKITTI dataset demonstrate that our method achieves a PQ dagger of 26.1% and outperforms previous state-of-the-art techniques by 2.49%.

키워드