Occlusion-aware 3D Priors for Deep Learning-based Applications

Abstract

This paper investigates the benefits of incorporating point visibility information of 3D point clouds within a deep learning framework, using occlusion-aware 3D priors. The presented methods for deriving the visibility of each point rely on ray-casting techniques, making the proposed solution generic and sensor independent. We demonstrate the benefits of integrating point visibility using two real-world applications. In a first application, a novel data augmentation technique is proposed leveraging occlusion-aware CAD 3D priors, resulting in state-of-the-art 3D vehicle detection. In a second application, we integrate visibility information into a vehicle pose estimation pipeline based on 3D priors. The presented techniques achieve state-of-the-art performance, significantly improving both translation and rotation accuracy on the Apollo3DCar dataset.