CalibRobustBEV: calibration-robust 3D object detection from BEV

Multi-modality sensor fusion has emerged as a prevailing trend in 3D object detection tasks. However, existing research predominantly emphasizes the efficient fusion of data from diverse sensors, overlooking the potential severe consequences of calibration failures. In this paper, we present an innovative analysis and prediction of scenarios that could lead to fusion algorithm failures, along with introducing remedial measures to enhance model robustness. Specifically, leveraging our predicted outcomes, we proactively generate similar hazardous scenarios during the model training phase to facilitate generalization capabilities. Subsequently, we introduce a query mechanism during data fusion to identify the appropriate fusion target in the event of miscalibration. Evaluation on the nuScenes dataset demonstrates that our approach can mitigate model instability by up to 90%, and our framework can be seamlessly adapted to other fusion algorithms.