Enhancing Autonomous Vehicle Perception through Multi-Sensor Fusion and Uncertainty-Aware Decision-Making

Abstract

Autonomous driving requires precise perception and decision making in non-static, complex and uncertain settings. Systems that only rely on one sensor, are typically not reliable in low visibility, occlusion or dynamic conditions, all of which will ultimately affect safe navigation. This research will assess the likelihood that multi-sensor fusion of RGB and depth/LiDAR, can improve the accuracy of perception, in addition to whether a model of uncertainty can be leveraged to heuristically mitigate sensor reliability. Finally, to also improve online decision-making in steering, throttle and braking driving agents. To this end, the proposed Uncertainty-Aware TransFuse fuses RGB and depth features using CNNs with Transformer-based attention. Incorporating uncertainty leverages weighted reliance upon sensor reliability/uncertainty heuristics during inference. Experimental results on the KITTI dataset demonstrated statistically significant improvements over the baseline, in navigation accuracy, object detection performance, and lane detection performance, even when the scenes were severely degraded. The proposed multi-sensor fusion system works in real-time at 32 FPS and reduces false detections in occlusion and low-light testing conditions. In short, multi-sensor fusion of RGB and depth, with uncertainty, increases overall robustness and safety. The Uncertainty-Aware TransFuse provides a robust model of real-world driving for reliable autonomous perceiving.