Vision-IMU Integrated Vehicle Pose Estimation based on Hybrid Multi-Feature Deep Neural Network and Federated Filter

In this paper, we propose a novel vehicle pose estimation methodology based on vision-aided IMU. The accuracy of the estimated vehicle pose is improved from two aspects: reducing the accumulated error of inertial sensor and adapting to different vehicle motion models. Firstly, a hybrid multi-feature deep neural network (HMF -DNN) is designed to estimate the vehicle motion parameters (three-dimensional velocity and three-dimensional angular velocity) and output the dynamic degree of vehicle motion. Since two consecutive frames of images are used as the input of the network, the estimated vehicle motion parameters are not affected by accumulative errors and can be used to correct the accumulative errors of the IMU. Furthermore, a Federated Unscented Kalman filter (FUKF) with different vehicle motion models is designed to estimate the vehicle pose. The FUKF fuses the vehicle motion parameters estimated by vision with the IMU data, and the information-sharing factors of the local filters are the probability of the vehicle dynamic degree output by the HMF -DNN. In other words, the motion model used to estimate the vehicle pose can be adaptive to the actual vehicle motion situation, thus the accuracy of the estimated vehicle pose can be improved. To verify the effectiveness of the proposed methodology, we used the real field test data from the KITTI dataset. The experimental results illustrate the feasibility of the proposed methodology and indicate that accurate vehicle pose can be obtained.