Sequentialimage learning-based localization for roadheaders

The coalmine comprehensive excavation face is characterized by high labor, poor safety, and low efficiency due to the special conditions of narrow space, complex technics, and extreme environment. Thus intelligent excavation technology and equipment should be developed to improve the safety and efficiency of roadway excavation. The roadheader is the core equipment for roadway construction, and the research on its automation and intelligence is of great value to society and economy. In order to realize the automation of roadheader, the problem of pose detection of the roadheader body should be solved first, and an accurate and robust self-positioning method is the premise to guarantee the effectiveness of intelligent excavation. Therefore, aiming at the problem of localization for the roadheader body, this paper proposes a time-sequence image learning-based real-time absolute 6-DoF pose detection method. Firstly, the visual positioning scheme is designed for a narrow and unstructured roadway environment by mounting a monocular camera on the roadheader body and setting an artificial feature object at a certain distance behind the roadheader. Then, the time-sequence image fusion deep learning model is presented, in which the encoder model of the well-trained multi-scale variational autoencoder-aided convolutional neural network(MSVAE-CNN) is employed to obtain the multi-scale latent features of each sample in time-series images and guarantee the robustness of image feature extraction. The feature fusion model for the time series images is constructed based on the bidirectional long short-term memory(LSTM) network, and the body poses corresponding to all time-step images are estimated simultaneously. The information of all other time steps can be used as prior knowledge when inferring the pose parameters corresponding to a certain time step image. And the accuracy and reliability of pose estimation can be improved by learning the feature correlation between image samples. In addition, the multi-head self-attention mechanism is used to further improve the feature fusion performance, and the tensor train decomposition is employed to compress the model, which can greatly reduce the number of model parameters with a slight loss of pose estimation accuracy. Finally, a small crawler-type vehicle is used for experimental study, and the NOKOV Motion Capture System is utilized to obtain the actual pose of the vehicle. The experimental results show that the estimation mean absolute errors of the proposed method are about 25 mm and 0.36° for the center offset and the yaw angle, respectively, which are fully capable of meeting the positioning requirements for roadheaders and significantly better than that of the MSVAE-CNN model. Thus the effectiveness and high performance of the proposed sequential image learning-based localization method can be demonstrated.