Research and test on key technologies of intelligent monitoring and controldriven by AR/VR for fully mechanized coal mining face

In order to break through the bottleneck of data driven real time monitoring and control technology and satisfy the in depth perception of the collaborative operation for a fully mechanized coal mining face，a Virtual Reality (VR) monitoring and control technology and its research achievements are taken as the technical foundation. Also，the HoloLens2 Augmented Reality (AR) headset is taken as a main development device. Advanced methods such as industrial Internet，cloud technology，and Digital Twin (DT) are combined to carry out the research and practice of intelligent monitoring and control technology for a fully mechanized coal mining face driven by AR/VR. The characteristics of AR technology are fully utilized to make up for the sense of separation between VR monitoring and control and real scenes，and to compensate for the shortcomings of a relatively single source of monitoring data，so that the intelligent monitoring and control system for fully mechanized coal mining faces can be improved and expanded. Three key technologies are proposed and the specific solutions of each technology are analyzed. The details are as follows:① A universal，efficient，scalable，and asynchronous network framework for client cloud collaboration is constructed. Based on this framework，the key technologies of multi terminal collaborative work are studied，and the function of spatial holographic mark of the fusion of virtual and reality is realized. This technology can provide convenience for multi person collaboration in the whole process of intelligent monitoring and control of fully mechanized coal mining face. ② The recognition and tracking method based on sparse matching is combined with the recognition and tracking method based on AR identification，so that the recognition and tracking of fully mechanized coal mining equipment in fully mechanized coal mining face is realized，which provides a prerequisite for the AR monitoring and control of fully mechanized coal mining face. ③ In the DT closed loop composed of physical fully mechanized coal mining face，cloud server and digital fully mechanized coal mining face，the position and posture monitoring and the operating status recognition of fully mechanized coal mining equipment are carried out by means of multi sensor and vision fusion. The reverse control of physical fully mechanized coal mining face is completed by means of AR interaction. Test verification is carried out in a laboratory environment. The test shows that the system has low data transmission pressure，low delay，smooth operation and no jamming. On the basis of retaining the existing advantages of VR monitoring and control，the degree of immersion，convenience and the integration of virtual and reality are further improved. It has the ability to monitor the operating status of fully mechanized coal mining equipment by one or more persons and control it in reverse.