Abstract: Intelligent monitoring of coal release is one of crucial directions for an intelligent fully mechanized caving in a top-coal caving mining operation. The current methods have some problems including a narrow scope of application, high error rate, and hardly fulfillment of the intelligent identification for the mixed media of thick coal seam in the caving process. An improved CBAM Faster R-CNN coal-gangue mixed release state identification method based on real-time regional convolutional neural network(Faster R-CNN) was presented in this paper. The method integrates the attention mechanism algorithm(CBAM) into the ResNet50 feature extraction network in the context of the coal-gangue falling state at a comprehensive top-coal caving face, investigates the fusion optimization method of the attention mechanism fusion in the ResNet50 backbone feature extraction network, determines the best fusion position with the coal-gangue falling state detection as the target, increases the ability to extract the weight information of coal-gangue, and makes the feature extraction process focus on the coal-gangue movement state parameters. A novel testing platform of the coal-gangue masses was constructed considering in-situ dusty environment. All initial images being captured by high-speed camera were totally pre-processed with dark channel defogging and fuzzy set enhancement. The model weighting optimization was quantitatively studied with the analysis of static images under different working conditions. Also, the identification influence on the mixed and release state adopting both the model weighting optimization and dust setting preprocess with the analysis of vast static images under different working conditions was quantitatively revealed. The relevant results show that the accuracy of preprocess image recognition based on CBAM Faster R-CNN is improved by 8.84% and the recall rate is improved by 6.68% being compared with the initial images. Besides, the average check rate and recall rate of CBAM Faster R-CNN are 82.63% and 86.53% being higher than classical model with respectively 74.69% and 79.93% in terms of the model optimization effect. F₁-score value is also improved by 7%. Therefore, the image recognition with hybrid preprocessing and CBAM Faster R-CNN provides a theoretical support for the accurate identification of the coal-gangue masses in a fully mechanized top-coal caving operation considering in-situ dusty setting.