Abstract: In order to realize the governance utilization of the mined-out and subsidence area, the stability of the mine goaf roof is studied and analyzed.Considering the nonlinear instability failure process of the gob roof rock mass, respectively build the gob roof stability analysis models from structural mechanics and rock mass constitutive relation, the two points of view.First the gob roof rock beam structure is simplified as a mechanical three hinged arch model, then using mechanics analysis, obtains the total potential energy of its mechanical structure system, using the catastrophe theory, establishes the cusp catastrophe model of the instability of the gob roof rock beam; then builds the general model of chimney caving of the gob roof rock based on the theory of limit equilibrium analysis, considering the wide application of practical engineering, and then establishes the rectangular stope roof chimney caving limit equilibrium analysis model based on the generalized H-B failure criteria.Finally the two models are analyzed and validated by different engineering examples, and the calculation results are consistent with the engineering actual situation, it shows that the two models are suitable for analysis of roof stability of mined-out area, and it provides a new research approach for stability analysis of goafs.