Abstract: As a research hotspot in the field of computer vision, behavior recognition has been widely applied in intelligent security, intelligent monitoring, intelligent medical treatment and other fields in today's society, and the application of deep learning, which has outstanding performance in computer vision, has a more significant effect on behavior recognition research. Compared with the traditional method based on manual feature extraction, the behavior recognition method based on deep learning has the advantages of fast speed, strong robustness and high accuracy. Therefore, this paper summarizes the video behavior recognition method based on deep learning. At home and abroad based on the latest published paper summarizes relevant literature, firstly analyzes the traditional behavior recognition method and the corresponding improvement points, according to different network architecture detailed carding behavior recognition method based on the deep study, then study the comparison of common recognition data sets and the performance of the algorithm in the data set is comparing the advantages and disadvantages, finally summarizes the research of this field, This paper focuses on the existing problems and looks into the future, hoping to enlighten and help researchers.