Efficient spatiotemporal CNNS for deep human action recognition | |
| Author | Khanal, Bishal |
| Call Number | AIT Thesis no.CS-23-04 |
| Subject(s) | Human activity recognition Neural networks (Computer science) |
| Note | A thesis submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Computer Science |
| Publisher | Asian Institute of Technology |
| Abstract | Human action recognition has important applications in video surveillance systems to identify human activities, such as fall detection in elderly people. Various architectures based on CNNs and GCNs have been used to achieve this task, but there is still a great deal of room for improvement. The main drawback of the state-of-the-art 3D CNN based model is its inference speed, and the main drawback of the GCN-based model is its accuracy. In this work, I present an efficient spatiotemporal CNNs architecture with competitive accuracy and inference speed. I combine the techniques used in a state of-the-art 3D CNN-based architecture and a GCN-based architecture in a single model. The existing 3D-CNN-based model uses a sequence of 2D heatmaps, while the existing GCN-based model uses coordinate information. I use a sequence of 1D heatmaps as an input, reducing computational complexity relative to the 2D heatmaps but retaining more information than just coordinates to better classify actions. In addition, I add a spatial and temporal feature extraction technique from a GCN-based model to the 3D CNN model to improve accuracy. On the NTU RGB+D cross-subject benchmark, my best-proposed model achieves 92.3% test accuracy using only joint information which is better than the existing GCN-based model and is faster than the existing 3D CNN-based model. Inference speed is around 130 FPS on a NVIDIA GeForce RTX 2080 Ti GPU. In the AIT action dataset, a dataset collected by the AIT AI Center, the best model achieves 90.8% test accuracy in a cross-subject evaluation. |
| Year | 2023 |
| Type | Thesis |
| School | School of Engineering and Technology |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Computer Science (CS) |
| Chairperson(s) | Dailey, Matthew N. |
| Examination Committee(s) | Chaklam Silpasuwanchai;Mongkol Ekpanyapong |
| Scholarship Donor(s) | His Majesty the King’s Scholarships (Thailand) |
| Degree | Thesis (M. Eng.) - Asian Institute of Technology, 2023 |