Efficient posevit : exploring efficient vision transformers and training strategies for fast human pose estimation | |
| Author | Lamsal, Diwas |
| Call Number | AIT Thesis no.CS-23-05 |
| Subject(s) | Image analysis Human-computer interaction |
| Note | A thesis submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Computer Science |
| Publisher | Asian Institute of Technology |
| Abstract | Human pose estimation (HPE) serves as a critical component in various downstream applications, including activity recognition and human-computer interaction. However, it faces challenges such as occlusion, complex body poses, diverse clothing and environments, and low-quality images. State-of-the-art HPE methods adept at addressing these challenges often prove to be computationally expensive to run on low-cost edge devices. On the other hand, the efficient models designed to run on edge devices are not as accurate, especially when faced with occlusion. In this thesis, I explored the use of computationally efficient vision transformer (ViT) backbones for the task of HPE to produce an efficient model capable of running on edge devices while also addressing common HPE challenges. Specifically, I replaced the ViT encoder in ViTPose with effi cient ViT variants, and performed a comprehensive comparison. The findings indicate that the EdgeNeXt-Base model offers an optimal speed-accuracy trade-off among the evaluated models. Through knowledge distillation and multi-task training strategies, the resulting models exhibit improved performance, particularly in handling occlusion. Additionally, I produced a wheelchair-based human pose estimation and activity recognition dataset aimed at developing a system that effectively monitors the activities of people sitting in wheelchairs. |
| Year | 2023 |
| Type | Thesis |
| School | School of Engineering and Technology |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Computer Science (CS) |
| Chairperson(s) | Dailey, Matthew N. |
| Examination Committee(s) | Mongkol Ekpanyapong;Chaklam Silpasuwanchai |
| Scholarship Donor(s) | AIT Scholarships |
| Degree | Thesis (M. Eng.) - Asian Institute of Technology, 2023 |