Event detail

Bo Lang, PhD
Electrical Engineering, University of Colorado Denver

Abstract

Automatic recognition of human actions is an important and challenging problem in surveillance and intelligence transportation areas. Dynamics of human body skeletons convey significant information for human action recognition, which attracted much attention in computer vision. Skeleton-based action recognition is widely used in recent applications due to its robustness to illumination and scene changes.

The skeletal modality can be represented naturally by a time series of body keyjoints with the form of 2D or 3D coordinates. Human motion can then be identified by analyzing its motion patterns. Conventional pose estimation approaches for getting skeleton data which are done on visible color imaging data could cause several problems in environments such as occlusion or complex backgrounds and low illumination. Whereas thermal camera is stable to human body detection regardless of the lighting condition. On the contrary, thermal data always lose the fine visual details of human objects, especially at long distance.

In this seminar, our multispectral pose estimation algorithm used to generate the skeleton body keyjoints data from multispectral images or videos will be presented firstly. Then, to capture richer dependencies besides the fixed skeleton graphs, we develop the Two Steam Feedback Attention Based GCN (2S-ATGCN) model for skeleton-based action recognition which stacks attention-based graph convolution and temporal convolution as a basic building block, to learn both spatial and temporal features from multispectral data in different environment. Experiments were conducted to evaluate both pose estimation and human action recognition parts on the typical datasets. Results have showed that the overall model achieves large improvement for human action recognition under the dark environment. The flexibility of 2S-ATGCN model also opens up many possible directions for future works.

Email Address: bo.lang@ucdenver.edu