Spatial-Temporal Attention Network with Multi-similarity Loss for Fine-Grained Skeleton-Based Action Recognition

In skeleton-based action recognition, the Graph Convolutional Networks (GCNs) have achieved remarkable results. However, in fine-grained skeleton-based action recognition, existing methods can’t distinguish highly similar sample pairs well. This is because that the current methods do not pay attention to highly similar sample pairs and can’t model complex actions through the spatial-temporal separation framework. In order to solve the above problems, we combine the multi-similarity loss function to weight the difficult sample pairs, and propose a novel ST-Attention module to construct the nodes connection between spatial and temporal. Finally, we used the Spatial-Temporal Attention Network with Multi-Similarity Loss (STATT-MS) to conduct experiments on NTU-RGBD-60, FSD-10 and UAV-human datasets and achieved state-of-the-art performance.