Gesture of Interest: Gesture Search for Multi-Person, Multi-Perspective TV Footage

Parian, Mahnaz and Walzer, Claire and Rossetto, Luca and Heller, Silvan and Dupont, Stéphane and Schuldt, Heiko. (2021) Gesture of Interest: Gesture Search for Multi-Person, Multi-Perspective TV Footage. In: Proceedings of the 18th International Conference on Content-Based Multimedia Indexing (CBMI '21). pp. 1-6.

[img] PDF - Published Version
Restricted to Repository staff only


Official URL: https://edoc.unibas.ch/87018/

Downloads: Statistics Overview


In real-world datasets, specifically in TV recordings, videos are often multi-person and multi-angle, which poses significant challenges for gesture recognition and retrieval. In addition to being of interest to linguists, gesture retrieval is a novel and challenging application for multimedia retrieval. In this paper, we propose a novel method for spatio-temporal gesture retrieval based on visual and pose information which can retrieve similar gestures in multi-person scenes through continuous shots. The attention-aware features, extracted from human pose keypoints, together with a sophisticated pre-processing module, alleviate the susceptibility of gesture retrieval to background noise and occlusion. We have evaluated our method on a subset of the NewsScape Dataset. Our experimental results demonstrate the effectiveness of the proposed method in retrieving similar results in occluded scenes as measured by the quality of the top 5 results.
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Informatik > Databases and Information Systems (Schuldt)
UniBasel Contributors:Heller, Silvan and Schuldt, Heiko and Parian-Scherb, Mahnaz and Walzer, Claire and Rossetto, Luca
Item Type:Conference or Workshop Item, refereed
Conference or workshop item Subtype:Conference Paper
Note:Publication type according to Uni Basel Research Database: Conference paper
Identification Number:
edoc DOI:
Last Modified:12 Apr 2022 10:21
Deposited On:12 Apr 2022 10:21

Repository Staff Only: item control page