Hey, vitrivr! - A Multimodal UI for Video Retrieval

Goel, Prateek and Giangreco, Ivan and Rossetto, Luca and Tanase, Claudiu and Schuldt, Heiko. (2017) Hey, vitrivr! - A Multimodal UI for Video Retrieval. In: Advances in Information Retrieval. ECIR 2017, 10193.

[img] PDF - Accepted Version

Official URL: http://edoc.unibas.ch/58194/

Downloads: Statistics Overview


In this paper, we present a multimodal web-based user interface for the vitrivr system. vitrivr is a modern, open-source video retrieval system for searching in large collections of video using a great variety of query modes, including query-by-sketch, query-by-example and query-by-motion. With the multimodal user interface, prospective users benefit from being able to naturally interact with the vitrivr system by using spoken commands and also by applying multimodal commands which combine spoken instructions with manual pointing. While the main strength of the UI is the seamless combination of speech-based and sketch-based interaction for multimedia similarity search, the speech modality has shown to be very effective for retrieval on its own. In particular, it helps overcoming accessibility boundaries and offering retrieval functionality for users with disabilities. Finally, for a holistic natural experience with the vitrivr system, we have integrated a speech synthesis engine that returns spoken answers to the user.
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Informatik > Databases and Information Systems (Schuldt)
UniBasel Contributors:Schuldt, Heiko and Giangreco, Ivan and Rossetto, Luca and Tanase, Claudiu-Ioan
Item Type:Conference or Workshop Item, refereed
Conference or workshop item Subtype:Conference Paper
Series Name:Lecture Notes in Computer Science
Note:Publication type according to Uni Basel Research Database: Conference paper
Identification Number:
edoc DOI:
Last Modified:11 Feb 2022 18:45
Deposited On:09 Mar 2018 14:27

Repository Staff Only: item control page