Cross-Modality Medical Image Retrieval with Deep Features

Mbilinyi, Ashery and Schuldt, Heiko. (2020) Cross-Modality Medical Image Retrieval with Deep Features. In: 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). pp. 2632-2639.

[img] PDF
Restricted to Repository staff only


Official URL: https://edoc.unibas.ch/81642/

Downloads: Statistics Overview


In medical imaging, modality refers to the technique and process used to create visual representations of a particular part of the body, organs, or tissues. Conventional modalities include X-ray, CT-scan, Magnetic Resonance Imaging (MRI), Ultrasound, and Positron Emission Tomography (PET). Depending on the modality used, the same disease can be detected differently, making a modality an essential filter in evaluating the relevance of search results when retrieving similar medical images. Traditionally, texture features have been used for content-based medical image retrieval. However, texture features are limited in capturing the semantic similarity between medical images, let alone their modalities. This paper explores deep features (features extracted by deep convolutional neural networks (CNN)) and analyzes their effectiveness in retrieving similar medical images, semantic-wise and modality-wise, from a collection with different medical image modalities. We have examined CNNs of different architectures pre-trained in natural images and CNNs we fine-tuned and fully-trained from scratch in medical images to extract deep features. Based on retrieval performance evaluation, we show that deep features, even though extracted by CNN pre-trained in natural images, still outperform texture features. On the other end, we show that deep features extracted by a smaller, simpler, and yet computationally efficient CNN we trained in medical images can compete with large and complex ImageNet CNNs fine-tuned or fully trained in medical images.
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Informatik > Databases and Information Systems (Schuldt)
UniBasel Contributors:Schuldt, Heiko and Mbilinyi, Ashery
Item Type:Conference or Workshop Item, refereed
Conference or workshop item Subtype:Conference Paper
Note:Publication type according to Uni Basel Research Database: Conference paper
Related URLs:
Identification Number:
edoc DOI:
Last Modified:31 Jan 2023 04:11
Deposited On:10 Feb 2021 16:34

Repository Staff Only: item control page