Occlusion-aware 3D Morphable Models and an Illumination Prior for Face Image Analysis

Egger, Bernhard and Schönborn, Sandro and Schneider, Andreas and Kortylewski, Adam and Morel-Forster, Andreas and Blumer, Clemens and Vetter, Thomas. (2018) Occlusion-aware 3D Morphable Models and an Illumination Prior for Face Image Analysis. International Journal of Computer Vision, 126 (12). pp. 1269-1287.

PDF - Accepted Version

Official URL: https://edoc.unibas.ch/69075/

Downloads: Statistics Overview


Faces in natural images are often occluded by a variety of objects. We propose a fully automated, probabilistic and occlusion-aware 3D morphable face model adaptation framework following an analysis-by-synthesis setup. The key idea is to segment the image into regions explained by separate models. Our framework includes a 3D morphable face model, a prototype-based beard model and a simple model for occlusions and background regions. The segmentation and all the model parameters have to be inferred from the single target image. Face model adaptation and segmentation are solved jointly using an expectation-maximization-like procedure. During the E-step, we update the segmentation and in the M-step the face model parameters are updated. For face model adaptation we apply a stochastic sampling strategy based on the Metropolis-Hastings algorithm. For segmentation, we apply loopy belief propagation for inference in a Markov random field. Illumination estimation is critical for occlusion handling. Our combined segmentation and model adaptation needs a proper initialization of the illumination parameters. We propose a RANSAC-based robust illumination estimation technique. By applying this method to a large face image database we obtain a first empirical distribution of real-world illumination conditions. The obtained empirical distribution is made publicly available and can be used as prior in probabilistic frameworks, for regularization or to synthesize data for deep learning methods.
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Ehemalige Einheiten Mathematik & Informatik > Computergraphik Bilderkennung (Vetter)
UniBasel Contributors:Vetter, Thomas and Kortylewski, Adam and Morel, Andreas
Item Type:Article, refereed
Article Subtype:Research Article
Note:Publication type according to Uni Basel Research Database: Journal article
Related URLs:
edoc DOI:
Last Modified:09 Jan 2020 09:44
Deposited On:26 Feb 2019 16:47

Repository Staff Only: item control page