Machine Learning for Informed Representation Learning

Samarin, Maxim. Machine Learning for Informed Representation Learning. 2022, Doctoral Thesis, University of Basel, Faculty of Science.

Available under License CC BY-NC-ND (Attribution-NonCommercial-NoDerivatives).


Official URL: https://edoc.unibas.ch/91785/

Downloads: Statistics Overview


The way we view reality and reason about the processes surrounding us is intimately connected to our perception and the representations we form about our observations and experiences. The popularity of machine learning and deep learning techniques in that regard stems from their ability to form useful representations by learning from large sets of observations. Typical application examples include image recognition or language processing for which artificial neural networks are powerful tools to extract regularity patterns or relevant statistics. In this thesis, we leverage and further develop this representation learning capability to address relevant but challenging real-world problems in geoscience and chemistry, to learn representations in an informed manner relevant to the task at hand, and reason about representation learning in neural networks, in general.
Firstly, we develop an approach for efficient and scalable semantic segmentation of degraded soil in alpine grasslands in remotely-sensed images based on convolutional neural networks. To this end, we consider different grassland erosion phenomena in several Swiss valleys. We find that we are able to monitor soil degradation consistent with state-of-the-art methods in geoscience and can improve detection of affected areas. Furthermore, our approach provides a scalable method for large-scale analysis which is infeasible with established methods.
Secondly, we address the question of how to identify suitable latent representations to enable generation of novel objects with selected properties. For this, we introduce a new deep generative model in the context of manifold learning and disentanglement. Our model improves targeted generation of novel objects by making use of property cycle consistency in property-relevant and property-invariant latent subspaces. We demonstrate the improvements on the generation of molecules with desired physical or chemical properties. Furthermore, we show that our model facilitates interpretability and exploration of the latent representation.
Thirdly, in the context of recent advances in deep learning theory and the neural tangent kernel, we empirically investigate the learning of feature representations in standard convolutional neural networks and corresponding random feature models given by the linearisation of the neural networks. We find that performance differences between standard and linearised networks generally increase with the difficulty of the task but decrease with the considered width or over-parametrisation of these networks. Our results indicate interesting implications for feature learning and random feature models as well as the generalisation performance of highly over-parametrised neural networks.
In summary, we employ and study feature learning in neural networks and review how we may use informed representation learning for challenging tasks.
Advisors:Roth, Volker
Committee Members:Schuldt, Heiko and Kanevski, Mikhail
Faculties and Departments:05 Faculty of Science > Departement Mathematik und Informatik > Informatik > Datenanalyse (Roth)
UniBasel Contributors:Roth, Volker and Schuldt, Heiko and Samarin, Maxim
Item Type:Thesis
Thesis Subtype:Doctoral Thesis
Thesis no:14903
Thesis status:Complete
Number of Pages:xiv, 209
Identification Number:
  • urn: urn:nbn:ch:bel-bau-diss149037
edoc DOI:
Last Modified:23 Feb 2023 07:50
Deposited On:04 Jan 2023 07:56

Repository Staff Only: item control page