Priberam

Few-shot learning with transformers via graph embeddings for molecular property discovery

Molecular property prediction is an essential task in drug discovery. Recently, deep neural networks have accelerated the discovery of compounds with improved molecular profiles for effective drug development. In particular, graph neural networks (GNNs) have played a pivotal role in identifying promising drug candidates with desirable molecular properties. However, it is common for only a few molecules to share the same set of properties, which presents a low-data problem unanswered by regular machine learning (ML) approaches. Transformer networks have also emerged as a promising solution to model the long-range dependence in molecular embeddings and achieve encouraging results across a wide range of molecular property prediction tasks. Nonetheless, these methods still require a large number of data points per task to achieve acceptable performance. In this seminar, we discuss how few-shot learning, GNNs and Transformer architectures can be used to face the challenges in molecular property prediction for drug discovery and development. In particular, we explain how molecules can be described by molecular graphs to aggregate the local spatial context using molecular graph embeddings, and how it is possible to preserve the global information in these molecular embeddings using Transformer attention networks. Furthermore, we also introduce the concept of a few-shot meta-learning framework which iteratively updates model parameters across few-shot tasks to predict new molecular properties with limited available data.

Luís H. M. Torres

Luís H. M. Torres received the BSc and MSc degree in Biomedical Engineering, specializing in Clinical Informatics and Bioinformatics, from the University of Coimbra. He is currently a research fellow at the Center of Informatics and Systems of the University of Coimbra (CISUC) and is pursuing the PhD degree in Computer Science at the Department of Informatics Engineering of the University of Coimbra. His research interests include deep learning (DL) and meta-learning applied to computational biology, genetics and drug discovery.CISUC