Few-shot learning with transformers via graph embeddings for molecular property discovery

Molecular property prediction is an essential task in drug discovery. Recently, deep neural networks have accelerated the discovery of compounds with improved molecular profiles for effective drug development. In particular, graph neural networks (GNNs) have played a pivotal role in identifying promising drug candidates with desirable molecular properties. However, it is common for only a few molecules to share the same set of properties, which presents a low-data problem unanswered by regular machine learning (ML) approaches. Transformer networks have also emerged as a promising solution to model the long-range dependence in molecular embeddings and achieve encouraging results across a wide range of molecular property prediction tasks. Nonetheless, these methods still require a large number of data points per task to achieve acceptable performance. In this seminar, we discuss how few-shot learning, GNNs and Transformer architectures can be used to face the challenges in molecular property prediction for drug discovery and development. In particular, we explain how molecules can be described by molecular graphs to aggregate the local spatial context using molecular graph embeddings, and how it is possible to preserve the global information in these molecular embeddings using Transformer attention networks. Furthermore, we also introduce the concept of a few-shot meta-learning framework which iteratively updates model parameters across few-shot tasks to predict new molecular properties with limited available data.