Science

Research

Team

About

Log in

Bringing Atomistic Deep Learning to Prime Time

Date Published

2021-12-09


Nathan C. Frey, Siddharth Samsi, Bharath Ramsundar, Connor W Coley, Vijay Gadepally

Download

Link

Abstract

Bringing Atomistic Deep Learning to Prime Time

GNNs and chemical fingerprints are the predominant approaches to representing molecules for property prediction. However, in NLP, transformers have become the de-facto standard for representation learning thanks to their strong downstream task transfer. In parallel, the software ecosystem around transformers is maturing rapidly, with libraries like HuggingFace and BertViz enabling streamlined training and introspection. In this work, we make one of the first attempts to systematically evaluate transformers on molecular property prediction tasks via our ChemBERTa model. ChemBERTa scales well with pretraining dataset size, offering competitive downstream performance on MoleculeNet and useful attention-based visualization

Company

Join us

Contact

© 2022 Deep Forest Sciences