Navigating Ultra-Large Virtual Chemical Spaces with Product-of-Experts Chemical Language Models

Ultra-large virtual chemical spaces have emerged as a valuable resource for drug discovery, providing access to billions of make-on-demand compounds with high synthetic success rates. Chemical language models can potentially accelerate the exploration of these vast spaces through direct compound generation. However, existing models are not designed to navigate specific virtual chemical spaces and often overlook synthetic accessibility. To address this gap, we introduce product-of-experts (PoE) chemical language models, a modular and scalable approach to navigating ultra-large virtual chemical spaces. This method allows for controlled compound generation within a desired chemical space by combining a prior model pre-trained on the target space with expert and anti-expert models fine-tuned using external property-specific datasets. We demonstrate that the PoE chemical language model can generate compounds with desirable properties, such as those that favorably dock to the dopamine receptor D2 (DRD2) and are predicted to cross the blood-brain barrier (BBB), while ensuring that the majority of generated compounds are present within the target chemical space. Our results highlight the potential of chemical language models for navigating ultra-large virtual chemical spaces, and we anticipate that this study will motivate further research in this direction. The source code and data are freely available at https://github.com/shuyana/poeclm/.