Novelty and Coverage: An Integrated Metric Addressing Quality Trade-offs in AI-Generated Compounds for Drug Discovery

Introduction: Generative AI models have been introduced to find novel chemical compounds for drug discovery. To evaluate different generative AI models quantitatively, computational metrics such as rule of five, validity, novelty, uniqueness and FCD are needed. However, conventional metrics still have limitations in terms of comprehensive evaluation including drug-likeness, diversity, and novelty. Method: In this paper, we propose a metric, novelty, and coverage (NC), which measures the structural similarity between the generated set and the actual ligand set. Novelty and coverage filters the generated set by comparing molecular properties with known drugs, and then calculates structural similarity and diversity, allowing a comprehensive evaluation based on the trade-off relationship between them. For the comparison of conventional metrics and NC, we used models from the MOSES platform in addition to the latest deep learning models. We built the two representative sets for the comparison of molecular properties (ChEMBL) and structural similarity (ZINC). Result: The results show that conventional metrics can be grouped into three clusters (drug-likeness, external diversity, internal diversity) and point out their limitations. We compared the results of conventional metrics and NC, and then verified that NC explained conventional metrics. NC explained comprehensive evaluation in drug discovery with harmonic mean between contrasting features in chemical generative models. Availability and implementation: Codes are freely available to non-commercial users at https://github.com/KyoungYeulLee/drug_novelty_and_coverage.