Identifying Domains of Applicability of Machine Learning Models for Materials Science

We present an extension to the usual machine learning process that allows for the identification of the domain of applicability of a fitted model, i.e., the region in its domain where it performs most accurately. This approach is applied to several vastly different but commonly used materials representations (namely the n-gram approach, SOAP, and the many body tenor representation), which are practically indistinguishable based on performance using a single error statistic. Moreover, these models appear unsatisfactory for screening applications as they fail to reliably identify the ground state polymorphs. When applying our newly developed analysis for each of the models, we can identify the domain of applicability for each model according to a simple set of interpretable conditions. We show that identification of the domain of applicability in the prediction of the formation energy enables a more accurate ground-state search - a crucial step for the discovery of novel materials.