Machine Learning Force Field Aided Cluster Expansion Approach to Configurationally Disordered Materials: Critical Assessment of Training Set Selection and Size Convergence

Cluster expansion (CE) is a powerful theoretical tool to study the configuration-dependent properties of substitutionally disordered systems. Typically, a CE model is built by fitting a few tens or hundreds of target quantities calculated by first-principles approaches. To validate the reliability of the model, a convergence test of cross-validation (CV) score to the training set size is commonly conducted to verify the sufficiency of training data. However, such test only confirms the convergence of the predictive capability of the CE model within the training set and it is unknown whether the convergence of the CV score would lead to robust thermodynamic simulation results such as order-disorder phase transition temperature $T_{\rm c}$. In this work, using carbon defective MoC$_{1-x}$ as a model system and aided by the machine-learning force field technique, a training data pool with about 13000 configurations has been efficiently obtained and used to generate different training sets of the same size randomly. By conducting parallel Monte Carlo simulations with the CE models trained with different randomly selected training set, the uncertainty in calculated $T_{\rm c}$ can be evaluated at different training set size. It is found that the training set size that is sufficient for the CV score to converge still leads to a significant uncertainty in the predicted $T_{\rm c}$, and that the latter can be considerably reduced by enlarging the training set to that of a few thousand configurations. This work highlights the importance of considering large training set for building the optimal CE model that can achieve robust statistical modeling results, and the facility provided by the machine-learning force field approach to efficiently produce adequate training data.