Concrete is a commonly used construction material due to its favorable engineering properties, such as high compressive strength, good durability, and resistance to corrosion. Accurate predictions of the compressive strength of this material significantly reduce the time and effort required by laboratory tests. The current paper aims to compare the performance of prominent machine learning-based approaches used for predicting the compressive strength of concrete. In addition, 11 historical datasets, collected from the literature, are used. The diversity of the input features, the data dimensionality, and the number of instances can be helpful to evaluate the generalization capability of the employed machine learning models. Repetitive data sampling processes, consisting of 20 independent runs, are carried out to obtain the machine learning models’ performances. Through experiments, it can be shown that the gradient boosting machines attain the best performance. Notably, the extreme gradient boosting machine has achieved the best outcome in five historical datasets.