Emerging Science of Machine Learning Benchmarks

The first comprehensive introduction to benchmarking, the engine behind progress in AIIn machine learning, researchers split their data into training and test sets, let model builders compete on the test set, and call it a benchmark. Statistical tradition prescribed locking test sets in a vault, but machine learning practitioners shared them freely. Benchmarking shouldn't have worked, but it did, ...