Practice of Crowdsourcing

Many data-intensive applications that use machine learning or artificial intelligence techniques depend on humans providing the initial dataset, enabling algorithms to process the rest or for other humans to evaluate the performance of such algorithms.

Not only can labeled data for training and evaluation be collected faster, cheaper, and easier than ever before, but we now see the emergence of hyb...