Applying Game Rating Systems to Data Labeling

In this tech talk, we show how game rating systems can be applied to NLP and Search relevance data labeling at Scale AI to track the performance of crowd-workers on our platform.

Agenda:

• The challenge of skill assessment
• Overview of game rating systems
• How game rating systems work
• Analysis of the results, pros & cons, and areas for future improvement