Interesting talk today by Sriganesh Madhvanath from HP Labs on the research problems in crowdsourcing. Many aspects were introduced:
- What should be automated, and what should be crowdsourced?
- How can we guarantee SLO's (service level objectives) when offering services to customers based on crowdsourcing?
- The three axes for trade-off: Error (Accuracy) vs. Time vs. Cost (trying to reduce one causes the others to increase).
- How to reduce the task into small sizes, which can be crowdsourced?
- How to design the user interface to improve the chances of task completion?
- How to combine the results of the crowdsourcing (majority vote, etc.)?
- How to learn the quality of workers? How to recruit for a given task?
- etc.
