1 |
Ethical Issues in Corpus Linguistics And Annotation: Pay Per Hit Does Not Affect Effective Hourly Rate For Linguistic Resource Development On Amazon Mechanical Turk
|
|
|
|
In: ETHics In Corpus collection, Annotation and Application workshop ; https://hal.inria.fr/hal-01324362 ; ETHics In Corpus collection, Annotation and Application workshop, May 2016, Portoroz, Slovenia (2016)
|
|
Abstract:
International audience ; Ethical issues reported with paid crowdsourcing include unfairly low wages. It is assumed that such issues are under the control of the task requester. Can one control the amount that a worker earns by controlling the amount that one pays? 412 linguistic data development tasks were submitted to Amazon Mechanical Turk. The pay per HIT was manipulated through a range of values. We examined the relationship between the pay that is offered per HIT and the effective pay rate. There is no such relationship. Paying more per HIT does not cause workers to earn more: the higher the pay per HIT, the more time workers spend on them (R = 0.92). So, the effective hourly rate stays roughly the same. The finding has clear implications for language resource builders who want to behave ethically: other means must be found in order to compensate workers fairly. The findings of this paper should not be taken as an endorsement of unfairly low pay rates for crowdsourcing workers. Rather, the intention is to point out that additional measures, such as pre-calculating and communicating to the workers an average hourly, rather than per-task, rate must be found in order to ensure an ethical rate of pay.
|
|
Keyword:
[INFO.INFO-TT]Computer Science [cs]/Document and Text Processing; Amazon Mechanical Turk; corpus annotation; corpus linguistics; crowdsourcing; ethics
|
|
URL: https://hal.inria.fr/hal-01324362/file/Cohen%20ETHI-CA%202016.pdf https://hal.inria.fr/hal-01324362/document https://hal.inria.fr/hal-01324362
|
|
BASE
|
|
Hide details
|
|
2 |
Crowdsourcing for Language Resource Development: Criticisms About Amazon Mechanical Turk Overpowering Use
|
|
|
|
In: Human Language Technology Challenges for Computer Science and Linguistics ; https://hal.inria.fr/hal-01053047 ; Vetulani, Zygmunt and Mariani, Joseph. Human Language Technology Challenges for Computer Science and Linguistics, 8387, Springer International Publishing, pp.303-314, 2014, Lecture Notes in Computer Science, 978-3-319-08957-7. ⟨10.1007/978-3-319-08958-4_25⟩ (2014)
|
|
BASE
|
|
Show details
|
|
3 |
Un turc mécanique pour les ressources linguistiques : critique de la myriadisation du travail parcellisé
|
|
|
|
In: TALN'2011 - Traitement Automatique des Langues Naturelles ; https://hal.inria.fr/inria-00617067 ; TALN'2011 - Traitement Automatique des Langues Naturelles, Jun 2011, Montpellier, France (2011)
|
|
BASE
|
|
Show details
|
|
4 |
Crowdsourcing for language resource development: critical analysis of amazon mechanical turk overpowering use
|
|
|
|
In: Language & Technology Conference : Human Language Technologies as a Challenge for Computer Science and Linguistics ; https://hal.archives-ouvertes.fr/hal-01840838 ; Language & Technology Conference : Human Language Technologies as a Challenge for Computer Science and Linguistics, Jan 2011, Poznan, Poland (2011)
|
|
BASE
|
|
Show details
|
|
5 |
Amazon Mechanical Turk: Gold Mine or Coal Mine?
|
|
|
|
In: ISSN: 0891-2017 ; EISSN: 1530-9312 ; Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-00569450 ; Computational Linguistics, Massachusetts Institute of Technology Press (MIT Press), 2011, pp.413-420. ⟨10.1162/COLI_a_00057⟩ (2011)
|
|
BASE
|
|
Show details
|
|
|
|