| 3 |
Phrase Detectives Corpus Version 2
|
|
|
|
Abstract:
*Introduction* Phrase Detectives Corpus Version 2 was developed by the School of Computer Science and Electronic Engineering at the University of Essex and consists of approximately 407,000 tokens across 537 documents anaphorically-annotated by the Phrase Detectives Game, an online interactive "game-with-a-purpose" (GWAP) designed to collect data about English anaphoric coreference. This release constitutes a new version of the Phrase Detectives Corpus (LDC2017T08) that adds significantly more annotated tokens to the data set and supplies for each markable a substantial number of judgments expressed by the players and a silver label annotation based on the probabilistic aggregation method for anaphoric information. GWAPs for creating language resources are growing. In general, they employ non-monetary incentives, such as entertainment, to motivate participation and can be successful for large-scale persistent annotation efforts. Two projects that collect linguistic resources via Phrase Detectives and other similar language-oriented GWAPs are DALI (Disagreements and Language Interpretation), led by Queen Mary University of London and the University of Essex, and the LDC NIEUW (Novel Incentives and Workflows in Linguistic Data Annotation) project through its game site Lingo Boingo, in collaboration with Queen Mary University, the University of Essex and other partners. *Data* The documents in the corpus are taken from Wikipedia articles and from narrative text in Project Gutenberg. The annotation is a simplified form of the coding scheme used in The ARRAU Corpus of Anaphoric Information (LDC2013T22). Players were asked to classify markables as referring or non-referring. Referring noun phrases could be classified either as discourse-new or discourse-old (referring to the same entity as a previous mention). Two types of non-referring expressions are identified: expletives and predicative NPs (called 'properties'). Discourse-old markables include so-called split antecedent plurals, as in Mary met John. They had dinner together. All player judgments are stored in MAS-XML format; they average 20 judgments per markable, up to 90 judgments in one case. A silver label extracted from those judgments using the MPA probabilistic annotation method (Paun et. al, 2018) is also provided. Wikipedia articles are presented as html, and all other source files are presented as plain text. All text is encoded as UTF-8. Annotations are released in three formats: (1) MAS-XML (the format in the first release), (2) a CONLL-style format based on the CoNLL 2011 and 2012 shared tasks on coreference and (3) CRAC 2018 format. *Samples* Please view the following samples: * Source * CoNLL * CRAC * MAS-XML *Updates* None at this time.
|
|
URL: https://catalog.ldc.upenn.edu/LDC2019T10
|
|
BASE
|
|
Hide details
|
|
|
| 5 |
Crowdsourcing and Aggregating Nested Markable Annotations ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 7 |
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
|
|
|
|
BASE
|
|
Show details
|
|
|
| 9 |
A Crowdsourced Corpus of Multiple Judgments and Disagreement on Anaphoric Interpretation
|
|
|
|
BASE
|
|
Show details
|
|
|
| 10 |
Exploring Language Style in Chatbots to Increase Perceived Product Value and User Engagement
|
|
|
|
BASE
|
|
Show details
|
|
|
| 11 |
A Probabilistic Annotation Model for Crowdsourcing Coreference
|
|
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Markup Infrastructure for the Anaphoric Bank: Supporting Web Collaboration
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
