1 |
Combining Static and Contextualised Multilingual Embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
Do Multilingual Language Models Capture Differing Moral Norms? ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Modeling Target-Side Morphology in Neural Machine Translation: A Comparison of Strategies ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Pushing the right buttons: adversarial evaluation of quality estimation
|
|
|
|
In: Proceedings of the Sixth Conference on Machine Translation ; 625 ; 638 (2022)
|
|
Abstract:
© (2021) The Authors. Published by Association for Computational Linguistics. This is an open access article available under a Creative Commons licence. The published version can be accessed at the following link on the publisher’s website: https://aclanthology.org/2021.wmt-1.67 ; Current Machine Translation (MT) systems achieve very good results on a growing variety of language pairs and datasets. However, they are known to produce fluent translation outputs that can contain important meaning errors, thus undermining their reliability in practice. Quality Estimation (QE) is the task of automatically assessing the performance of MT systems at test time. Thus, in order to be useful, QE systems should be able to detect such errors. However, this ability is yet to be tested in the current evaluation practices, where QE systems are assessed only in terms of their correlation with human judgements. In this work, we bridge this gap by proposing a general methodology for adversarial testing of QE for MT. First, we show that despite a high correlation with human judgements achieved by the recent SOTA, certain types of meaning errors are still problematic for QE to detect. Second, we show that on average, the ability of a given model to discriminate between meaningpreserving and meaning-altering perturbations is predictive of its overall performance, thus potentially allowing for comparing QE systems without relying on manual quality annotation.
|
|
Keyword:
adversarial evaluation; machine translation; quality estimation
|
|
URL: http://hdl.handle.net/2436/624376
|
|
BASE
|
|
Hide details
|
|
5 |
Do not neglect related languages: The case of low-resource Occitan cross-lingual word embeddings ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Anchor-based Bilingual Word Embeddings for Low-Resource Languages ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Improving Machine Translation of Rare and Unseen Word Senses ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Combining Word Embeddings with Bilingual Orthography Embeddings for Bilingual Dictionary Induction ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Combining Word Embeddings with Bilingual Orthography Embeddings for Bilingual Dictionary Induction
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Pragmatic information in translation: a corpus-based study of tense and mood in English and German ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
ContraCAT: Contrastive Coreference Analytical Templates for Machine Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Anchor-based Bilingual Word Embeddings for Low-Resource Languages ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
On the Language Neutrality of Pre-trained Multilingual Representations ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Embedding Learning Through Multilingual Concept Induction ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Embedding Learning Through Multilingual Concept Induction ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|