| 1 |
Towards Explainable Evaluation Metrics for Natural Language Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 2 |
Pushing the right buttons: adversarial evaluation of quality estimation
|
|
|
|
In: Proceedings of the Sixth Conference on Machine Translation ; 625 ; 638 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Continual Quality Estimation with Online Bayesian Meta-Learning ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Findings of the WMT 2021 Shared Task on Quality Estimation ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 8 |
Pushing the Right Buttons: Adversarial Evaluation of Quality Estimation ...
|
|
|
|
Abstract:
Current Machine Translation (MT) systems achieve very good results on a growing variety of language pairs and datasets. However, they are known to produce fluent translation outputs that can contain important meaning errors, thus undermining their reliability in practice. Quality Estimation (QE) is the task of automatically assessing the performance of MT systems at test time. Thus, in order to be useful, QE systems should be able to detect such errors. However, this ability is yet to be tested in the current evaluation practices, where QE systems are assessed only in terms of their correlation with human judgements. In this work, we bridge this gap by proposing a general methodology for adversarial testing of QE for MT. First, we show that despite a high correlation with human judgements achieved by the recent SOTA, certain types of meaning errors are still problematic for QE to detect. Second, we show that on average, the ability of a given model to discriminate between meaning-preserving and ...
|
|
Keyword:
Bilingual Lexicon Induction; Computational Linguistics; Language Models; Machine Learning; Machine Learning and Data Mining; Machine translation; Natural Language Processing
|
|
URL: https://underline.io/lecture/39483-pushing-the-right-buttons-adversarial-evaluation-of-quality-estimation
https://dx.doi.org/10.48448/ekzz-hh47
|
|
BASE
|
|
Hide details
|
|
|
| 10 |
deepQuest-py: large and distilled models for quality estimation
|
|
|
|
BASE
|
|
Show details
|
|
|
| 11 |
Findings of the WMT 2021 shared task on quality estimation
|
|
|
|
In: 689 ; 730 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 12 |
deepQuest-py: large and distilled models for quality estimation
|
|
|
|
In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations ; 382 ; 389 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 13 |
Backtranslation feedback improves user confidence in MT, not quality
|
|
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Knowledge distillation for quality estimation
|
|
|
|
In: 5091 ; 5099 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 15 |
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 16 |
Unsupervised quality estimation for neural machine translation
|
|
|
|
In: 8 ; 539 ; 555 (2020)
|
|
BASE
|
|
Show details
|
|
|
| 17 |
An exploratory study on multilingual quality estimation
|
|
|
|
In: 366 ; 377 (2020)
|
|
BASE
|
|
Show details
|
|
|
| 18 |
BERGAMOT-LATTE submissions for the WMT20 quality estimation shared task
|
|
|
|
In: 1010 ; 1017 (2020)
|
|
BASE
|
|
Show details
|
|
|
| 19 |
Findings of the WMT 2020 shared task on quality estimation
|
|
|
|
In: 743 ; 764 (2020)
|
|
BASE
|
|
Show details
|
|
|
| 20 |
MLQE-PE: A multilingual quality estimation and post-editing dataset
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
