1 |
The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation ...
|
|
Goyal, Naman; Gao, Cynthia; Chaudhary, Vishrav; Chen, Peng-Jen; Wenzek, Guillaume; Ju, Da; Krishnan, Sanjana; Ranzato, Marc'Aurelio; Guzman, Francisco; Fan, Angela. - : arXiv, 2021
|
|
Abstract:
One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted domains, or are low quality because they are constructed using semi-automatic procedures. In this work, we introduce the FLORES-101 evaluation benchmark, consisting of 3001 sentences extracted from English Wikipedia and covering a variety of different topics and domains. These sentences have been translated in 101 languages by professional translators through a carefully controlled process. The resulting dataset enables better assessment of model quality on the long tail of low-resource languages, including the evaluation of many-to-many multilingual translation systems, as all translations are multilingually aligned. By publicly releasing such a high-quality and high-coverage dataset, we hope to foster progress in the machine translation community and beyond. ...
|
|
Keyword:
Artificial Intelligence cs.AI; Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2106.03193 https://arxiv.org/abs/2106.03193
|
|
BASE
|
|
Hide details
|
|
2 |
LAWDR: Language-Agnostic Weighted Document Representations from Pre-trained Models ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Improving Zero-Shot Translation by Disentangling Positional Information ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
As Easy as 1, 2, 3: Behavioural Testing of NMT Systems for Numerical Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Detecting Hallucinated Content in Conditional Neural Sequence Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Alternative Input Signals Ease Transfer in Multilingual Machine Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Improving Zero-Shot Translation by Disentangling Positional Information ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Improving Zero-Shot Translation by Disentangling Positional Information
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover's Distance ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Improving Zero-Shot Translation by Disentangling Positional Information ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Unsupervised quality estimation for neural machine translation
|
|
|
|
In: 8 ; 539 ; 555 (2020)
|
|
BASE
|
|
Show details
|
|
17 |
An exploratory study on multilingual quality estimation
|
|
|
|
In: 366 ; 377 (2020)
|
|
BASE
|
|
Show details
|
|
18 |
BERGAMOT-LATTE submissions for the WMT20 quality estimation shared task
|
|
|
|
In: 1010 ; 1017 (2020)
|
|
BASE
|
|
Show details
|
|
19 |
Findings of the WMT 2020 shared task on quality estimation
|
|
|
|
In: 743 ; 764 (2020)
|
|
BASE
|
|
Show details
|
|
20 |
MLQE-PE: A multilingual quality estimation and post-editing dataset
|
|
|
|
BASE
|
|
Show details
|
|
|
|