1 |
Correcting Chinese Spelling Errors with Phonetic Pre-training ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora ...
|
|
|
|
Abstract:
Recent studies have demonstrated that pre-trained cross-lingual models achieve impressive performance in downstream cross-lingual tasks. This improvement benefits from learning a large amount of monolingual and parallel corpora. Although it is generally acknowledged that parallel corpora are critical for improving the model performance, existing methods are often constrained by the size of parallel corpora, especially for low-resource languages. In this paper, we propose ERNIE-M, a new training method that encourages the model to align the representation of multiple languages with monolingual corpora, to overcome the constraint that the parallel corpus size places on the model performance. Our key insight is to integrate back-translation into the pre-training process. We generate pseudo-parallel sentence pairs on a monolingual corpus to enable the learning of semantic alignments between different languages, thereby enhancing the semantic modeling of cross-lingual models. Experimental results show that ... : Accepted by EMNLP 2021 (main conference, long paper) ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2012.15674 https://dx.doi.org/10.48550/arxiv.2012.15674
|
|
BASE
|
|
Hide details
|
|
5 |
BiobankConnect: software to rapidly connect data elements for pooled analysis across biobanks using ontological and lexical indexing
|
|
|
|
BASE
|
|
Show details
|
|
6 |
BiobankConnect: software to rapidly connect data elements for pooled analysis across biobanks using ontological and lexical indexing
|
|
|
|
BASE
|
|
Show details
|
|
|
|