1 |
Correcting Chinese Spelling Errors with Phonetic Pre-training ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation ...
|
|
Sun, Yu; Wang, Shuohuan; Feng, Shikun; Ding, Siyu; Pang, Chao; Shang, Junyuan; Liu, Jiaxiang; Chen, Xuyi; Zhao, Yanbin; Lu, Yuxiang; Liu, Weixin; Wu, Zhihua; Gong, Weibao; Liang, Jianzhong; Shang, Zhizhou; Sun, Peng; Liu, Wei; Ouyang, Xuan; Yu, Dianhai; Tian, Hao; Wu, Hua; Wang, Haifeng. - : arXiv, 2021
|
|
Abstract:
Pre-trained models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. Recent works such as T5 and GPT-3 have shown that scaling up pre-trained language models can improve their generalization abilities. Particularly, the GPT-3 model with 175 billion parameters shows its strong task-agnostic zero-shot/few-shot learning capabilities. Despite their success, these large-scale models are trained on plain texts without introducing knowledge such as linguistic knowledge and world knowledge. In addition, most large-scale models are trained in an auto-regressive way. As a result, this kind of traditional fine-tuning approach demonstrates relatively weak performance when solving downstream language understanding tasks. In order to solve the above problems, we propose a unified framework named ERNIE 3.0 for pre-training large-scale knowledge enhanced models. It fuses auto-regressive network and auto-encoding network, so that the trained model can be easily tailored for both ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2107.02137 https://dx.doi.org/10.48550/arxiv.2107.02137
|
|
BASE
|
|
Hide details
|
|
3 |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
BiobankConnect: software to rapidly connect data elements for pooled analysis across biobanks using ontological and lexical indexing
|
|
|
|
BASE
|
|
Show details
|
|
6 |
BiobankConnect: software to rapidly connect data elements for pooled analysis across biobanks using ontological and lexical indexing
|
|
|
|
BASE
|
|
Show details
|
|
|
|