4 |
Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Deep learning for sentence clustering in essay grading support ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Towards Fully Bilingual Deep Language Modeling ...
|
|
|
|
Abstract:
Language models based on deep neural networks have facilitated great advances in natural language processing and understanding tasks in recent years. While models covering a large number of languages have been introduced, their multilinguality has come at a cost in terms of monolingual performance, and the best-performing models at most tasks not involving cross-lingual transfer remain monolingual. In this paper, we consider the question of whether it is possible to pre-train a bilingual model for two remotely related languages without compromising performance at either language. We collect pre-training data, create a Finnish-English bilingual BERT model and evaluate its performance on datasets used to evaluate the corresponding monolingual models. Our bilingual model performs on par with Google's original English BERT on GLUE and nearly matches the performance of monolingual Finnish BERT on a range of Finnish NLP tasks, clearly outperforming multilingual BERT. We find that when the model vocabulary size is ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2010.11639 https://arxiv.org/abs/2010.11639
|
|
BASE
|
|
Hide details
|
|
10 |
WikiBERT models: deep transfer learning for many languages ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Exploring Cross-sentence Contexts for Named Entity Recognition with BERT ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Dependency parsing of biomedical text with BERT
|
|
|
|
In: BMC Bioinformatics (2020)
|
|
BASE
|
|
Show details
|
|
17 |
A neural classification method for supporting the creation of BioVerbNet ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
A neural classification method for supporting the creation of BioVerbNet ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
A neural classification method for supporting the creation of BioVerbNet ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
A Neural Classification Method for Supporting the Creation of BioVerbNet ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|