1 |
Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP
|
|
|
|
In: https://hal.inria.fr/hal-03540069 ; 2022 (2022)
|
|
BASE
|
|
Show details
|
|
2 |
Automatic Normalisation of Early Modern French
|
|
|
|
In: https://hal.inria.fr/hal-03540226 ; 2022 (2022)
|
|
BASE
|
|
Show details
|
|
3 |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus
|
|
|
|
In: https://hal.inria.fr/hal-03536361 ; 2022 (2022)
|
|
BASE
|
|
Show details
|
|
4 |
Rethinking Automatic Evaluation in Sentence Simplification
|
|
|
|
In: https://hal.inria.fr/hal-03199901 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
5 |
Multilingual Unsupervised Sentence Simplification
|
|
|
|
In: https://hal.inria.fr/hal-03109299 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
6 |
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT
|
|
|
|
In: https://hal.inria.fr/hal-03161685 ; 2021 (2021)
|
|
Abstract:
Accepted at EACL 2021 ; Multilingual pretrained language models have demonstrated remarkable zero-shot cross-lingual transfer capabilities. Such transfer emerges by fine-tuning on a task of interest in one language and evaluating on a distinct language, not seen during the fine-tuning. Despite promising results, we still lack a proper understanding of the source of this transfer. Using a novel layer ablation technique and analyses of the model's internal representations, we show that multilingual BERT, a popular multilingual language model, can be viewed as the stacking of two sub-networks: a multilingual encoder followed by a task-specific language-agnostic predictor. While the encoder is crucial for cross-lingual transfer and remains mostly unchanged during fine-tuning, the task predictor has little importance on the transfer and can be reinitialized during fine-tuning. We present extensive experiments with three distinct tasks, seventeen typologically diverse languages and multiple domains to support our hypothesis.
|
|
Keyword:
[INFO.INFO-TT]Computer Science [cs]/Document and Text Processing
|
|
URL: https://hal.inria.fr/hal-03161685
|
|
BASE
|
|
Hide details
|
|
7 |
Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi
|
|
|
|
In: https://hal.inria.fr/hal-03161677 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
8 |
Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets
|
|
|
|
In: https://hal.inria.fr/hal-03177623 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
9 |
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering
|
|
|
|
In: https://hal.inria.fr/hal-03109187 ; 2021 (2021)
|
|
BASE
|
|
Show details
|
|
10 |
Can Character-based Language Models Improve Downstream Task Performance in Low-Resource and Noisy Language Scenarios? ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
First Align, then Predict: Understanding the Cross-Lingual Ability of Multilingual BERT ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models
|
|
|
|
In: https://hal.inria.fr/hal-03109106 ; 2020 (2020)
|
|
BASE
|
|
Show details
|
|
13 |
Can Multilingual Language Models Transfer to an Unseen Dialect? A Case Study on North African Arabizi ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Synthetic Data Augmentation for Zero-Shot Cross-Lingual Question Answering ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
When Being Unseen from mBERT is just the Beginning: Handling New Languages With Multilingual Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
MUSS: Multilingual Unsupervised Sentence Simplification by Mining Paraphrases ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations ...
|
|
|
|
BASE
|
|
Show details
|
|
18 |
Controllable Sentence Simplification
|
|
|
|
In: https://hal.inria.fr/hal-02445874 ; 2019 (2019)
|
|
BASE
|
|
Show details
|
|
19 |
CamemBERT: a Tasty French Language Model
|
|
|
|
In: https://hal.inria.fr/hal-02445946 ; 2019 (2019)
|
|
BASE
|
|
Show details
|
|
20 |
Modeling German Verb Argument Structures: LSTMs vs. Humans
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-02417640 ; 2019 (2019)
|
|
BASE
|
|
Show details
|
|
|
|