4 |
Integrating Unsupervised Data Generation into Self-Supervised Neural Machine Translation for Low-Resource Languages ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classification ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
The European Language Technology Landscape in 2020: Language-Centric and Human-Centric AI for Cross-Cultural Communication in Multilingual Europe ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Linguistically inspired morphological inflection with a sequence to sequence model ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Probing Word Translations in the Transformer and Trading Decoder for Encoder Layers ...
|
|
|
|
Abstract:
Due to its effectiveness and performance, the Transformer translation model has attracted wide attention, most recently in terms of probing-based approaches. Previous work focuses on using or probing source linguistic features in the encoder. To date, the way word translation evolves in Transformer layers has not yet been investigated. Naively, one might assume that encoder layers capture source information while decoder layers translate. In this work, we show that this is not quite the case: translation already happens progressively in encoder layers and even in the input embeddings. More surprisingly, we find that some of the lower decoder layers do not actually do that much decoding. We show all of this in terms of a probing approach where we project representations of the layer analyzed to the final trained and frozen classifier level of the Transformer decoder to measure word translation accuracy. Our findings motivate and explain a Transformer configuration change: if translation already happens in the ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://dx.doi.org/10.48550/arxiv.2003.09586 https://arxiv.org/abs/2003.09586
|
|
BASE
|
|
Hide details
|
|
9 |
INFODENS: An Open-source Framework for Learning Text Representations ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Massively Multilingual Neural Grapheme-to-Phoneme Conversion ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Predicting the Law Area and Decisions of French Supreme Court Cases ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|