DE eng

Search in the Catalogues and Directories

Hits 1 – 8 of 8

1
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation ...
BASE
Show details
2
CLEVE: Contrastive Pre-training for Event Extraction ...
BASE
Show details
3
Rethinking Stealthiness of Backdoor Attack against NLP Models ...
BASE
Show details
4
Prevent the Language Model from being Overconfident in Neural Machine Translation ...
Abstract: Read paper: https://www.aclanthology.org/2021.acl-long.268 Abstract: The Neural Machine Translation (NMT) model is essentially a joint language model conditioned on both the source sentence and partial translation. Therefore, the NMT model naturally involves the mechanism of the Language Model (LM) that predicts the next token only based on partial translation. Despite its success, NMT still suffers from the hallucination problem, generating fluent but inadequate translations. The main reason is that NMT pays excessive attention to the partial translation while neglecting the source sentence to some extent, namely overconfidence of the LM. Accordingly, we define the Margin between the NMT and the LM, calculated by subtracting the predicted probability of the LM from that of the NMT model for each token. The Margin is negatively correlated to the overconfidence degree of the LM. Based on the property, we propose a Margin-based Token-level Objective (MTO) and a Margin-based Sentence-level Objective (MSO) to ...
Keyword: Computational Linguistics; Condensed Matter Physics; Deep Learning; Electromagnetism; FOS Physical sciences; Information and Knowledge Engineering; Neural Network; Semantics
URL: https://dx.doi.org/10.48448/h7at-dc65
https://underline.io/lecture/25608-prevent-the-language-model-from-being-overconfident-in-neural-machine-translation
BASE
Hide details
5
KACC: A Multi-task Benchmark for Knowledge Abstraction, Concretization and Completion ...
BASE
Show details
6
Modeling Bilingual Conversational Characteristics for Neural Chat Translation ...
BASE
Show details
7
Target-oriented Fine-tuning for Zero-Resource Named Entity Recognition ...
BASE
Show details
8
Exploring Dynamic Selection of Branch Expansion Orders for Code Generation ...
BASE
Show details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
8
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern