5 |
BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Including Signed Languages in Natural Language Processing ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Including Signed Languages in Natural Language Processing ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Provable Limitations of Acquiring Meaning from Ungrounded Form: What will Future Language Models Understand? ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Measuring and Improving Consistency in Pretrained Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
Aligning Faithful Interpretations with their Social Attribution ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Amnesic Probing: Behavioral Explanation With Amnesic Counterfactuals ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Asking It All: Generating Contextualized Questions for any Semantic Role ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ...
|
|
|
|
Abstract:
When language models process syntactically complex sentences, do they use their representations of syntax in a manner that is consistent with the grammar of the language? We propose AlterRep, an intervention-based method to address this question. For any linguistic feature of a given sentence, AlterRep generates counterfactual representations by altering how the feature is encoded, while leaving intact all other aspects of the original representation. By measuring the change in a model's word prediction behavior when these counterfactual representations are substituted for the original ones, we can draw conclusions about the causal effect of the linguistic feature in question on the model's behavior. We apply this method to study how BERT models of different sizes process relative clauses (RCs). We find that BERT variants use RC boundary information during word prediction in a manner that is consistent with the rules of English grammar; this RC boundary information generalizes to a considerable extent across ... : Equal contribution by SR and GP. Accepted in CoNLL 2021 ...
|
|
Keyword:
Computation and Language cs.CL; FOS Computer and information sciences
|
|
URL: https://arxiv.org/abs/2105.06965 https://dx.doi.org/10.48550/arxiv.2105.06965
|
|
BASE
|
|
Hide details
|
|
18 |
Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Simple, Interpretable and Stable Method for Detecting Words with Usage Change across Corpora
|
|
|
|
In: ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics ; https://hal.inria.fr/hal-03161637 ; ACL 2020 - 58th Annual Meeting of the Association for Computational Linguistics, Jul 2020, Seattle / Virtual, United States. pp.538-555, ⟨10.18653/v1/2020.acl-main.51⟩ (2020)
|
|
BASE
|
|
Show details
|
|
|
|