1 |
Parameter space factorization for zero-shot learning across tasks and languages ...
|
|
|
|
BASE
|
|
Show details
|
|
2 |
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models ...
|
|
|
|
Abstract:
Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised techniques. Inspired by this line of work, in this paper we propose a fully unsupervised approach to improving word-in-context (WiC) representations in PLMs, achieved via a simple and efficient WiC-targeted fine-tuning procedure: MirrorWiC. The proposed method leverages only raw texts sampled from Wikipedia, assuming no sense-annotated data, and learns context-aware word representations within a standard contrastive learning setup. We experiment with a series of standard and comprehensive WiC benchmarks across multiple languages. Our proposed fully unsupervised MirrorWiC models obtain substantial gains over off-the-shelf PLMs across all monolingual, multilingual and cross-lingual setups. Moreover, on some standard WiC benchmarks, MirrorWiC is even on-par with supervised models fine-tuned with in-task data and sense labels. ...
|
|
Keyword:
cs.CL
|
|
URL: https://dx.doi.org/10.17863/cam.78495 https://www.repository.cam.ac.uk/handle/1810/331050
|
|
BASE
|
|
Hide details
|
|
3 |
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Semantic Data Set Construction from Human Clustering and Spatial Arrangement ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Parameter space factorization for zero-shot learning across tasks and languages
|
|
|
|
In: Transactions of the Association for Computational Linguistics, 9 (2021)
|
|
BASE
|
|
Show details
|
|
8 |
AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples ...
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
Improving Machine Translation of Rare and Unseen Word Senses ...
|
|
|
|
BASE
|
|
Show details
|
|
11 |
LexFit: Lexical Fine-Tuning of Pretrained Language Models ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Verb Knowledge Injection for Multilingual Event Processing ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
A Closer Look at Few-Shot Crosslingual Transfer: The Choice of Shots Matters ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine. ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine. ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine ...
|
|
|
|
BASE
|
|
Show details
|
|
17 |
BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine.
|
|
|
|
In: nlmid: 101531992 ; essn: 2041-1480 (2021)
|
|
BASE
|
|
Show details
|
|
18 |
BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine.
|
|
|
|
BASE
|
|
Show details
|
|
19 |
BioVerbNet: a large semantic-syntactic classification of verbs in biomedicine
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Multi-SimLex: A Large-Scale Evaluation of Multilingual and Cross-Lingual Lexical Semantic Similarity
|
|
|
|
In: ISSN: 0891-2017 ; EISSN: 1530-9312 ; Computational Linguistics ; https://hal.archives-ouvertes.fr/hal-02975786 ; Computational Linguistics, Massachusetts Institute of Technology Press (MIT Press), 2020, 46 (4), pp.847-897 ; https://direct.mit.edu/coli/article/46/4/847/97326/Multi-SimLex-A-Large-Scale-Evaluation-of (2020)
|
|
BASE
|
|
Show details
|
|
|
|