DE eng

Search in the Catalogues and Directories

Hits 1 – 10 of 10

1
Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity ...
Abstract: This paper analyzes three formal models of Transformer encoders that differ in the form of their self-attention mechanism: unique hard attention (UHAT); generalized unique hard attention (GUHAT), which generalizes UHAT; and averaging hard attention (AHAT). We show that UHAT and GUHAT Transformers, viewed as string acceptors, can only recognize formal languages in the complexity class AC$^0$, the class of languages recognizable by families of Boolean circuits of constant depth and polynomial size. This upper bound subsumes Hahn's (2020) results that GUHAT cannot recognize the DYCK languages or the PARITY language, since those languages are outside AC$^0$ (Furst et al., 1984). In contrast, the non-AC$^0$ languages MAJORITY and DYCK-1 are recognizable by AHAT networks, implying that AHAT can recognize languages that UHAT and GUHAT cannot. ... : To appear in Transactions of the Association for Computational Linguistics ...
Keyword: Artificial Intelligence cs.AI; Computation and Language cs.CL; Computational Complexity cs.CC; Formal Languages and Automata Theory cs.FL; FOS Computer and information sciences; Machine Learning cs.LG
URL: https://arxiv.org/abs/2204.06618
https://dx.doi.org/10.48550/arxiv.2204.06618
BASE
Hide details
2
An Adversarial Benchmark for Fake News Detection Models ...
BASE
Show details
3
Attribution Analysis of Grammatical Dependencies in LSTMs ...
Hao, Yiding. - : arXiv, 2020
BASE
Show details
4
Probabilistic Predictions of People Perusing: Evaluating Metrics of Language Model Performance for Psycholinguistic Modeling ...
BASE
Show details
5
Rhythmic Syncope in Subregular Phonology
In: University of Pennsylvania Working Papers in Linguistics (2020)
BASE
Show details
6
Metrical Grids and Generalized Tier Projection
In: Proceedings of the Society for Computation in Linguistics (2020)
BASE
Show details
7
Computing Vowel Harmony: The Generative Capacity of Search & Copy
In: Proceedings of the Annual Meetings on Phonology; Proceedings of the 2019 Annual Meeting on Phonology ; 2377-3324 (2020)
BASE
Show details
8
Action-Sensitive Phonological Dependencies ...
Hao, Yiding; Bowers, Dustin. - : arXiv, 2019
BASE
Show details
9
Finding Syntactic Representations in Neural Stacks ...
BASE
Show details
10
Learnability and Overgeneration in Computational Syntax
In: Proceedings of the Society for Computation in Linguistics (2019)
BASE
Show details

Catalogues
0
0
0
0
0
0
0
Bibliographies
0
0
0
0
0
0
0
0
0
Linked Open Data catalogues
0
Online resources
0
0
0
0
Open access documents
10
0
0
0
0
© 2013 - 2024 Lin|gu|is|tik | Imprint | Privacy Policy | Datenschutzeinstellungen ändern