| 1 |
Cross-Situational Learning Towards Robot Grounding
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-03628290 ; 2022 (2022)
|
|
BASE
|
|
Show details
|
|
|
| 2 |
Cross-Situational Learning Towards Robot Grounding
|
|
|
|
In: https://hal.archives-ouvertes.fr/hal-03628290 ; 2022 (2022)
|
|
Abstract:
How do children acquire language through unsupervised or noisy supervision? How do their brain process language? We take this perspective to machine learning and robotics, where part of the problem is understanding how language models can perform grounded language acquisition through noisy supervision and discussing how they can account for brain learning dynamics. Most prior works have tracked the co-occurrence between single words and referents to model how infants learn wordreferent mappings. This paper studies cross-situational learning (CSL) with full sentences: we want to understand brain mechanisms that enable children to learn mappings between words and their meanings from full sentences in early language learning. We investigate the CSL task on a few training examples with two sequence-based models: (i) Echo State Networks (ESN) and (ii) Long-Short Term Memory Networks (LSTM). Most importantly, we explore several word representations including One-Hot, GloVe, pretrained BERT, and fine-tuned BERT representations (last layer token representations) to perform the CSL task. We apply our approach to three diverse datasets (two grounded language datasets and a robotic dataset) and observe that (1) One-Hot, GloVe, and pretrained BERT representations are less efficient when compared to representations obtained from fine-tuned BERT. (2) ESN online with final learning (FL) yields superior performance over ESN online continual learning (CL), offline learning, and LSTMs, indicating the more biological plausibility of ESNs and the cognitive process of sentence reading. (2) LSTM with fewer hidden units showcases higher performance for small datasets, but LSTM with more hidden units is Cross-Situational Learning needed to perform reasonably well on larger corpora. (4) ESNs demonstrate better generalization than LSTM models for increasingly large vocabularies. Overall, these models are able to learn from scratch to link complex relations between words and their corresponding meaning concepts, handling polysemous and synonymous words. Moreover, we argue that such models can extend to help current human-robot interaction studies on language grounding and better understand children's developmental language acquisition. We make the code publicly available * .
|
|
Keyword:
[INFO.INFO-AI]Computer Science [cs]/Artificial Intelligence [cs.AI]; [INFO.INFO-CL]Computer Science [cs]/Computation and Language [cs.CL]; [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG]; [INFO.INFO-NE]Computer Science [cs]/Neural and Evolutionary Computing [cs.NE]; [INFO.INFO-RB]Computer Science [cs]/Robotics [cs.RO]; [SDV.NEU]Life Sciences [q-bio]/Neurons and Cognition [q-bio.NC]; BERT; cross-situational learning; echo state networks; grounded language; LSTM
|
|
URL: https://hal.archives-ouvertes.fr/hal-03628290/document
https://hal.archives-ouvertes.fr/hal-03628290/file/Journal_of_Social_and_Robotics.pdf
https://hal.archives-ouvertes.fr/hal-03628290
|
|
BASE
|
|
Hide details
|
|
|
| 3 |
Why, What and How to help each Citizen to Understand Artificial Intelligence?
|
|
|
|
In: EISSN: 0933-1875 ; KI - Künstliche Intelligenz ; https://hal.inria.fr/hal-03024034 ; KI - Künstliche Intelligenz, Springer Nature, 2021, pp.1610-1987 (2021)
|
|
BASE
|
|
Show details
|
|
|
| 4 |
Formalizing Problem Solving in Computational Thinking : an Ontology approach
|
|
|
|
In: IEEE ICDL 2021 – International Conference on Development and Learning 2021 ; https://hal.inria.fr/hal-03324136 ; IEEE ICDL 2021 – International Conference on Development and Learning 2021, Aug 2021, Beijing, China (2021)
|
|
BASE
|
|
Show details
|
|
|
| 5 |
Ontology as neuronal-space manifold: Towards symbolic and numerical artificial embedding
|
|
|
|
In: KRHCAI 2021 Workshop on Knowledge Representation for Hybrid & Compositional AI @ KR2021 ; https://hal.inria.fr/hal-03360307 ; KRHCAI 2021 Workshop on Knowledge Representation for Hybrid & Compositional AI @ KR2021, Nov 2021, Hanoi, Vietnam (2021)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Ontology as neuronal-space manifold: Towards symbolic and numerical artificial embedding
|
|
|
|
In: KRHCAI 2021 Workshop on Knowledge Representation for Hybrid & Compositional AI @ KR2021 ; https://hal.inria.fr/hal-03360307 ; KRHCAI 2021 Workshop on Knowledge Representation for Hybrid & Compositional AI @ KR2021, Nov 2021, Hanoi, Vietnam (2021)
|
|
BASE
|
|
Show details
|
|
|
| 7 |
Ontology as neuronal-space manifold: Towards symbolic and numerical artificial embedding
|
|
|
|
In: KRHCAI 2021 Workshop on Knowledge Representation for Hybrid & Compositional AI @ KR2021 ; https://hal.inria.fr/hal-03360307 ; KRHCAI 2021 Workshop on Knowledge Representation for Hybrid & Compositional AI @ KR2021, Nov 2021, Hanoi, Vietnam (2021)
|
|
BASE
|
|
Show details
|
|
|
| 8 |
Learning to Parse Sentences with Cross-Situational Learning using Different Word Embeddings Towards Robot Grounding ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 9 |
Cognitive Architecture and Software Environment for the Design and Experimentation of Survival Behaviors in Artificial Agents
|
|
|
|
In: IJCCI 2018 - 10th International Joint Conference on Computational Intelligence ; https://hal.inria.fr/hal-01931497 ; IJCCI 2018 - 10th International Joint Conference on Computational Intelligence, Sep 2018, Seville, Spain (2018)
|
|
BASE
|
|
Show details
|
|
|
| 10 |
Terres neuves agricoles, terres d'élevage en sursis » : trajectoires actuelles et recomposition des espaces agropastoraux dans le Sud- Ouest nigérien
|
|
|
|
In: 5ème Rencontres des études africaines en France ; https://hal.archives-ouvertes.fr/hal-02421705 ; 5ème Rencontres des études africaines en France, Jul 2018, Marseille, France (2018)
|
|
BASE
|
|
Show details
|
|
|
| 12 |
Dynamique des ressources pastorales, perception des changements et stratégies d’adaptation des agropasteurs sahéliens : exemple de la commune de Hombori (Mali).
|
|
|
|
In: 3ème conférence internationale AMMA-France ; https://hal.archives-ouvertes.fr/hal-02421770 ; 3ème conférence internationale AMMA-France, Nov 2010, Toulouse, France (2010)
|
|
BASE
|
|
Show details
|
|
|
| 13 |
Spatio-Temporal and Complex-Valued Models based on SOM map applied to Speech Recognition
|
|
|
|
In: Twentieth International Joint Conference on Artificial Intelligence - IJCAI'2007 ; https://hal.inria.fr/inria-00118122 ; Twentieth International Joint Conference on Artificial Intelligence - IJCAI'2007, Jan 2007, Hyderabad, India (2007)
|
|
BASE
|
|
Show details
|
|
|
| 14 |
Spatio-temporal biologically inspired models for clean and noisy speech recognition
|
|
|
|
In: ISSN: 0925-2312 ; Neurocomputing ; https://hal.inria.fr/inria-00186512 ; Neurocomputing, Elsevier, 2007, 71 (1-3), pp.131--136. ⟨10.1016/j.neucom.2007.08.009⟩ (2007)
|
|
BASE
|
|
Show details
|
|
|
| 15 |
Towards Word Semantics from Multi-modal Acoustico-Motor Integration: Application of the Bijama Model to the Setting of Action-Dependant Phonetic Representations
|
|
|
|
In: Biomimetic Neural Learning for Intelligent Robots: Intelligent Systems, Cognitive Robotics, and Neuroscience ; https://hal.inria.fr/inria-00000634 ; Stefan Wermter and Günther Palm and Mark Elshaw. Biomimetic Neural Learning for Intelligent Robots: Intelligent Systems, Cognitive Robotics, and Neuroscience, 3575 (3575), Springer-Verlag, pp.144--161, 2005, Lecture Notes in Artificial Intelligence, 3-540-27440-5. ⟨10.1007/b139051⟩ (2005)
|
|
BASE
|
|
Show details
|
|
|
| 16 |
Multi-criteria self-organization: Example of motor-dependent phonetic representation for a multi-modal robot
|
|
|
|
In: Neurobotics Workshop of the 27th german conference on Artificial Intelligence ; https://hal.inria.fr/inria-00099901 ; Neurobotics Workshop of the 27th german conference on Artificial Intelligence, Sep 2004, Ulm, Germany, 12 p (2004)
|
|
BASE
|
|
Show details
|
|
|
|
|
