1 |
GatorTron: A Large Clinical Language Model to Unlock Patient Information from Unstructured Electronic Health Records ...
|
|
Yang, Xi; PourNejatian, Nima; Shin, Hoo Chang; Smith, Kaleb E; Parisien, Christopher; Compas, Colin; Martin, Cheryl; Flores, Mona G; Zhang, Ying; Magoc, Tanja; Harle, Christopher A; Lipori, Gloria; Mitchell, Duane A; Hogan, William R; Shenkman, Elizabeth A; Bian, Jiang; Wu, Yonghui. - : arXiv, 2022
|
|
Abstract:
Objective: To develop a large pretrained clinical language model from scratch using transformer architecture; systematically examine how transformer models of different sizes could help 5 clinical natural language processing (NLP) tasks at different linguistic levels. Methods: We created a large corpus with >90 billion words from clinical narratives (>82 billion words), scientific literature (6 billion words), and general English text (2.5 billion words). We developed GatorTron models from scratch using the BERT architecture of different sizes including 345 million, 3.9 billion, and 8.9 billion parameters, compared GatorTron with three existing transformer models in the clinical and biomedical domain on 5 different clinical NLP tasks including clinical concept extraction, relation extraction, semantic textual similarity, natural language inference, and medical question answering, to examine how large transformer models could help clinical NLP at different linguistic levels. Results and Conclusion: ... : 24 pages, 2 figures, 3 tables ...
|
|
Keyword:
Artificial Intelligence cs.AI; Computation and Language cs.CL; FOS Computer and information sciences; Machine Learning cs.LG
|
|
URL: https://dx.doi.org/10.48550/arxiv.2203.03540 https://arxiv.org/abs/2203.03540
|
|
BASE
|
|
Hide details
|
|
2 |
Tracing Text Provenance via Context-Aware Lexical Substitution ...
|
|
|
|
BASE
|
|
Show details
|
|
3 |
Assessing mental health signals among sexual and gender minorities using Twitter data ...
|
|
|
|
BASE
|
|
Show details
|
|
4 |
Assessing mental health signals among sexual and gender minorities using Twitter data ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
A Study of Deep Learning Methods for De-identification of Clinical Notes at Cross Institute Settings
|
|
|
|
BASE
|
|
Show details
|
|
6 |
A study of deep learning methods for de-identification of clinical notes in cross-institute settings
|
|
|
|
BASE
|
|
Show details
|
|
7 |
MADEx: A System for Detecting Medications, Adverse Drug Events, and their Relations from Clinical Notes
|
|
|
|
BASE
|
|
Show details
|
|
8 |
Rome Foundation-Asian working team report: Asian functional gastrointestinal disorder symptom clusters
|
|
|
|
BASE
|
|
Show details
|
|
9 |
Assessing Mental Health Signals among Sexual and Gender Minorities using Twitter Data
|
|
|
|
BASE
|
|
Show details
|
|
10 |
An Analytical Study of Not-negation and No-negation Translated in the Chinese Version of the Fantasy Fiction The Hobbit
|
|
Yang, Xi. - : The University of Queensland, School of Languages and Cultures, 2018
|
|
BASE
|
|
Show details
|
|
11 |
Potential screening and early diagnosis method for cancer: Tongue diagnosis
|
|
|
|
BASE
|
|
Show details
|
|
12 |
Mirror neuron system based therapy for aphasia rehabilitation
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Research On The Implications Of Business English Teaching On Bilingual Courses In Business Communication.
|
|
|
|
BASE
|
|
Show details
|
|
|
|