3 |
Evaluation of Unsupervised Automatic Readability Assessors Using Rank Correlations ...
|
|
|
|
Abstract:
Automatic readability assessment (ARA) is the task of automatically assessing readability with little or no human supervision. ARA is essential for many second language acquisition applications to reduce the workload of annotators, who are usually language teachers. Previous unsupervised approaches manually searched textual features that correlated well with readability labels, such as perplexity scores of large language models. This paper argues that, to evaluate an assessors' performance, rank-correlation coefficients should be used instead of Pearson's correlation coefficient ($\rho$). In the experiments, we show that its performance can be easily underestimated using Pearson's $\rho$, which is significantly affected by the linearity of the output readability scores. We also propose a lightweight unsupervised readability assessor that achieved the best performance in both the rank correlations and Pearson's $\rho$ among all unsupervised assessors compared. ...
|
|
URL: https://underline.io/lecture/39298-evaluation-of-unsupervised-automatic-readability-assessors-using-rank-correlations https://dx.doi.org/10.48448/b4y1-p893
|
|
BASE
|
|
Hide details
|
|
4 |
Analysis of Language Change in Collaborative Instruction Following ...
|
|
|
|
BASE
|
|
Show details
|
|
5 |
Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora ...
|
|
|
|
BASE
|
|
Show details
|
|
6 |
Cross-lingual Aspect-based Sentiment Analysis with Aspect Term Code-Switching ...
|
|
|
|
BASE
|
|
Show details
|
|
7 |
Cross-lingual Transfer for Text Classification with Dictionary-based Heterogeneous Graph ...
|
|
|
|
BASE
|
|
Show details
|
|
8 |
NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset ...
|
|
|
|
BASE
|
|
Show details
|
|
10 |
An Unsupervised Method for Building Sentence Simplification Corpora in Multiple Languages ...
|
|
|
|
BASE
|
|
Show details
|
|
12 |
SD-QA: Spoken Dialectal Question Answering for the Real World ...
|
|
|
|
BASE
|
|
Show details
|
|
13 |
Plan-then-Generate: Controlled Data-to-Text Generation via Planning ...
|
|
|
|
BASE
|
|
Show details
|
|
14 |
Sparsity and Sentence Structure in Encoder-Decoder Attention of Summarization Systems ...
|
|
|
|
BASE
|
|
Show details
|
|
15 |
Identity-Based Patterns in Deep Convolutional Networks: Generative Adversarial Phonology and Reduplication ...
|
|
|
|
BASE
|
|
Show details
|
|
16 |
Live Session - 4E: Phonology, Morphology and Word Segmentation ...
|
|
|
|
BASE
|
|
Show details
|
|
19 |
Rule-based Morphological Inflection Improves Neural Terminology Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
20 |
Translating Headers of Tabular Data: A Pilot Study of Schema Translation ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|