| 1 |
Re-envisioning assessment of interlanguage pragmatic competence through computer-mediated communicative tasks
|
|
|
|
Abstract:
Assessment of second language (L2) pragmatic knowledge is still a new and understudied area of research. Some researchers (Hudson, Detmer, & Brown, 1992, 1995; Roever, 2006, Walters, 2007) have played an important role in advancing the field, but their theories have followed speech act framework often criticized for pragmatic construct under-representation (Grabowski, 2009; Roever, 2011; Youn, 2015). Methodologically, past studies mainly used closed role-plays based on predetermined interactional outcomes (Youn, 2015). Kasper and Rose (2002) doubted the validity and authenticity of closed role-play tasks. In order to address these research gaps, and as a contribution to the general understanding of Second Language Pragmatic Testing (SLPT), this study combines second language pragmatics and computer-mediated communication to assess the pragmatic knowledge of second language users of English. It uses Purpura’s (2004) framework of communicative language ability for developing interactive, email-based role-play tasks to assess test takers’ pragmatic ability with regard to sociolinguistic, sociocultural and psychological meanings, and the use of polite formulaic expressions. Using mixed methods (Greene, 2007), qualitative and quantitative evidence was provided to support test inferences. A thorough needs analysis was conducted first through semi-structured interviews and then through an online survey by involving different ESL stakeholders including 153 faculty members at a large Mid-Western university. The online questionnaire results provided insight into the instructors’ perceptions of different email situations. The results of the needs analysis also helped to determine appropriate role-play situations. A set of communicative role-play tasks were developed following Davidson and Lynch’s (2002) test specification theory. Role-play cards were used to enhance standardization, and test takers were allowed to communicate naturally without following fixed interactional outcomes. A group of 52 graduate ESL students completed email role-play tasks. Two native-speaking raters evaluated the pragmatic ability of test takers and assigned scores using an empirically driven analytical scoring rubric on the email threads. Given that most students scored high, there appears to be a correlation between the ESL proficiency level of students and their scores in the pragmatic ability tests. Furthermore, inter-rater reliability analysis shows an overall high inter-rater reliability (0.85). There was some agreement between the hypothesized task difficulty typology and actual scores of three ability groups assessed in the present study. Qualitative analysis of interactive email data revealed a lack of knowledge of norms (as expected in Midwestern US academic settings) of appropriateness and politeness by the lower proficiency groups. Therefore, the low stakes test might have a great potential for developing instructional materials in an academic email communication context. Based on the findings of the present study, suggestions on inclusion of sociopragmatic competence into the ESL writing curriculum are made. Systematic curricular inclusion of email pragmatics in ESL courses will assist ESL learners in developing their email pragmatic competence in academic settings.
|
|
Keyword:
Communicative; Computer-Mediated; ESL; Interactive; Interlanguage Pragmatics; Language Assessment; Testing
|
|
URL: http://hdl.handle.net/2142/105126
|
|
BASE
|
|
Hide details
|
|
|
| 5 |
Perspectives of language assessment training for teachers and testing professionals
|
|
|
|
BASE
|
|
Show details
|
|
|
| 11 |
Testing the Measurement Invariance of Large-Scale Assessment of Foreign Language Proficiency Using Multiple Group Covariance Structure Analyses and Item Response Modeling
|
|
|
|
BASE
|
|
Show details
|
|
|
| 16 |
Collaboration in Language Testing: Investigating the Validity of a New Rating Procedure for EFL Writing Assessment in Korea
|
|
|
|
BASE
|
|
Show details
|
|
|
| 18 |
History of the Baccalaureat: A Study of the Interaction Between Educational Legislation, Government Policy, and Language Theory in the National Language Examination
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
