Does Lexical Frequency affect rater judgement of essays? An experimental design using quantitative and qualitative data
Abstract
Many correlational studies show a positive relation between written assessments of language and use of more diverse vocabulary (Lexical Diversity) and more infrequent words (Lexical Frequency). However, there have been no experimental studies that have isolated the effects of Lexical Frequency from Lexical Diversity. In the present study, 14 raters judged two versions of the same essay that differed only in Lexical Frequency. A Paired T-test showed no difference in mean scores between essays (t(13) = .396, p = .70) when the Lexical Frequency of 23.5% of Content Words were changed in a 347 word essay. Comments explaining scores given to essays showed that features other than vocabulary had a far greater influence on rater judgement. It is possible that the Lexical Frequency manipulations were not great enough to affect rater judgement, whether subliminal or conscious. Implications of these results for standardized language proficiency tests and future research in vocabulary are discussed.