The Challenge of Assessing Content and Coherence


  • Cvetka Sokolov University of Ljubljana, Faculty of Arts, Slovenia



writing assessment, analytic scoring, content and coherence, scoring scales, descriptors


Content and coherence are the categories most difficult to evaluate fairly when raters use analytic scoring scales. Readers inevitably interpret texts in their own idiosyncratic ways, depending on their knowledge, experience, ethical considerations, and other personal biases that they cannot completely set aside when grading a text. This is also true for descriptors, which are themselves short texts. To make matters worse, due to the very nature of writing but also the lack of consensus among experts in discourse research, writing theory, and writing assessment, descriptors are categorized vaguely and inconsistently. As a result, raters seeking useful evaluation criteria are confronted with descriptors that cover the same concept, such as “relevance”, being categorized in one set of criteria as relating to the content of the written text and in another as belonging to the category of coherence. Nevertheless, the objectivity of the evaluation of written work can be increased. The article examines the relationship between content and coherence, which is reflected in the way the two concepts are defined in the relevant literature, as well as in some descriptors used in two grading scales used in Slovenia. The empirical part of the paper presents a case study involving 46 secondary school teachers, whose responses to a questionnaire confirm the subjectivity of the understanding of individual descriptors and the need for adequate training of teachers in the use of analytic scoring scales, regular standardization in the schools where they work, evaluation of the assessment scales they use and their possible adaptation.


Download data is not yet available.


ALARO, Abebayehu Anjulo (2020) An Assessment of Cohesion and Coherence in Students’ Descriptive and Narrative Essays. Journal of Literature, Languages and Linguistics 64, 41–46.

BACHA, Nahla (2001) Writing evaluation: what can analytic versus holistic essay scoring tell us? System 29, 371–383. DOI:10.1016/S0346-251X(01)00025-2. DOI:

BAE, Jungok (2001) Cohesion and Coherence in Children’s Written English: Immersion and English-Only Classes. Issues in Applied Linguistics 12 (1), 51–88. DOI:

BAŠ, Ivica/Saša BENULIČ/Margaret DALRYMPLE/Vineta ERŽEN/Soča FIDLER,/Majda GRABAR/Meta GROSMAN/Aleša JUVANC/Smiljana KOMAR/Cvetka SOKOLOV/ Rastislav ŠUŠTARŠIČ (1996) Angleščina pri maturi: Kako se uspešno pripravimo na preizkus znanja iz angleškega jezika. Ljubljana: Državni izpitni center.

BEAN, John C. (2011) Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking and Active Learning in the Classroom. 2nd edn. San Francisco: Jossey-Bass.

BRIESMASTER, Mark/Paulo ETCHEGARAY (2017) Coherence and cohesion in EFL students’ writing production: The impact of a metacognition-based intervention. Íkala, Revista de Lenguaje y Cultura 22 (2), 183 –202. DOI: 10.17533/udea.ikala.v22n02a02. DOI:

BROWN, H. Douglas (2004) Language Assessment: Principles and Classroom Practices. New York: Longman.

BROWN, H. Douglas/Priyanvada ABEYWICKRAMA (2019) Language Assessment: Principles and Classroom Practices. 3rd edn. Hoboken: Pearson Education ESL.

BROWN, Gillian/George YULE (1983) Discourse Analysis. Cambridge: Cambridge University Press. DOI:

CARREL, Patricia L (1982) Cohesion is not coherence. TESOL Quarterly 16 (4), 479–488. DOI:

CARTER, Ronald (1993) Introducing Applied Linguistics. Harmondsworth, Middlesex: Penguin.

CUSHING WEIGLE, Sara (1998) Using FACETS to model rater training effects. Language Testing 15, 263–287. DOI:

CUSHING WEIGLE, Sara (2010) Assessing Writing. Cambridge: Cambridge University Press.

DOUGLAS, Yellowlees (2015) The Reader’s Brain: How Neuroscience Can Make You a Better Writer. Cambridge: Cambridge University Press. DOI:

ECKES, Thomas (2008) Rater types in writing performance assessments: A classification approach to rater variability. Language Testing 25 (2), 155–185. DOI:

ELBOW, Peter (1996) Writing Assessment: Do It Better, Do It Less. E. M. White/W. D. Lutz/S. Kamusikiri (eds.), Assessment of Writing: Politics, Policies, Practices. New York: The Modern Language Association of America, 120–134.

ENKVIST, Nils Erik (1990) Seven Problems in the Study of Coherence and Interpretability. U. Connor/A. M. Johns (eds.), Coherence in Writing: Research and Pedagogical Perspectives. Alexandria: TESOL, 11–28.

GLENN, Cheryl/Melissa A. GOLDTHWAITE (2014) The St. Martin’s Guide to Teaching Writing. 3rd edn. Boston/New York: St. Martin’s.

HALLIDAY, M. A. K./Ruqaiya HASAN (1976) Cohesion in English. London/New York: Longman.

HOEY, Michael (1991) Patterns of Lexis in Text. Oxford: Oxford University Press.

HOLDSTEIN, Deborah H. (1996) Gender, Feminism, and Institution-Wide Assessment Programs. E. M. White/W, D. Lutz/S. Kamusikiri (eds.), Assessment of Writing: Politics, Policies, Practices. New York: The Modern Language Association of America, 204–225.

HUGHES, Arthur (1989/2003) Testing for Language Teachers. 2nd edn. Cambridge: Cambridge University Press.

HYLAND, Ken (2002) Teaching and Researching Writing. Harlow/London/New York: Longman.

HYLAND, Ken (2003) Second Language Writing. Cambridge: Cambridge University Press. DOI:

IB (1992) Workshops in English B for New Teachers: Working Materials. Cardiff: International Baccalaureate Europe.

ILC, Gašper/Alenka KETIŠ/Aleksandra KOMADINA/Ana LIKAR/Simona MEGLIČ/Irena ZORKO NOVAK (2018) Angleščina: Predmetni izpitni katalog za splošno maturo. Ljubljana: Državni izpitni center.

JACOBS, Holly L./Stephen A. ZINKGRAF/Deanna R. WORMUTH/V. Faye HARTFIEL/ Jane B. HUGHEY (1981) Testing ESL composition: A practical approach. Rowley, MA: Newbury House.

KNOCH, Ute (2007) ‘Little coherence, considerable strain for reader’: A comparison between two rating scales for the assessment of coherence. Assessing Writing 12 (2), 108–128. DOI:

McNAMARA, Tim (2000) Language Testing. Oxford: Oxford University Press.

NUNAN, David (1993) Introducing Discourse Analysis. Harmondsworth: Penguin.

PARKS, A. Franklin/James A. LEVERNIER/Ida MASTERS HOLLOWELL (1991) Structuring Paragraphs: A Guide to Effective Writing. 3rd edn. New York: St. Martin’s Press.

RICHARDS, Jack C./John TALBOT PLATT/Heidi PLATT (1992) Dictionary of Language Teaching and Applied Linguistics. 2nd edn. Harlow: Longman.

RICHARDS, Keith (2011) Case Study. E. Hinkel (ed.), Handbook of Research in Second Language Teaching and Learning. 2nd edn. New York/London: Routledge, 207–221.

SHAW, Stuart D./Cyril Weir (2007) Examining Writing. Cambridge: Cambridge University Press.

SKOUFAKI, Sophia (2020) Rhetorical Structure Theory and coherence break identification. Text & Talk 40 (1), 99–124. DOI:

SOKOLOV, Cvetka (1999) Pisni sestavek pri študentih angleščine (MA Thesis). Ljubljana: Filozofska fakulteta.

SOKOLOV, Cvetka (2000) Pisna zmožnost – motnje v koherenci. In: Meta GROSMAN (ed.) Angleščina – prenovi na pot. Ljubljana: Zavod RS za šolstvo, 97–136.

SOKOLOV, Cvetka (2013) Pomen standardizacije ocenjevanja pisni sestavkov pri poučevanju angleščine kot tujega jezika/The Role of Standardization in Assessing Writing in Teaching English as a Foreign Language (PhD Thesis). Ljubljana: Filozofska fakulteta.

SOKOLOV, Cvetka et al. (2014) Written Composition: Analytic Scoring Scales. Unpublished. Ljubljana: Filozofska fakulteta.

SOKOLOV, Cvetka (2018) Analiza zgledov meril za razčlenjevalno ocenjevanje raz-

lagalnih in utemeljevalnih pisnih sestavkov. Vestnik 10 (1), 169–186. DOI: 10.4312/vestnik.10.169-186. DOI:

SPANDEL, Vicki/Richard J. STIGGINS (1990) Creating Writers: Linking Assessment and Writing Instruction. New York/London: Longman.

VOGRINC, Janez (2008) Kvalitativno raziskovanje na pedagoškem področju. Ljubljana: Pedagoška fakulteta.

WEIR, Cyril/Jon ROBERTS (1994) Evaluation in ELT. Oxford UK/Cambridge USA: Blackwell.

WHITE, Edward M (1996) Power and Agenda Setting in Writing Assessment. E. M.

WHITE, W. D. Lutz/S. Kamusikiri (eds.), Assessment of Writing: Politics, Policies, Practices. New York: The Modern Language Association of America, 9–24.

WIDDOWSON, Henry (1978) Teaching Language as Communication. Oxford: Oxford University Press.

WIKBORG, Eleanor (1990) Types of Coherence Breaks in Swedish Student Writing: Misleading Paragraph Division. U. Connor /A. M. Johns (eds.), Coherence in Writing: Research and Pedagogical Perspectives. Alexandria: TESOL, 133–148.

WILSON, Gerald L./Michael S. HANNA (1993) Groups in Context: Leadership and Participation in Small Groups. 3rd edn. New York: McGraw-Hill.

YU, Guoxing (2007) Student’s Voices in the Evaluation of Written Summaries. Language Testing 24 (4), 239–572. DOI:




How to Cite

Sokolov, C. (2022). The Challenge of Assessing Content and Coherence. Journal for Foreign Languages, 14(1), 275–292.