The Challenge of Assessing Content and Coherence

Cvetka Sokolov

doi:10.4312/vestnik.14.275-292

Avtorji

Cvetka Sokolov Univerza v Ljubljani, Filozofska fakulteta

DOI:

https://doi.org/10.4312/vestnik.14.275-292

Ključne besede:

ocenjevanje pisanja, analitično ocenjevanje, vsebina in koherenca, ocenjevalna merila, opisniki

Povzetek

Pri analitičnem ocenjevanju pisnih sestavkov je ocenjevanje vsebine in koherence velik izziv. Bralci besedila namreč interpretirajo po svoje, v skladu s svojim znanjem, izkušnjami, vrednostnim sistemom in drugimi vidiki svoje osebnosti, ki povzročajo pristranskost, pa naj se ocenjevalci še tako trudijo, da bi pisne izdelke vrednotili objektivno. Pod vplivom navedenih dejavnikov berejo tudi opisnike, ki so sami po sebi niz kratkih besedil. Enotnost razumevanja opisnikov otežujeta njihova splošnost in nedoslednost pri njihovem uvrščanju v ustrezne ocenjevalne kategorije, ki sta delno posledica narave pisanja, delno pa neusklajenosti strokovnjakov na raziskovalnih področjih diskurza, teorije pisanja in ocenjevanja pisanja. Tako so ocenjevalci, ki iščejo ustrezna merila za ocenjevanje pisnih sestavkov, soočeni s pojmom, kot je na primer “relevantnost/primernost utemeljitev/dokazov”, ki je v nekaterih merilih za ocenjevaje pisnih sestavkov uvrščen v ocenjevalno kategorijo vsebine, v drugih pa v ocenjevalno kategorijo koherence. Kljub temu je veljavnost in zanesljivost ocenjevanja mogoče izboljšati. Članek se posveča odnosu med vsebino in koherenco, ki ga odražajo definicije obeh pojmov v relevantnih virih in opisniki v dveh sklopih meril za ocenjevane pisnih sestavkov v rabi v Sloveniji. Empirični del vsebuje študijo primera, ki vključuje 46 učiteljic in učiteljev na slovenskih srednjih šolah, katerih odzivi na vprašalnik potrjujejo subjektivno razumevanje opisnikov and potrebo po izobraževanju učiteljic in učiteljev na področju uporabe analitičnih ocenjevalnih meril, potrebo po redni standardizaciji ocenjevanja na šoli, kjer poučujejo, in po razmisleku, ali bi bilo opisnike v rabi priporočljivo izboljšati, na primer prilagoditi in/ali dopolniti.

Prenosi

Podatki o prenosih še niso na voljo.

Literatura

ALARO, Abebayehu Anjulo (2020) An Assessment of Cohesion and Coherence in Students’ Descriptive and Narrative Essays. Journal of Literature, Languages and Linguistics 64, 41–46.

BACHA, Nahla (2001) Writing evaluation: what can analytic versus holistic essay scoring tell us? System 29, 371–383. DOI:10.1016/S0346-251X(01)00025-2. DOI: https://doi.org/10.1016/S0346-251X(01)00025-2

BAE, Jungok (2001) Cohesion and Coherence in Children’s Written English: Immersion and English-Only Classes. Issues in Applied Linguistics 12 (1), 51–88. https://doi.org/10.5070/L4121005043. DOI: https://doi.org/10.5070/L4121005043

BAŠ, Ivica/Saša BENULIČ/Margaret DALRYMPLE/Vineta ERŽEN/Soča FIDLER,/Majda GRABAR/Meta GROSMAN/Aleša JUVANC/Smiljana KOMAR/Cvetka SOKOLOV/ Rastislav ŠUŠTARŠIČ (1996) Angleščina pri maturi: Kako se uspešno pripravimo na preizkus znanja iz angleškega jezika. Ljubljana: Državni izpitni center.

BEAN, John C. (2011) Engaging Ideas: The Professor’s Guide to Integrating Writing, Critical Thinking and Active Learning in the Classroom. 2nd edn. San Francisco: Jossey-Bass.

BRIESMASTER, Mark/Paulo ETCHEGARAY (2017) Coherence and cohesion in EFL students’ writing production: The impact of a metacognition-based intervention. Íkala, Revista de Lenguaje y Cultura 22 (2), 183 –202. DOI: 10.17533/udea.ikala.v22n02a02. DOI: https://doi.org/10.17533/udea.ikala.v22n02a02

BROWN, H. Douglas (2004) Language Assessment: Principles and Classroom Practices. New York: Longman.

BROWN, H. Douglas/Priyanvada ABEYWICKRAMA (2019) Language Assessment: Principles and Classroom Practices. 3rd edn. Hoboken: Pearson Education ESL.

BROWN, Gillian/George YULE (1983) Discourse Analysis. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511805226

CARREL, Patricia L (1982) Cohesion is not coherence. TESOL Quarterly 16 (4), 479–488. https://doi.org/10.2307/3586466. DOI: https://doi.org/10.2307/3586466

CARTER, Ronald (1993) Introducing Applied Linguistics. Harmondsworth, Middlesex: Penguin.

CUSHING WEIGLE, Sara (1998) Using FACETS to model rater training effects. Language Testing 15, 263–287. https://doi.org/10.1177/026553229801500205. DOI: https://doi.org/10.1177/026553229801500205

CUSHING WEIGLE, Sara (2010) Assessing Writing. Cambridge: Cambridge University Press.

DOUGLAS, Yellowlees (2015) The Reader’s Brain: How Neuroscience Can Make You a Better Writer. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9781316178942

ECKES, Thomas (2008) Rater types in writing performance assessments: A classification approach to rater variability. Language Testing 25 (2), 155–185. https://doi.org/10.1177/0265532207086780. DOI: https://doi.org/10.1177/0265532207086780

ELBOW, Peter (1996) Writing Assessment: Do It Better, Do It Less. E. M. White/W. D. Lutz/S. Kamusikiri (eds.), Assessment of Writing: Politics, Policies, Practices. New York: The Modern Language Association of America, 120–134.

ENKVIST, Nils Erik (1990) Seven Problems in the Study of Coherence and Interpretability. U. Connor/A. M. Johns (eds.), Coherence in Writing: Research and Pedagogical Perspectives. Alexandria: TESOL, 11–28.

GLENN, Cheryl/Melissa A. GOLDTHWAITE (2014) The St. Martin’s Guide to Teaching Writing. 3rd edn. Boston/New York: St. Martin’s.

HALLIDAY, M. A. K./Ruqaiya HASAN (1976) Cohesion in English. London/New York: Longman.

HOEY, Michael (1991) Patterns of Lexis in Text. Oxford: Oxford University Press.

HOLDSTEIN, Deborah H. (1996) Gender, Feminism, and Institution-Wide Assessment Programs. E. M. White/W, D. Lutz/S. Kamusikiri (eds.), Assessment of Writing: Politics, Policies, Practices. New York: The Modern Language Association of America, 204–225.

HUGHES, Arthur (1989/2003) Testing for Language Teachers. 2nd edn. Cambridge: Cambridge University Press.

HYLAND, Ken (2002) Teaching and Researching Writing. Harlow/London/New York: Longman.

HYLAND, Ken (2003) Second Language Writing. Cambridge: Cambridge University Press. DOI: https://doi.org/10.1017/CBO9780511667251

IB (1992) Workshops in English B for New Teachers: Working Materials. Cardiff: International Baccalaureate Europe.

ILC, Gašper/Alenka KETIŠ/Aleksandra KOMADINA/Ana LIKAR/Simona MEGLIČ/Irena ZORKO NOVAK (2018) Angleščina: Predmetni izpitni katalog za splošno maturo. Ljubljana: Državni izpitni center.

JACOBS, Holly L./Stephen A. ZINKGRAF/Deanna R. WORMUTH/V. Faye HARTFIEL/ Jane B. HUGHEY (1981) Testing ESL composition: A practical approach. Rowley, MA: Newbury House.

KNOCH, Ute (2007) ‘Little coherence, considerable strain for reader’: A comparison between two rating scales for the assessment of coherence. Assessing Writing 12 (2), 108–128. https://doi.org/10.1016/j.asw.2007.07.002. DOI: https://doi.org/10.1016/j.asw.2007.07.002

McNAMARA, Tim (2000) Language Testing. Oxford: Oxford University Press.

NUNAN, David (1993) Introducing Discourse Analysis. Harmondsworth: Penguin.

PARKS, A. Franklin/James A. LEVERNIER/Ida MASTERS HOLLOWELL (1991) Structuring Paragraphs: A Guide to Effective Writing. 3rd edn. New York: St. Martin’s Press.

RICHARDS, Jack C./John TALBOT PLATT/Heidi PLATT (1992) Dictionary of Language Teaching and Applied Linguistics. 2nd edn. Harlow: Longman.

RICHARDS, Keith (2011) Case Study. E. Hinkel (ed.), Handbook of Research in Second Language Teaching and Learning. 2nd edn. New York/London: Routledge, 207–221.

SHAW, Stuart D./Cyril Weir (2007) Examining Writing. Cambridge: Cambridge University Press.

SKOUFAKI, Sophia (2020) Rhetorical Structure Theory and coherence break identification. Text & Talk 40 (1), 99–124. https://doi-org.nukweb.nuk.uni-lj.si/10.1515/text-2019-2050. DOI: https://doi.org/10.1515/text-2019-2050

SOKOLOV, Cvetka (1999) Pisni sestavek pri študentih angleščine (MA Thesis). Ljubljana: Filozofska fakulteta.

SOKOLOV, Cvetka (2000) Pisna zmožnost – motnje v koherenci. In: Meta GROSMAN (ed.) Angleščina – prenovi na pot. Ljubljana: Zavod RS za šolstvo, 97–136.

SOKOLOV, Cvetka (2013) Pomen standardizacije ocenjevanja pisni sestavkov pri poučevanju angleščine kot tujega jezika/The Role of Standardization in Assessing Writing in Teaching English as a Foreign Language (PhD Thesis). Ljubljana: Filozofska fakulteta.

SOKOLOV, Cvetka et al. (2014) Written Composition: Analytic Scoring Scales. Unpublished. Ljubljana: Filozofska fakulteta.

SOKOLOV, Cvetka (2018) Analiza zgledov meril za razčlenjevalno ocenjevanje raz-

lagalnih in utemeljevalnih pisnih sestavkov. Vestnik 10 (1), 169–186. DOI: 10.4312/vestnik.10.169-186. DOI: https://doi.org/10.4312/vestnik.10.169-186

SPANDEL, Vicki/Richard J. STIGGINS (1990) Creating Writers: Linking Assessment and Writing Instruction. New York/London: Longman.

VOGRINC, Janez (2008) Kvalitativno raziskovanje na pedagoškem področju. Ljubljana: Pedagoška fakulteta.

WEIR, Cyril/Jon ROBERTS (1994) Evaluation in ELT. Oxford UK/Cambridge USA: Blackwell.

WHITE, Edward M (1996) Power and Agenda Setting in Writing Assessment. E. M.

WHITE, W. D. Lutz/S. Kamusikiri (eds.), Assessment of Writing: Politics, Policies, Practices. New York: The Modern Language Association of America, 9–24.

WIDDOWSON, Henry (1978) Teaching Language as Communication. Oxford: Oxford University Press.

WIKBORG, Eleanor (1990) Types of Coherence Breaks in Swedish Student Writing: Misleading Paragraph Division. U. Connor /A. M. Johns (eds.), Coherence in Writing: Research and Pedagogical Perspectives. Alexandria: TESOL, 133–148.

WILSON, Gerald L./Michael S. HANNA (1993) Groups in Context: Leadership and Participation in Small Groups. 3rd edn. New York: McGraw-Hill.

YU, Guoxing (2007) Student’s Voices in the Evaluation of Written Summaries. Language Testing 24 (4), 239–572. https://doi.org/10.1177/0265532207080780. DOI: https://doi.org/10.1177/0265532207080780