Appraisal Analysis and AI Chatbots: Do We Even Need Humans?

Agata Križan; Aja Barbič

doi:10.4312/elope.22.1.35-52

Authors

Agata Križan University of Maribor
Aja Barbič University of Maribor

DOI:

https://doi.org/10.4312/elope.22.1.35-52

Keywords:

appraisal, human and AI comparative analysis, ChatGPT, MS Copilot, Claude

Abstract

Artificial intelligence (AI) is rapidly transforming various fields, including linguistics, by offering new tools for the analysis and generation of human language. As AI tools, particularly chatbots, have become increasingly sophisticated, questions have arisen about their capacity to replicate complex human linguistic processes, such as those covered by the appraisal framework developed by Martin and White (2005). The appraisal framework examines how three main categories – attitude, graduation, and engagement – are expressed in discourse at the semantic level. This paper investigates how AI chatbots, MS Copilot, ChatGPT, and Claude approach appraisal analysis in a selected text, highlighting similarities and notable differences in comparison to human analysis. The findings, although based on analysis of a single text, provide valuable insights into the advantages and drawbacks of AI in mimicking human-like appraisal analysis, which might be beneficial when conducting appraisal research.

Metrics

Metrics Loading ...

Downloads

Download data is not yet available.

References

Crosthwaite, Peter, and Vit Baisa. 2023. “Generative AI and the end of corpus-assisted data-driven learning? Not so fast!” Applied Corpus Linguistics 3 (3): 100066. https://doi.org/10.1016/j.acorp.2023 .100066.

Curry, Niall, Paul Baker, and Gavin Brookes. 2024. “Generative AI for corpus approaches to discourse studies: A critical evaluation of ChatGPT.” Applied Corpus Linguistics 4 (1): 100082. https://doi.org/10.1016/j.acorp.2023.100082.

deVore, Nicholas. 1949. “The urges and the emotions”. In New Frontiers of Psychology, by Nicholas deVore, 42–53. Philosophical Library. https://doi.org/10.1037/13248-006.

Halliday, Michael A.K., and Christian M.I. Matthiessen. 2004. An Introduction to Functional Grammar. Hodder Arnold.

Hazemali, David, Janez Osojnik, Tomaž Onič, Tadej Todorović, and Mladen Borovič. 2024. “Evaluating chatbot assistance in historical document analysis.” Moderna arhivistika 7 (2): 53–83. https://doi.org/10.54356/ma/2024/biub3010.

Hood, Susan. 2004. “Appraising research: Taking a stance in academic writing.” PhD diss., University of Technology, Sydney. http://www.grammatics.com/appraisal/suehoodphd/hood_title_page.pdf.

Hunston, Susan, and Geoff Thompson, eds. 1999. Evaluation in Text: Authorial Stance and the Construction of Discourse. Oxford University Press.

Imamović, Mirela, Silvana Deilen, Ekaterina Lapshinova-Koltunski, and Dylan Glynn. 2024. “Using ChatGPT for annotation of attitude within the appraisal theory: Lessons learned.” In Proceedings of the 18th Linguistic Annotation Workshop (LAW-XVIII), edited by Sophie Henning and Manfred Stede, 112–23. Association for Computational Linguistics.

Koeva, Svetla. 2024. “Large language models in linguistic research: The Pilot and the Copilot.” In Proceedings of the Sixth International Conference on Computational Linguistics in Bulgaria (CLIB 2024), 319–28. Department of Computational Linguistics, Institute for Bulgarian Language, Bulgarian Academy of Sciences. https://aclanthology.org/2024.clib-1.35/.

Kress, Gunther. 1988. Communication and Culture: An Introduction. New South Wales University Press.

Križan, Agata. 2016. “The Language of appraisal in British advertisements: The construal of attitudinal judgement.” ELOPE: English Language Overseas Perspectives and Enquiries 13 (2): 199–220. https://doi.org/10.4312/elope.13.1.15-29.

Leech, Geoffrey. 1993. “Corpus annotation schemes.” Literary and Linguistic Computing 8 (4): 275–81.

Lozić, Edisa, and Benjamin Štular. 2023. “Fluent but not factual: A comparative analysis of ChatGPT and other AI chatbots’ proficiency and originality in scientific writing for humanities.” Future Internet 15 (10): 336. https://doi.org/10.3390/fi15100336.

Macken-Horarik, Mary. 2003. “APPRAISAL and the special instructiveness of narrative.” Text - Interdisciplinary Journal for the Study of Discourse 23 (2): 285–312. https://doi.org10.1515/text.2003.012.

Martin, Jim R., and Peter Robert Rupert White. 2005. The Language of Evaluation: Appraisal in English. Palgrave Macmillan.

Orel Kos, Silvana. 2024. “Introduction of machine translation into audiovisual translation teaching.” ELOPE: English Language Overseas Perspectives and Enquiries 21 (1): 185–208. https://doi.org/10.4312/elope.21.1.185-208.

Page, Ruth E. 2003. “An analysis of APPRAISAL in childbirth narratives with special consideration of gender and storytelling style.” Text 23 (2): 211–37. https://doi.org/10.1515/text.2003.009.

Shahriar, Sakib, and Kadhim Hayawi. 2023. “Let’s have a chat! A conversation with ChatGPT: Technology, applications, and limitations.” Artificial Intelligence and Applications 2 (1): 11–20. https://doi.org/10.47852/bonviewaia3202939.

Thompson, Geoff. 2008. “Appraising glances: Evaluating Martin’s model of APPRAISAL.” Word 59 (1–2): 169–87. https://doi.org/10.1080/00437956.2008.11432585.

—. 2014. “AFFECT and emotion, target-value mismatches, and Russian dolls: Refining the APPRAISAL model.” In Evaluation in Context, edited by Geoff Thompson and Laura Alba-Juez, 47–66. John Benjamins.

Tica, Lena, and Ivana Krsmanović. 2024. “Overcoming the writer’s block? Exploring students’ motivation and perspectives on using ChatGPT as a writing assistance tool in ESP.” ELOPE: English Language Overseas Perspectives and Enquiries 21 (1): 129–49. https://doi.org/10.4312/elope.21.1.129-149.

Uchida, Satoru. 2024. “Using early LLMs for corpus linguistics: Examining ChatGPT’s potential and limitations.” Applied Corpus Linguistics 4 (1): 100089. https://doi.org/10.1016/j.acorp.2024.100089.

White, Peter Robert Rupert. 2000. Functional Grammar. Centre for English Language Studies, University of Birmingham.