Examining the Part-of-speech Features in Assessing the Readability of Vietnamese Texts

An-Vinh Luong; Diep Nguyen; Dien Dinh

doi:10.4312/ala.10.2.127-142

Authors

An-Vinh Luong Computational Linguistics Center, University of Science, Ho Chi Minh City, Vietnam http://orcid.org/0000-0002-0754-7995
Diep Nguyen Department of Linguistics, University of Social Sciences & Humanities, Ho Chi Minh City, Vietnam
Dien Dinh Computational Linguistics Center, University of Science, Ho Chi Minh City, Vietnam https://orcid.org/0000-0003-2069-1016

DOI:

https://doi.org/10.4312/ala.10.2.127-142

Keywords:

text readability, text difficulty, Vietnamese text readability, text classification, school textbooks

Abstract

The readability of the text plays a very important role in selecting appropriate materials for the level of the reader. Text readability in Vietnamese language has received a lot of attention in recent years, however, studies have mainly been limited to simple statistics at the level of a sentence length, word length, etc. In this article, we investigate the role of word-level grammatical characteristics in assessing the difficulty of texts in Vietnamese textbooks. We have used machine learning models (for instance, Decision Tree, K-nearest neighbor, Support Vector Machines, etc.) to evaluate the accuracy of classifying texts according to readability, using grammatical features in word level along with other statistical characteristics. Empirical results show that the presence of POS-level characteristics increases the accuracy of the classification by 2-4%.

Downloads

Download data is not yet available.

References

Al-Tamimi, A. K., Jaradat, M., Aljarrah, N., & Ghanim, S. (2014). AARI: Automatic Arabic readability index. International Arab Journal of Information Technology, 11(4), 370-378.

Alves, M. J. (2009). Loanwords in Vietnamese. Loanwords in the world’s language: A Comparative Handbook, 617-637. DOI: https://doi.org/10.1515/9783110218442.617

Brown, J. D., Janssen, G., Trace, J., & Kozhevnikova, L. (2012). A preliminary study of cloze procedure as a tool for estimating English readability for Russian students. In Second Language Studies Paper (pp. 1-22) . University of Hawai'i at Manoa.

Carver, R. P. (1976). Word Length, Prose Difficulty, and Reading Rate. Journal of Reading Behavior, 8(2), 193-203. DOI: https://doi.org/10.1080/10862967609547176

Chall, J. S., & Dale, E. (1995). Readability Revisited: The New Dale-Chall Readability Formula. Northampton, Massachusetts: Brookline Books.

Chen, X., & Meurers, D. (2018). Word frequency and readability: Predicting the text-level readability with a lexical-level attribute. Journal of Research in Reading, 41(3), 486-510. DOI: https://doi.org/10.1111/1467-9817.12121

Chen, Y.-T., Chen, Y.-H., & Cheng, Y.-C. (2013). Assessing Chinese Readability using Term Frequency and Lexical Chain. IJCLCLP, 18(2), 1-18.

Coco, L., Colina, S., Atcherson, S. R., & Marrone, N. (2017). Readability Level of Spanish-Language Patient-Reported Outcome Measures in Audiology and Otolaryngology. American journal of audiology, 26(3), 309-317. doi:10.1044/2017_AJA-17-0018 DOI: https://doi.org/10.1044/2017_AJA-17-0018

Collins-Thompson, K., & Callan, J. (2005). Predicting Reading Difficulty with Statistical Language Models. J. Am. Soc. Inf. Sci. Technol., 56(13), 1448-1462. DOI: https://doi.org/10.1002/asi.20243

Dale, E., & Chall, J. S. (1948). A Formula for Predicting Readability. Educational Research Bulletin, 11-28.

DeFrancis, J. (1977). Colonialism and language policy in Viet Nam. The Hague: Mouton. DOI: https://doi.org/10.1515/9783110802405

Dinh, D., Nguyen, T. N., & Ho, H. T. (2018). Building a corpus-based frequency dictionary of Vietnamese. In (pp. 72-98).

Nguyễn Điệp T. N., Lươnga.-V., & Đinh Điền. (2019). Affection of the part of speech elements in Vietnamese text readability. Acta Linguistica Asiatica, 9(1), 105-118. https://doi.org/10.4312/ala.9.1.105-118 DOI: https://doi.org/10.4312/ala.9.1.105-118

Feng, L., Jansche, M., Huenerfauth, M., & Elhadad, N. e., mie. (2010). A Comparison of Features for Automatic Readability Assessment. Paper presented at the Proceedings of the 23rd International Conference on Computational Linguistics: Posters, Stroudsburg, PA, USA.

François, T., & Fairon, C. (2012). An AI readability formula for French as a foreign language. Paper presented at the Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.

Gunning, R. (1952). The technique of clear writing. New York: McGraw-Hill Book Co.

Heilman, M., Collins-Thompson, K., Callan, J., & Eskenazi, M. (2007, April). Combining Lexical and Grammatical Features to Improve Readability Measures for First and Second Language Texts. Paper presented at the Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, Rochester, New York.

Parker, R. I., Hasbrouck, J. E., & Weaver, L. R. (2001). Spanish readability formulas for elementary-level texts: A validation study. Reading & Writing Quarterly, 17(4), 307-322. doi:10.1080/105735601317095052 DOI: https://doi.org/10.1080/105735601317095052

Jiang, Z., Sun, G., Gu, Q., Yu, L., & Chen, D. (2015). An Extended Graph-Based Label Propagation Method for Readability Assessment. Paper presented at the Web Technologies and Applications, Cham. DOI: https://doi.org/10.1007/978-3-319-25255-1_40

Kincaid, J. P., Fishburne, R. P., Rogers, R. L., & Chissom, B. S. (1975). Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel. Technical Training, Research B (February), 49. DOI: https://doi.org/10.21236/ADA006655

Luong, A.-V., Nguyen, D., & Dinh, D. (2018a, November). A New Formula for Vietnamese Text Readability Assessment. Paper presented at the 2018 10th International Conference on Knowledge and Systems Engineering (KSE). DOI: https://doi.org/10.1109/KSE.2018.8573379

Luong, A.-V., Nguyen, D., & Dinh, D. (2018b, November). Assessing the Readability of Literary Texts in Vietnamese Textbooks. Paper presented at the 2018 5th NAFOSTED Conference on Information and Computer Science (NICS). DOI: https://doi.org/10.1109/NICS.2018.8606848

Mc Laughlin, G. H. (1969). SMOG grading-a new readability formula. Journal of Reading, 12(8), 639-646.

Nguyen, L. T., & Henkin, A. B. (1982). A Readability Formula for Vietnamese. Journal of Reading, 26(3), 243-251.

Nguyen, L. T., & Henkin, A. B. (1985). A Second Generation Readability Formula for Vietnamese. Journal of Reading, 29(3), 219-225.

Pitler, E., & Nenkova, A. (2008). Revisiting readability: A unified framework for predicting text quality. Paper presented at the Proceedings of the conference on empirical methods in natural language processing. DOI: https://doi.org/10.3115/1613715.1613742

Saddiki, H., Bouzoubaa, K., & Cavalli-Sforza, V. (2015). Text readability for Arabic as a foreign language. Paper presented at the Computer Systems and Applications (AICCSA), 2015 IEEE/ACS 12th International Conference of. DOI: https://doi.org/10.1109/AICCSA.2015.7507232

Saddiki, H., Habash, N., Cavalli-Sforza, V., & Al Khalil, M. (2018, July). Feature Optimization for Predicting Readability of Arabic L1 and L2. Paper presented at the Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications, Melbourne, Australia. DOI: https://doi.org/10.18653/v1/W18-3703

Schwarm, S. E., & Ostendorf, M. (2005). Reading Level Assessment Using Support Vector Machines and Statistical Language Models. Paper presented at the Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, Stroudsburg, PA, USA. DOI: https://doi.org/10.3115/1219840.1219905

Sherman, L. A. (1893). Analytics of literature: a manual for the objective study of English prose and poetry. Boston, England: Ginn.

Si, L., & Callan, J. (2001). A Statistical Model for Scientific Readability. Paper presented at the Proceedings of the Tenth International Conference on Information and Knowledge Management, New York, NY, USA. DOI: https://doi.org/10.1145/502585.502695

Spaulding, S. (1956). A Spanish Readability Formula. The Modern Language Journal, 40(8), 433-441. doi:10.1111/j.1540-4781.1956.tb02145.x DOI: https://doi.org/10.1111/j.1540-4781.1956.tb02145.x

Tran, T., Pham, T., Ngo, H., Dien, D., & Collier, N. (2007). Named Entity Recognition in Vietnamese documents. Progress in Informatics, 5-13. doi:10.2201/NiiPi.2007.4.2 DOI: https://doi.org/10.2201/NiiPi.2007.4.2

Al-Ajlan, A. A., Al-Khalifa, H. S., & Al-Salman, A. S. (2008, November). Towards the development of an automatic readability measurements for arabic language. Paper presented at the 2008 Third International Conference on Digital Information Management, University of East London, London, UK. DOI: https://doi.org/10.1109/ICDIM.2008.4746711

Vajjala, S., & Meurers, D. (2012, June). On Improving the Accuracy of Readability Classification using Insights from Second Language Acquisition. Paper presented at the Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, Montréal, Canada.

Sun, G., Jiang, Z., Gu, Q., & Chen, D. (2014, September). Linear model incorporating feature ranking for Chinese documents readability. Paper presented at the The 9th International Symposium on Chinese Spoken Language Processing, Singapore. DOI: https://doi.org/10.1109/ISCSLP.2014.6936601

François, T. (2014, November). An analysis of a French as a Foreign Language Corpus for Readability Assessment. Paper presented at the Proceedings of the third workshop on NLP for computer-assisted language learning, Uppsala, Sweden.

Wang, S., & Andersen, E. (2016, December). Grammatical Templates: Improving Text Difficulty Evaluation for Language Learners. Paper presented at the Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan.

Luong, A.-V., Nguyen, D., & Dinh, D. (2017, October). Examining the text-length factor in evaluating the readability of literary texts in Vietnamese textbooks. Paper presented at the 2017 9th International Conference on Knowledge and Systems Engineering (KSE). DOI: https://doi.org/10.1109/KSE.2017.8119431

Al Khalil, M., Saddiki, H., Habash, N., & Alfalasi, L. (2018, May). A Leveled Reading Corpus of Modern Standard Arabic. Paper presented at the Proceedings of the 11th Language Resources and Evaluation Conference, Miyazaki, Japan.

Jiang, Z., Gu, Q., Yin, Y., & Chen, D. (2018, August). Enriching Word Embeddings with Domain Knowledge for Readability Assessment. Paper presented at the Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, New Mexico, USA.

Luong, A.-V., & Tran, P. (2019, November). Assessing the Readability of Vietnamese Texts Through Comparison. Paper presented at the Computational Data and Social Networks, Cham. DOI: https://doi.org/10.1007/978-3-030-34980-6_22