AN EFFICIENT METHOD TO EVALUATE AND ENHANCE SPORT JUDGES’ PERFORMANCE DURING COMPETITION: A CASE STUDY IN ACROBATIC GYMNASTICS

Authors

DOI:

https://doi.org/10.52165/sgj.16.3.387-400

Keywords:

excel-based tool, acrobatic gymnastics, judges, evaluation, ACROAJIR®

Abstract

Who judges the gymnastics judges? How do we measure their accuracy and concordance? How do we know if the judging process is fair? The superior jury has this responsibility. However, they normally lack time to provide effective feedback during competitions.

Using macro functions to process statistical and mathematical statements, we designed and validated an automated Excel-based tool called the Automatic Acrobatic Gymnastics Judges Individual Report Tool to evaluate judges’ performances quickly and easily during competition, automatically creating and exporting individual reports showing each judge’s accuracy and concordance performance on a daily basis, rather than after the competition is ended.

We present empirical data for 76 experienced international judges evaluating acrobatic gymnastics routines in four major official events. A total of 1240 individual reports were analyzed and sent confidentially to the judges during the competition, and 952 were analyzed to evaluate whether this feedback was effective in improving judges’ performance during the competition.

The tool provides efficient and easily understood evidence-based feedback on acrobatic gymnastics judges’ performance during competition, quickly and automatically creating, analyzing and sending individualized information to judges, thus helping with specific Technical Committee scoring control tasks during competitions. We suggest that judges' performances remain high or are enhanced after receiving daily evaluation during major competition events.

Downloads

Download data is not yet available.

Author Biographies

  • Juan Antonio Leon - Prados

    Training, Testing and Gymnastics

  • Carmen Ramos
    Statistic and Operational Research Department. Social and Communication Sciences Faculty, Cadiz

    University, Cadiz

References

Akoglu, H. (2018). User’s guide to correlation coefficients. Turkish Journal of Emergency Medicine, 18(3), 91–93. https://doi.org/10.1016/j.tjem.2018.08.001

Anderlucci, L., Lubisco, A., & Mignani, S. (2020). Investigating the Judges Performance in a National Competition of Sport Dance. Social Indicators Research. https://doi.org/10.1007/s11205-019-02256-z

Auweele, Y. Vanden, Boen, F., De Geest, A., & Feys, J. (2004). Judging bias in synchronized swimming: Open feedback leads to nonperformance-based conformity. Journal of Sport and Exercise Psychology, 26(4), 561–571. https://doi.org/10.1123/jsep.26.4.561

Boen, F., Ginis, P., & Smits, T. (2013). Judges in judo conform to the referee because of the reactive feedback system. European Journal of Sport Science, 13(6), 599–604. https://doi.org/10.1080/17461391.2012.756070

Boen, F., van Hoye, K., Auweele, Y. Vanden, Feys, J., & Smits, T. (2008). Open feedback in gymnastic judging causes conformity bias based on informational influencing. Journal of Sports Sciences, 26(6), 621–628. https://doi.org/10.1080/02640410701670393

Boen, F., Vanden Auweele, Y., Claes, E., Feys, J., & De Cuyper, B. (2006). The impact of open feedback on conformity among judges in rope skipping. Psychology of Sport and Exercise, 7(6), 577–590. https://doi.org/10.1016/j.psychsport.2005.12.001

Bučar, M., Čuk, I., Pajek, J., Karacsony, I., & Leskošek, B. (2012). Reliability and validity of judging in women’s artistic gymnastics at University Games 2009. European Journal of Sport Science, 12(3), 207–215. https://doi.org/10.1080/17461391.2010.551416

Dallas, G., Mavidis, A., & Chairopoulou, C. (2011). Influence of angle of view on judges’ evaluations of inverted cross in men’s rings. Perceptual and Motor Skills, 112(1), 109–121. https://doi.org/10.2466/05.22.24.27.PMS.112.1.109-121

Damisch, L., Mussweiler, T., & Plessner, H. (2006). Olympic medals as fruits of comparison? Assimilation and contrast in sequential performance judgments. Journal of Experimental Psychology: Applied, 12(3), 166–178. https://doi.org/10.1037/1076-898X.12.3.166

De Bruin, W. B. (2005). Save the last dance for me: Unwanted serial position effects in jury evaluations. Acta psychologica, 118(3), 245-260. https://doi.org/10.1016/j.actpsy.2004.08.005

Dosseville, F., Laborde, S., & Garncarzyk, C. (2014). Current research in sports officiating and decision-making. In C. Mohiyeddini (Ed.), Contemporary topics and trends in the psychology of sports (pp.13–38). Nova Publishers.

Fasold, F., Memmert, D., & Unkelbach, C. (2012). Extreme judgments depend on the expectation of following judgments: A calibration analysis. Psychology of Sport and Exercise, 13(2), 197-200. https://doi.org/10.1016/j.psychsport.2011.11.004

FIG. (2015). Regulations for the judges’ evaluation programme (JEP) “former fairbrother system” and its’ application. Federation International de Gymnastique.

FIG. (2020). 2017- 2020 Fig Judges’ Rules Specific Rules for Acrobatic. In Specific Rules for Acrobatic Gymnastics. Federation International de Gymnastique.

FIG. (2017). Appendix to the Codes of Points (COP). Federation International de Gymnastique.

Flessas, K., Mylonas, D., Panagiotaropoulou, G., Tsopani, D., Korda, A., Siettos, C., Di Cagno, A., Evdokimidis, I., & Smyrnis, N. (2015). Judging the judges’ performance in rhythmic gymnastics. Medicine and Science in Sports and Exercise, 47(3), 640–648. https://doi.org/10.1249/MSS.0000000000000425

Gambarelli, G., Iaquinta, G., & Piazza, M. (2012). Anti-collusion indices and averages for the evaluation of performances and judges. Journal of Sports Sciences, 30(4), 411–417. https://doi.org/10.1080/02640414.2011.651153

Heiniger, S., & Mercier, H. (2018). National Bias of International Gymnastics Judges during the 2013-2016 Olympic Cycle. http://arxiv.org/abs/1807.10033

Heiniger, S., & Mercier, H. (2021). Judging the judges: evaluating the accuracy and national bias of international gymnastics judges. Journal of Quantitative Analysis in Sports, 17(4), 289-305. https://doi.org/10.1515/jqas-2019-0113

Kramer RSS. (2017) Sequential effects in Olympic synchronized diving scores. Royal Society Open science. 4: 160812. http://dx.doi.org/10.1098/rsos.160812

Kendall, M. G., & Babington-Smith, B. (1939). The Problem of m Rankings. The Annals of Mathematical Statistics, 10(3), 275–287. https://doi.org/http://dx.doi.org/10.1214/aoms/1177732186

León-Prados, J. A., & Jemni, M. (2022). Reliability and agreement in technical and artistic scores during real-time judging in two European acrobatic gymnastic events. International Journal of Performance Analysis in Sport, 22(1), 132–148. https://doi.org/10.1080/24748668.2021.1996913

León-Prados, J. A., & Rosales, A. (2019). ACROAJIR®; Automatic ACRO Judges Individual Report Tool. Universidad Pablo de Olavide.

Leskošek, B., Čuk, I., & Peixoto, C. J. D. (2018). Inter-rater reliability and validity of scoring men’s individual trampoline routines at European championships 2014. Science of Gymnastics Journal, 10(1), 69–79.

Lin, L. I. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics, 45(1), 255–265.

McBride, G. (2005). A proposal for strength-of-agreement criteria for Lin’s Concordance Correlation Coefficient. NIWA Client Report, HAM2005-062 https://www.medcalc.org/download/pdf/McBride2005.pdf

Mercier, H., & Heiniger, S. (2018). Judging the Judges: Evaluating the Performance of International Gymnastics Judges. http://arxiv.org/abs/1807.10021

Mercier, H., & Klahn, C. (2017). Judging the judges : Evaluating the performance of international gymnastics judges. MIT Sloan Sports Analytics. https://arxiv.org/pdf/1807.10021.pdf

Plessner, H., & Schallies, E. (2005). Judging the cross on rings: A matter of achieving shape constancy. Applied Cognitive Psychology, 19(9), 1145–1156. https://doi.org/10.1002/acp.1136

Premelč, J., Vučković, G., James, N., & Leskošek, B. (2019). Reliability of judging in DanceSport. Frontiers in Psychology, 10. 1001. https://doi.org/10.3389/fpsyg.2019.01001

Ste-Marie, D. M. (2000). Expertise in women’s gymnastics judging: an observational approach. Perceptual and Motor Skills, 90(2), 543–546. https://doi.org/10.2466/pms.2000.90.2.543

Van Bokhorst, L. G., Knapová, L., Majoranc, K., Szebeni, Z. K., Táborský, A., Tomić, D., & Cañadas, E. (2016). “It’s always the judge’s fault”: Attention, emotion recognition, and expertise in rhythmic gymnastics assessment. Frontiers in Psychology, 7(JUL). https://doi.org/10.3389/fpsyg.2016.01008

Wallis, W. A. (1939). The Correlation Ratio for Ranked Data. Journal of the American Statistical Association, 34(207), 533–538. https://doi.org/10.1080/01621459.1939.10503552

Downloads

Published

2024-10-30

Issue

Section

Articles

How to Cite

Leon - Prados, J., & Ramos, C. (2024). AN EFFICIENT METHOD TO EVALUATE AND ENHANCE SPORT JUDGES’ PERFORMANCE DURING COMPETITION: A CASE STUDY IN ACROBATIC GYMNASTICS. Science of Gymnastics Journal, 16(3), 387-400. https://doi.org/10.52165/sgj.16.3.387-400