An AI-based Approach for Grading Students' Collaboration

Abstract

Soft skills (such as communication and collaboration) are rarely addressed in programming courses, mostly because they are difficult to teach, assess, and grade. A quantitative, modular, AI-based approach for assessing and grading students' collaboration has been examined in this paper. The pedagogical underpinning of the approach includes a pedagogical framework and a quantitative soft skill assessment rubric, which have been adapted and used in an extracurricular Java programming course. The objective was to identify pros and cons of using different AI methods within this approach when it comes to assessing and grading collaboration in group programming projects. More specifically, fuzzy rules and several machine learning methods (ML onward) have been examined to see which one would yield the best results regarding performance, interpretability/explainability of recommendations and feasibility/practicality. The data used for training and testing span four academic years, and the results suggest that almost all of the examined AI methods, when used within the proposed AI-based approach, can provide adequate grading recommendations as long as teachers cover other aspects of the assessment not covered by the rubrics: code quality, plagiarism, and project completion. The fuzzy-rule-based method requires time and effort to be spent on (manual) creation and tuning of fuzzy rules and sets, whereas the examined ML methods require lesser initial investments but do need historical data for training. On the other hand, the fuzzy-rule-based method can provide the best explanations on how the assessment/grading was made - something that proved to be very important to teachers. IEEE