Effectiveness of an AI-based learning assistant as a medium for strengthening students' higher order thinking skills (HOTS)

Abstract

This study examines the use of an AI-based learning assistant (ChatGPT) and changes in students’ Higher-Order Thinking Skills (HOTS) in the Wired and Wireless Networking course within the Computer and Networking Engineering (TKJ) program. HOTS encompasses the abilities to analyze (C4), evaluate (C5), and create (C6) based on Bloom’s revised taxonomy. The study employed a quantitative approach with a one-group time-series pre-experimental design, involving 15 11th-grade students selected through purposive sampling. The researchers collected data through problem-based HOTS essay tests, questionnaires, and observations. The instruments showed content validity with Aiken’s V values ranging from 0.838 to 0.852, and the HOTS test showed reliability with Cronbach’s α = 0.934. The Lilliefors test indicated that the data were normally distributed across the three sessions. The mean HOTS score increased from 66.41 in Session 1 to 72.88 in Session 2 and 77.56 in Session 3. A repeated-measures ANOVA analysis revealed a significant difference, F (2,28) = 54.993, p < 0.001, ηp² = 0.797, with a large effect size. The Greenhouse–Geisser correction still yielded significant results. The overall N-Gain value of 0.335 fell into the moderate category. At the same time, student acceptance of the AI-Based Learning Assistant reached 80.00%, which is classified as good. The findings indicate an increase in HOTS during AI-assisted learning.