Wenbin Gan

Wenbin Gan

Researcher, National Institute of Information and Communications Technology (NICT)

Learner Knowledge Assesssment

One of the fundamental tasks when providing personalized tutoring services to learners in online learning systems, such as intelligent tutoring systems and massive open online courses, is to assess the Learners’ Knowledge. Generally, the knowledge construction procedure is constantly evolving because learners dynamically learn and forget over time.

The COVID-19 pandemic has challenged conventional classroom-based teaching and sped up digital transformation in education systems. With the rise of online education platforms, there is an increasing need for machines to track the knowledge of students and tailor their learning experience. How to effectively infer and track the learning progress of a learner through his/her online interaction with learning materials? This is known as the Knowledge Tracing (KT) problem in the literature. Effectively solving the KT problem would unlock the potential of computer-aided education applications such as intelligent tutoring systems, curriculum learning, and learning materials’ recommendation [refer to: Knowledge Tracing: A Survey].

To this end, I have proposed several novel models to trace the evolution of each learner’s knowledge acquisition during exercise activities by incorporating both learner and learning domain modeling.


KTM-DLF (Knowledge Tracing Machine by modeling cognitive item Difficulty and Learning and Forgetting)


Introduction: Knowledge tracing (KT) is essential for adaptive learning to obtain learners’ current states of knowledge for the purpose of providing adaptive service. Generally, the knowledge construction procedure is constantly evolving because students dynamically learn and forget over time. Unfortunately, to the best of our knowledge most existing approaches consider only a fragment of the information that relates to learning or forgetting, and the problem of making use of rich information during learners’ learning interactions to achieve more precise prediction of learner performance in KT remains under-explored. Moreover, existing work either neglects the problem difficulty or assumes that it is constant, and this is unrealistic in the actual learning process as problem difficulty affects performance undoubtedly and also varies overtime in terms of the cognitive challenge it presents to individual learners. To this end, we herein propose a novel model, KTM-DLF (Knowledge Tracing Machine by modeling cognitive item Difficulty and Learning and Forgetting), to trace the evolution of each learner’s knowledge acquisition during exercise activities by modeling his or her dynamic knowledge construction procedure and cognitive item difficulty. Specifically, we first specify the concept of cognitive item difficulty and propose a method to model the cognitive item difficulty adaptively based on learners’ learning histories. Then, based on two classical theories (the learning curve theory and the Ebbinghaus forgetting curve theory), we propose a method for modeling learners’ learning and forgetting over time. Finally, the KTM-DLFmodel is proposed to incorporate learners’ abilities, the cognitive item difficulty, and the two dynamic procedures (learning and forgetting) together. We then use the factorization machine framework to embed features in high dimensions and model pairwise interactions to increase the model’s accuracy. Extensive experiments have been conducted on three public real-world datasets, and the results confirm that our proposed model outperforms the other state-of-the-art educational data mining models.


[Keywords]: Knowledge tracing · Learner modeling · Knowledge construction procedure · Cognitive item difficulty · Learning and forgetting

KIEDLKD (knowledge interaction-enhanced dynamic learner knowledge diagnosis)


Introduction: One of the fundamental tasks when providing personalized tutoring services to learners in online learn- ing systems, such as intelligent tutoring systems and massive open online courses, is the learner knowl- edge diagnosis (LKD). LKD obtains the learner knowledge proficiency on skills by modeling their learning performance. Learners’ knowledge construction process is not static, but evolves overtime; hence, the evolution of learners’ knowledge proficiency must be dynamically traced. Moreover, considering the wide usage of online learning systems by large numbers of learners, the LKD task also needs to meet the requirements of large-scale assessment and interpretability to explain the diagnosed results. The existing models are either designed for static scenarios or find it difficult to explain the causality between learner performance and knowledge proficiency, as well as the item characteristics. To solve these issues, we pro- pose herein a novel model, called the knowledge interaction-enhanced dynamic LKD (KIEDLKD), to develop learner performance, and hence, dynamically diagnose and trace the evolution of each learner’s knowl- edge proficiency during the exercising activities. We first propose a dynamic LKD framework by unifying the strength of the memory capacity of the key-value memory network to enhance the representation of the knowledge state during learner performance modeling and the interpretability of the Item Response Theory (IRT) to explain the learner performance in terms of knowledge proficiency and item character- istics (i.e., item difficulty and discrimination). In this framework, we diagnose and trace each learner’s knowledge proficiency on each knowledge concept (KC) over time and store them into an auxiliary mem- ory using the key-value memory network. We further infer their general proficiencies and the IRT-based item characteristics using another neural network. Moreover, we propose the knowledge interaction con- cept among KCs and incorporate it into the LKD procedure to further exploit the long-term dependencies in the exercising sequences, thereby devising the KIEDLKD model. We also incorporate the learner- oriented cognitive item difficulty into our model, based on each learner’s exercising history, to adaptively model the item difficulty. Based on these factors, our KIEDLKD model can not only output the learners’ knowledge proficiency in a multi-granularity manner but also output the item characteristics, making it possible to interpret the learner performances in terms of their current knowledge states and item characteristics. Extensive experiments are conducted from six perspectives on five real-world datasets to test our model.The results of learner performance prediction demonstrate the superiority of our model on the LKD task. It can also automatically discover the underlying interaction between each pair of latent KCs, and the underlying concepts for each exercise. The ablation study verifies the contributions of each component in our model. Moreover, it can depict the evolution of learner knowledge proficiency in a multi-granularity manner and provide additional information for skill domain analysis, which enables the interpretability of our model.


[Keywords]: Learner knowledge diagnosis, Knowledge tracing, Knowledge interaction, Learner performance modeling, Cognitive diagnosis assessment, Item response theory, Intelligent tutoring system

KSGKT (Knowledge Structure‐enhanced Graph Representation Learning Model for Knowledge Tracing)


Introduction: Knowledge tracing (KT) is a fundamental personalized-tutoring technique for learners in online learning systems. Recent KT methods employ flexible deep neural network-based models that excel at this task. However, the adequacy of KT is still challenged by the sparseness of the learners’ exercise data. To alleviate the sparseness problem, most of the exiting KT studies are performed at the skill-level rather than the question-level, as questions are often numerous and associated with much fewer skills. However, at the skill level, KT neglects the distinctive information related to the questions themselves and their relations. In this case, the models can imprecisely infer the learners’ knowledge states and might fail to capture the long-term dependencies in the exercising sequences. In the knowledge domain, skills are naturally linked as a graph (with the edges being the prerequisite relations between pedagogical concepts). We refer to such a graph as a knowledge structure (KS). Incorporating a KS into the KT procedure can potentially resolve both the sparseness and information loss, but this avenue has been underexplored because obtaining the complete KS of a domain is challenging and labor-intensive. In this paper, we propose a novel KS-enhanced graph representation learning model for KT with an attention mechanism (KSGKT). We first explore eight methods that automatically infer the domain KS from learner response data and integrate it into the KT procedure. Leveraging a graph representation learning model, we then obtain the question and skill embeddings from the KS-enhanced graph. To incorporate more distinctive information on the questions, we extract the cognitive question difficulty from the learning history of each learner. We then propose a convolutional representation method that fuses these disctinctive features, thus obtaining a comprehensive representation of each question. These representations are input to the proposed KT model, and the long-term dependencies are handled by the attention mechanism. The model finally predicts the learner’s performance on new problems. Extensive experiments conducted from six perspectives on three real-world data sets demonstrated the superiority and interpretability of our model for learner-performance modeling. Based on the KT results, we also suggest three potential applications of our model.


[Keywords]: cognitive question difficulty, graph representation learning, intelligent tutoring systems, knowledge structure discovery, knowledge tracing, learner proficiency estimation