Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Using machine learning to improve Q-matrix validation

Using machine learning to improve Q-matrix validation The Q-matrix, which specifies the relationship between items and attributes, is a crucial component of cognitive diagnostic models (CDMs). A precisely specified Q-matrix allows for valid cognitive diagnostic assessments. In practice, a Q-matrix is usually developed by domain experts, and noted as being subjective and potentially containing misspecifications which can decrease the classification accuracy of examinees. To overcome this, some promising validation methods have been proposed, such as the general discrimination index (GDI) method and the Hull method. In this article, we propose four new methods for Q-matrix validation based on random forest and feed-forward neural network techniques. Proportion of variance accounted for (PVAF) and coefficient of determination (i.e., the McFadden pseudo-R2) are used as input features for developing the machine learning models. Two simulation studies are carried out to examine the feasibility of the proposed methods. Finally, a sub-dataset of the PISA 2000 reading assessment is analyzed as illustration. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Behavior Research Methods Springer Journals

Using machine learning to improve Q-matrix validation

Behavior Research Methods , Volume OnlineFirst – May 25, 2023

Loading next page...
 
/lp/springer-journals/using-machine-learning-to-improve-q-matrix-validation-77T02K3hMc

References (42)

Publisher
Springer Journals
Copyright
Copyright © The Psychonomic Society, Inc. 2023. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
eISSN
1554-3528
DOI
10.3758/s13428-023-02126-0
Publisher site
See Article on Publisher Site

Abstract

The Q-matrix, which specifies the relationship between items and attributes, is a crucial component of cognitive diagnostic models (CDMs). A precisely specified Q-matrix allows for valid cognitive diagnostic assessments. In practice, a Q-matrix is usually developed by domain experts, and noted as being subjective and potentially containing misspecifications which can decrease the classification accuracy of examinees. To overcome this, some promising validation methods have been proposed, such as the general discrimination index (GDI) method and the Hull method. In this article, we propose four new methods for Q-matrix validation based on random forest and feed-forward neural network techniques. Proportion of variance accounted for (PVAF) and coefficient of determination (i.e., the McFadden pseudo-R2) are used as input features for developing the machine learning models. Two simulation studies are carried out to examine the feasibility of the proposed methods. Finally, a sub-dataset of the PISA 2000 reading assessment is analyzed as illustration.

Journal

Behavior Research MethodsSpringer Journals

Published: May 25, 2023

Keywords: Cognitive diagnosis; Q-matrix validation; Machine learning; G-DINA; Classification

There are no references for this article.