您当前的位置: > 详细浏览

认知诊断模型Q矩阵修正:完整信息矩阵的作用

摘要:Q矩阵是CDM的核心元素之一,反映了测验的内部结构和内容设计,通常由领域专家根据经验进行主观界定,因此需要对可能存在的错误进行修正。本研究提出了一种新的Q矩阵修正方法基于完整经验交叉相乘信息矩阵的Wald-XPD方法。采用Monte Carlo模拟检验了新方法的表现,并与同类方法进行了比较。研究表明:新开发的Wald-XPD方法在Q矩阵恢复率、保留正确标定属性的比例以及修正错误标定属性的比例这三个主要指标上均有较好的表现,且整体上优于其他方法,尤其是在修正错误标定的属性方面。通过实证数据展示了Wald-XPD方法在Q矩阵修正中的良好表现。总之,本研究为Q矩阵修正提供了有效的方法。

英文摘要:A Q-matrix, which defines the relations between latent attributes and items, is a central building block of the cognitive diagnostic models (CDMs). In practice, a Q-matrix is usually specified subjectively by domain experts, which might contain some misspecifications. The misspecified Q-matrix could cause several serious problems, such as inaccurate model parameters and erroneous attribute profile classifications. Several Q-matrix validation methods have been developed in the literature, such as the G-DINA discrimination index (GDI), Wald test based on an incomplete information matrix (Wald-IC), and Hull methods. Although these methods have shown promising results on Q-matrix recovery rate (QRR) and true positive rate (TPR), a common drawback of these methods is that they obtain poor results on true negative rate (TNR). It is important to note that the worse performance of the Wald-IC method on TNR might be caused by the incorrect computation of the information matrix. A new Q-matrix validation method is proposed in this paper that constructs a Wald test with a complete empirical cross-product information matrix (XPD). A simulation study was conducted to evaluate the performance of the Wald-XPD method and compare it with GDI, Wald-IC, and Hull methods. Five factors that may influence the performance of Q-matrix validation were manipulated. Attribute patterns were generated following either a uniform distribution or a higher-order distribution. The misspecification rate was set to two levels: $QM\text{=}0.15$and$QM\text{=}0.3$. Two sample sizes were manipulated: 500 and 1000. The three levels of IQ were defined as high IQ, ${{P}_{j}}\left( 0 \right)\sim U(0,0.2)$and${{P}_{j}}\left( 1 \right)\sim U(0.8,1)$; medium IQ, ${{P}_{j}}\left( 0 \right)\sim U(0.1,0.3)$ and ${{P}_{j}}\left( 1 \right)\sim U(0.7,0.9)$; and low IQ, ${{P}_{j}}\left( 0 \right)\sim U(0.2,0.4)$ and ${{P}_{j}}\left( 1 \right)\sim U(0.6,0.8)$. The number of attributes was fixed at $K\text{=}4$. Two ratios of the number of items to attribute were considered in the study: $J=16$$\left[ (K\text{=}4)\times (JK\text{=}4) \right]$ and $J=32$$\left[ (K\text{=}4)\times (JK\text{=}8) \right]$. The simulation results showed the following. (1) The Wald-XPD method always provided the best results or was close to the best-performing method across the different factor levels, especially in the terms of the TNR. The HullP and Wald-IC methods produced larger values of QRR and TPR but smaller values of TNR. A similar pattern was observed between HullP and HullR, with HullP being better than HullR. Among the Q-matrix validation methods considered in this study, the GDI method was the worst performer. (2) The results from the comparison of the HullP, Wald-IC, and Wald-XPD methods suggested that the Wald-XPD method is more preferred for Q-matrix validation. Even though the HullP and Wald-IC methods could provide higher TPR values when the conditions were particularly unfavorable (e.g., low item quality, short test length, and low sample size), they obtain very low TNR values. The practical application of the Wald-XPD method was illustrated using real data. In conclusion, the Wald-XPD method has excellent power to detect and correct misspecified q-entry. In addition, it is a generic method that can serve as an important complement to domain experts judgement, which could reduce their workload.

版本历史

[V1] 2022-07-15 21:05:42 chinaXiv:202207.00131V1 下载全文
点击下载全文
同行评议状态
待评议
许可声明
metrics指标
  • 点击量255
  • 下载量83
评论
分享
邀请专家评阅