|[英文摘要]Cognitive Diagnostic Computerized Adaptive Testing (CD-CAT) combines the advantages of cognitive diagnosis and CAT, which could improve the efficiency and accuracy of CD-CAT. CD-CAT can be divided into two types: dichotomous and polytomous. Presently, the majority of researches on CD-CAT are based on dichotomous CD-CAT. However, among the practical tests in psychology and education, there are many polytomous items, which can be further divided into nominal polytomous and ordinal polytomous items according to whether there is an order or grade between every response category. Nominal polytomous items are items whose response categories are independent and without orders or grades between every response category. Although researchers have developed (ordinal) polytomous CDMs and corresponding CD-CAT, few nominal CDMs and CD-CAT are based on nominal responses.
This study introduces seven commonly used item selection methods in dichotomous CD-CAT into NCD-CAT (CD-CAT based on nominal response models). PMR (pattern match ratio) and test efficiency index are evaluated under different conditions between these item selection methods. Here are details of two simulation studies below. Study 1 compared the performance of NR_PWKL, NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, NR_MI, and NR_GDI methods under different test lengths (5, 10, 15, 20) and item pool qualities (high and low) in NCD-CAT. Results showed that: (1) the PMRs of NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI are higher than or equal to that of NR_PWKL, especially in short tests. (2) as test length gets longer, that PMR advantage is missing, which is the same as the results of Zheng and Chang (2016). (3) compared to test length, item quality has a greater impact on PMR. For instance, with item quality descending, the PMR declined about 30% among all conditions. Study 2 is an experiment on variable-length NCD-CAT that was conducted to compare the performance of each item selection method under the conditions of three maximum posterior probabilities (0.8, 0.85, 0.9) and two item qualities (high and low). The results showed that: (1)under all experimental conditions the average test lengths of NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI are shorter than that of NR_PWKL; the difference is more than 0.738. (2)affected by item quality, the average length of NR_GDI is smaller than that of NR_PWKL under high-quality conditions and larger than it under low-quality conditions.
To sum up, this study compared the performance of 7 commonly used item selection methods of dichotomous CD-CAT in NCD-CAT with different conditions (fixed and variable length). The simulation study showed that under most conditions, the NR_PWCDI, NR_PWACDI, NR_MPWKL, NR_SHE, and NR_MI methods performed well when compared to baseline algorithm NR_PWKL. This study has expanded the alternatives of item selection methods in NCD-CAT.
|[V1]||2022-05-08 11:34:10||chinaXiv:202205.00030v1(View This Version)||Download|