Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison

Jose Quito Pedrajita

doi:doi:10.11648/j.edu.20150404.11

| Peer-Reviewed

Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison

Jose Quito Pedrajita

Published in Education Journal (Volume 4, Issue 4)

Received: 16 May 2015 Accepted: 1 June 2015 Published: 23 June 2015

Views: Downloads:

Download PDF

Share This Article

Twitter
Linked In
Facebook

Abstract

This study provides a demonstration of differential item functioning (DIF) analysis. It made use of test scores of 200 junior high school students on a Chemistry Achievement Test, a measure tested for its psychometric properties. One hundred students came from a public school, while the other 100 were private school examinees; one hundred students were males and the other 100 were females; and 95 students were of low ability and 105 students were of high ability based on their English II grades. Four contingency table approaches, the Chi-Square, Distractor Response Analysis, Logistic Regression and the Mantel-Haenszel Statistic, were applied in the DIF analysis to identify test items indicating bias between examinees matched on school type, gender, and English ability. Thereafter, the results for the four approaches were compared. The findings revealed the presence of items indicating school type-, gender-, and English ability-based DIF. There was a high degree of correspondence between the Logistic Regression and the Mantel-Haenszel Statistic in identifying potentially biased test items.

Published in	Education Journal (Volume 4, Issue 4)
DOI	10.11648/j.edu.20150404.11
Page(s)	139-148
Creative Commons	This is an Open Access article, distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution and reproduction in any medium or format, provided the original work is properly cited.
Copyright	Copyright © The Author(s), 2015. Published by Science Publishing Group

Keywords

Contingency Table Approaches, Differential Item Functioning, Differential Item Functioning Analysis, Item Bias

References

[1]	Baker, F. & Kim, S. H. (2004). Item Response Theory Parameters Estimation Techniques. New York: Marcel Dekker Inc. 2nd edition.
[2]	Barnett, S. & Ercikan, K. (2006). Examining sources of gender DIF in mathematics assessments using a confirmatory multidimensional model approach. Applied Measurement in Education. 19(4), 289-304.
[3]	Camilli, G. and Shepard, L. (1994). Methods for Identifying Biased Test Items. Volume 4, Sage Publications, Inc., California.
[4]	Embretson, S. E. & Reise, S. P. (2000). Item Response Theory for Psychologists. London: Lawrence Erlbaum Associates, Publishers.
[5]	Fidalgo, A. M. (2011). A new approach for differential item functioning detection using Mantel-Haenszel Methods. The GMHDIF Program. The Spanish Journal of Psychology, 14:2.
[6]	Gierl, M. J. (1999). Differential item functioning on the Alberta Education Social Studies 30 Diploma Examination. Canadian Social Studies, Vol. 33, No. 2.
[7]	Hambleton, R. K., Swaminathan, H., & Rogers, J. H. (1991). Fundamentals of Item Response Theory (IRT). London: Sage Publications.
[8]	Kamata, A. and Vaughn B. (2004). An introduction to differential item functioning analysis. Learning Disabilities: A Contemporary Journal 2(2), 49 – 69.
[9]	Kanjee, A. (2007). Using logistic regression to detect bias when multiple groups are tested. South African Journal of Psychology, 37, 47 – 61
[10]	Kristjansson, E., Aylesworth, R., McDowell, I., & Zumbo, B. D. (2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65:6, 935-953.
[11]	Le, V. (1999). Identifying differential item functioning on the NELS: 88 History Achievement Test. Center for the Study of Evaluation, National Center for Research on Evaluation, Standards, and Student Testing. Retrieved November 2011 from ProQuest Journals.
[12]	Mazor, K. E., Kanjee, A., and Clauser, B. E. (1995). Using logistic regression and the Mantel-Haenszel with multiple ability estimates to detect differential item functioning. Journal of Educational Measurement, 32, 131 – 144.
[13]	Navas-Ara, M. J., & Gomez-Benito, J. (2002). Effects of ability scale purification on the identification of differential item functioning. European Journal of Psychological Assessment. Vol. 18, No. 1, pp. 9-15.
[14]	Nijenhuis, J. T., Tolboom, E., Resing, W., & Bleichrodt, N. (2004). Does cultural background influence the intellectual performance of children from immigrant groups? The RAKIT Intelligence Test for Immigrant Children. European Journal of Psychological Assessment. Vol. 20, No. 1, pp. 10-26.
[15]	Osterlind, Steven J. (1983). Test Item Bias. Sage Publications, Inc., California.
[16]	Osterlind, S. J. & Everson, H. T. (2009). Differential Item Functioning. 2nd Edition, CA: Sage Publications.
[17]	Penfield, R. P. & Camilli, G. (2007). Differential item functioning and item bias. In C. Rao & S. Sinharay (Eds.) Handbook of Statistics Psychometrics, Vol. 26.
[18]	Reynolds R. C., Livingston, R. B., & Willson, V. (2006). Measurement and Assessment in Education. Boston: Pearson.
[19]	Rogers, H. J. & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105 – 116.
[20]	Roussos, L. A. & Stout, W. (2004). Differential item functioning analysis: Detecting DIF items and testing DIF hypotheses. In D. Kaplan (Ed.) The Sage Handbook of Quantitative Methodology for Social Sciences. Thousand Oaks: Sage.
[21]	Sheppard, R., Han, K., Colarell, S. M., Dai, G., & King, D. W. (2006). Differential Item Functioning by Sex and Race in the Hogan Personality Inventory. SAGE Publications. Retrieved June 1, 2010 from ProQuest Educational Journals
[22]	Stoneberg, B. D. (2004). A study of gender-based differential item functioning (DIF) in the Spring 2003 Idaho Standards Achievement Tests applying the Simultaneous Bias Test (SIBTEST) and the Mantel-Haenzel Chi Square Test. The University of Maryland Measurement, Statistics, and Evaluation Department and the National Center for Education Statistics (NCES) Assessment Division.
[23]	Swaminathan, H. and Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361 – 370.
[24]	Wang, N. & Lane, S. (1996). Detection of gender-related differential item functioning in a mathematics performance test. Applied Measurement in Education, 9:2, 175-199.
[25]	Wiberg, M. (2009). Differential item functioning in mastery tests: A comparison of three methods using real data. International Journal of Testing. 9, 41-59.
[26]	Wolf, L. R. & Phyllis, R. (1990). The SAT gender gap. Women and Languge. Vol. 13, Issue 2. Retrieved September 26, 2011 from ProQuest Journals.
[27]	Zheng, Y., Geirl, M. J., & Cui, Y. (2007). Using real data to compare DIF detection and effect size measures among Mantel-Haenzel, SIBTEST, and Logistic Regression procedures. Paper presented at NCME 2007, Chicago.

Cite This Article

Plain Text BibTeX RIS

APA Style

Jose Quito Pedrajita. (2015). Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison. Education Journal, 4(4), 139-148. https://doi.org/10.11648/j.edu.20150404.11

Copy | Download

ACS Style

Jose Quito Pedrajita. Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison. Educ. J. 2015, 4(4), 139-148. doi: 10.11648/j.edu.20150404.11

Copy | Download

AMA Style

Jose Quito Pedrajita. Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison. Educ J. 2015;4(4):139-148. doi: 10.11648/j.edu.20150404.11

Copy | Download

@article{10.11648/j.edu.20150404.11,
  author = {Jose Quito Pedrajita},
  title = {Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison},
  journal = {Education Journal},
  volume = {4},
  number = {4},
  pages = {139-148},
  doi = {10.11648/j.edu.20150404.11},
  url = {https://doi.org/10.11648/j.edu.20150404.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.edu.20150404.11},
  abstract = {This study provides a demonstration of differential item functioning (DIF) analysis. It made use of test scores of 200 junior high school students on a Chemistry Achievement Test, a measure tested for its psychometric properties. One hundred students came from a public school, while the other 100 were private school examinees; one hundred students were males and the other 100 were females; and 95 students were of low ability and 105 students were of high ability based on their English II grades. Four contingency table approaches, the Chi-Square, Distractor Response Analysis, Logistic Regression and the Mantel-Haenszel Statistic, were applied in the DIF analysis to identify test items indicating bias between examinees matched on school type, gender, and English ability. Thereafter, the results for the four approaches were compared. The findings revealed the presence of items indicating school type-, gender-, and English ability-based DIF. There was a high degree of correspondence between the Logistic Regression and the Mantel-Haenszel Statistic in identifying potentially biased test items.},
 year = {2015}
}

Copy | Download

TY  - JOUR
T1  - Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison
AU  - Jose Quito Pedrajita
Y1  - 2015/06/23
PY  - 2015
N1  - https://doi.org/10.11648/j.edu.20150404.11
DO  - 10.11648/j.edu.20150404.11
T2  - Education Journal
JF  - Education Journal
JO  - Education Journal
SP  - 139
EP  - 148
PB  - Science Publishing Group
SN  - 2327-2619
UR  - https://doi.org/10.11648/j.edu.20150404.11
AB  - This study provides a demonstration of differential item functioning (DIF) analysis. It made use of test scores of 200 junior high school students on a Chemistry Achievement Test, a measure tested for its psychometric properties. One hundred students came from a public school, while the other 100 were private school examinees; one hundred students were males and the other 100 were females; and 95 students were of low ability and 105 students were of high ability based on their English II grades. Four contingency table approaches, the Chi-Square, Distractor Response Analysis, Logistic Regression and the Mantel-Haenszel Statistic, were applied in the DIF analysis to identify test items indicating bias between examinees matched on school type, gender, and English ability. Thereafter, the results for the four approaches were compared. The findings revealed the presence of items indicating school type-, gender-, and English ability-based DIF. There was a high degree of correspondence between the Logistic Regression and the Mantel-Haenszel Statistic in identifying potentially biased test items.
VL  - 4
IS  - 4
ER  -

Copy | Download

Author Information

Jose Quito Pedrajita

Educational Research and Evaluation Department, Division of Educational Leadership and Professional Services, University of the Philippines College of Education, Diliman, Quezon City, Philippines

Download PDF

Sections

Plain Text BibTeX RIS

APA Style

Jose Quito Pedrajita. (2015). Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison. Education Journal, 4(4), 139-148. https://doi.org/10.11648/j.edu.20150404.11

Copy | Download

ACS Style

Jose Quito Pedrajita. Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison. Educ. J. 2015, 4(4), 139-148. doi: 10.11648/j.edu.20150404.11

Copy | Download

AMA Style

Jose Quito Pedrajita. Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison. Educ J. 2015;4(4):139-148. doi: 10.11648/j.edu.20150404.11

Copy | Download

@article{10.11648/j.edu.20150404.11,
  author = {Jose Quito Pedrajita},
  title = {Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison},
  journal = {Education Journal},
  volume = {4},
  number = {4},
  pages = {139-148},
  doi = {10.11648/j.edu.20150404.11},
  url = {https://doi.org/10.11648/j.edu.20150404.11},
  eprint = {https://article.sciencepublishinggroup.com/pdf/10.11648.j.edu.20150404.11},
  abstract = {This study provides a demonstration of differential item functioning (DIF) analysis. It made use of test scores of 200 junior high school students on a Chemistry Achievement Test, a measure tested for its psychometric properties. One hundred students came from a public school, while the other 100 were private school examinees; one hundred students were males and the other 100 were females; and 95 students were of low ability and 105 students were of high ability based on their English II grades. Four contingency table approaches, the Chi-Square, Distractor Response Analysis, Logistic Regression and the Mantel-Haenszel Statistic, were applied in the DIF analysis to identify test items indicating bias between examinees matched on school type, gender, and English ability. Thereafter, the results for the four approaches were compared. The findings revealed the presence of items indicating school type-, gender-, and English ability-based DIF. There was a high degree of correspondence between the Logistic Regression and the Mantel-Haenszel Statistic in identifying potentially biased test items.},
 year = {2015}
}

Copy | Download

TY  - JOUR
T1  - Using Contingency Table Approaches in Differential Item Functioning Analysis: A Comparison
AU  - Jose Quito Pedrajita
Y1  - 2015/06/23
PY  - 2015
N1  - https://doi.org/10.11648/j.edu.20150404.11
DO  - 10.11648/j.edu.20150404.11
T2  - Education Journal
JF  - Education Journal
JO  - Education Journal
SP  - 139
EP  - 148
PB  - Science Publishing Group
SN  - 2327-2619
UR  - https://doi.org/10.11648/j.edu.20150404.11
AB  - This study provides a demonstration of differential item functioning (DIF) analysis. It made use of test scores of 200 junior high school students on a Chemistry Achievement Test, a measure tested for its psychometric properties. One hundred students came from a public school, while the other 100 were private school examinees; one hundred students were males and the other 100 were females; and 95 students were of low ability and 105 students were of high ability based on their English II grades. Four contingency table approaches, the Chi-Square, Distractor Response Analysis, Logistic Regression and the Mantel-Haenszel Statistic, were applied in the DIF analysis to identify test items indicating bias between examinees matched on school type, gender, and English ability. Thereafter, the results for the four approaches were compared. The findings revealed the presence of items indicating school type-, gender-, and English ability-based DIF. There was a high degree of correspondence between the Logistic Regression and the Mantel-Haenszel Statistic in identifying potentially biased test items.
VL  - 4
IS  - 4
ER  -

Copy | Download