Developing and validating instrument of alignment understanding with school assessment practice

Abdullah, Eftah Moh; Shukor, Abd Aziz Abd; Abdullah, Norazilawati; Arip, Mohammad Aziz Shah Mohamed

International Journal of Advanced and Applied Sciences

Int. j. adv. appl. sci.

EISSN: 2313-3724

Print ISSN: 2313-626X

Volume 3, Issue 11 (November 2016), Pages: 16-22

Title: Developing and validating instrument of alignment understanding with school assessment practice

Author(s): Eftah Moh Abdullah *, Abd Aziz Abd Shukor, Norazilawati Abdullah, Mohammad Aziz Shah Mohamed Arip

Affiliation(s):

Faculty of Education and Human Development, Sultan Idris Education University, Tanjung Malim, Malaysia

https://doi.org/10.21833/ijaas.2016.11.004

Full Text - PDF XML

Abstract:

This study aims to develop and validate the Understanding Alignment with School Assessment Practice instrument (2KAPS). The instrument consists of 27 items. The 2KAPS questionnaire validation involves 109 teachers who taught Form 1 and Form 2 students (teachers directly involved in the School Assessment Practice implementation) in one district in Perak. The instrument was developed in several stages such as building the understanding alignment model and an assessment practice generated based on alignment models from the literature review, determining the main constructs in the assessment expectations, determining the chosen practice in line with the assessment expectations, the use of the Likert scale with three categories (Full agreement=3, Lack of agreement=2 and No Agreement=1) which indicated that there was an alignment between the assessment practice and the teachers’ assessment understanding, acquiring the content validity from experts and the analysis of items using the Rasch Measurement Model. The instrument validity and reliability had been conducted by identifying the Rasch fit statistics, item difficulty, unidimensionality, item reliability as well as 2KAPS item map. The Rasch analysis showed that the item reliability was valued at 0.92 while the Cronbach Alpha value was 0.90. All the items fit the model as their MNSQ values were between 0.7 and 1.35. The dispersion of items from 2KAPS data was 3.29 which indicated the existence of 3 to 4 item strata. No item showed a negative point measure correlation or less than 0.2 and this generally indicated that the item discrimination was very good. The data showed that the mean for a person was measured at 1.19 logits with a standard deviation of 1.12 logits while the item mean value was zero with a standard deviation of 0.52. This indicates that the position of item and person does not fully match and thus shows a medium difficulty. The overall item quality was good and all 27 items of 2KAPS were retained.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Keywords: Assessment understanding, Assessment practice, Instrument development, Validation, Alignment

Article History: Received 17 August 2016, Received in revised form 20 October 2016, Accepted 30 October 2016

Digital Object Identifier: https://doi.org/10.21833/ijaas.2016.11.004

Citation:

Abdullah EM, Shukor AAA, Abdullah N, and Arip MASM (2016). Developing and validating instrument of alignment understanding with school assessment practice. International Journal of Advanced and Applied Sciences, 3(11): 16-22

http://www.science-gate.com/IJAAS/V3I11/Eftah.html

References:

Biggs J (1996). Enhancing teaching through constructive alignment. Higher Education, 32(3): 347-364. http://dx.doi.org/10.1007/BF00138871

Biggs J (1999). Teaching for Quality Learning at University. SRHE and Open University Press, Buckingham, UK.

Bond TG and Fox CM (2003). Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Journal of Educational Measurement, 40(2), pp.185-187. http://dx.doi.org/10.1111/j.1745-3984.2003.tb01103.x

Boss T, Endorf D and Duckendahl C (2001). Informing state assessment from the local level: A district's reflections. Annual Meeting of the Mid-Western Education Research Association, Chicago, Illinois, USA.

Cronbach LJ and Thorndike RL (1971). Educational measurement. Test Validation, American Council in Education, Washington, DC, USA: 443-507.

Green KE and Frantom CG (2002). Survey development and validation with the Rasch model. In International Conference on Questionnaire Development, Evaluation, and Testing, Charleston, USA.

Huba ME and Freed JE (2000). Learner centered assessment on college campuses: Shifting the focus from teaching to learning. Community College Journal of Research and Practice, 24(9): 759-766.

Klenowski V (2011). Assessment for learning in the accountability era: Queensland, Australia. Studies in Educational Evaluation, 37(1): 78-83. http://dx.doi.org/10.1016/j.stueduc.2011.03.003

Linacre JM (1994). Sample Size and Item Calibration Stability. Rasch Measurement Transactions, 7 (4): 328.

Linacre JM (2006). Data variance explained by Rasch measures. Rasch Measurement Transactions, 20(1): 1045.

Linacre JM (2007). Standard errors and reliabilities: Rasch and raw score. Rasch Measurement Transactions, 20(4): 1086.

Linacre JM and Fisher WP (2012). Harvey Goldstein's objections to Rasch measurement: A response from Linacre and Fisher. Rasch Measurement Transactions, 26(3): 1383-1389.

Massey A (1995). Evaluation and analysis of examination data: Some guidelines for reporting and interpretation. UCLES Internal Report, Cambridge, UK.

McMillan JH (2007). Formative classroom assessment: The key to improving student achievement. Formative Classroom Assessment: Theory Into Practice, Teachers College Press, New York, USA: 1-7. PMCid:PMC3313470

Moss PA (1992). Shifting conceptions of validity in educational measurement: Implications for performance assessment. Review of Educational Research, 62(3): 229-258. http://dx.doi.org/10.3102/00346543062003229

Newmann FM (1993). Beyond Common Sense in Educational Restructuring The Issues of Content and Linkage. Educational Researcher, 22(2): 4-22. http://dx.doi.org/10.3102/0013189X022002004

Ohlsen MT (2007). Classroom assessment practices of secondary school members of NCTM. American Secondary Education, 36(1): 4-13

Pray WS and Popovich NG (1985). The development of a standardized competency examination for doctor of pharmacy students. American Journal of Pharmaceutical Education, 49(1): 1-9.

Smith LI (2002). A Tutorial on Principal Components Analysis. Available online at: http://reflect.otago. ac.nz/cosc453/student_tutorials/principal_components.pdf; 2002.

Smith RM and Miao CY (1994). Assessing unidimensionality for Rasch measurement. Objective Measurement: Theory into Practice, 2: 316-327.

Spady WG (1994). Outcome-Based education: Critical issues and answers. American Association of School Administrators, Arlington, USA. PMCid:PMC294331

Stiggins RJ (2002). Assessment crisis: The absence of assessment for learning. Phi Delta Kappan, 83(10): 758-765. http://dx.doi.org/10.1177/003172170208301010

Webb NL (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Research Monograph No. 6, National Institute for Science Education Publications, Washington, USA.

Wright BD and Stone MH (1979). Best Test Design (Rasch Measurement Series). MESA Press, Chicago, USA.