International Journal of Advanced and Applied Sciences
Int. j. adv. appl. sci.
EISSN: 2313-3724
Print ISSN: 2313-626X
Volume 4, Issue 3 (March 2017), Pages: 141-147
Title: Using G-theory in the development of performance assessment of the socio-emotional domain of children
Author(s): M.R. Nor Mashitah 1, 2, *, M.N. Mariani 1, Shahrir Jamaluddin 1, Mohd Nazri Abdul Rahman 1
Affiliation(s):
1Department of Educational Psychology and Counseling, Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia
2Department of Early Childhood and Education, Faculty of Education and Human Development, Sultan Idris Education University, Perak, Malaysia
https://doi.org/10.21833/ijaas.2017.03.022
Abstract:
Performance assessment socio-emotional domain, through the ability of children in the context of the performance criteria. This study, investigates potential applications of Generalizability theory (G-theory) in the development of such a performance-based assessment procedure. 77 kindergarten children were assessed as participants in this study. Firstly, analysis of variance showed that nested rater variance component in person and item (r:pi) component accounted for the highest percentage of the total variance, 0.24942; 42.2% and the smallest, variance of items 0.04232; 7.2%. Secondly, through analysis in G-study, 94% of the overall variance can be explained by the design. Next, based on optimization analysis in D-study that the overall absolute Coefficient G reading remains at 0.97, which was an acceptable value. Lastly, for reliability test from G-facets analysis, the overall cognitive domain reliability was recorded at 0.96 as the reliability of the 38 items was ranging to 0.96. This study base on Theory-G had an impact on minimizing the error of measurement and determining the appropriateness use of items in the administration of the assessment.
© 2017 The Authors. Published by IASE.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
Keywords: Socio-emotional domain performancebased assessment, G-Theory
Article History: Received 11 November 2016, Received in revised form 17 January 2017, Accepted 21 January 2017
Digital Object Identifier:
https://doi.org/10.21833/ijaas.2017.03.022
Citation:
Mashitah MRN, Mariani MN, Jamaluddin S, and Rahman MNA (2017). Using G-theory in the development of performance assessment of the socio-emotional domain of children. International Journal of Advanced and Applied Sciences, 4(3): 141-147
http://www.science-gate.com/IJAAS/V4I3/Mashitah.html
References:
Arterberry BJ, Martens MP, Cadigan JM, and Rohrer D (2014). Application of generalizability theory to the big five inventory. Personality and Individual Differences, 69: 98-103. https://doi.org/10.1016/j.paid.2014.05.015 PMid:25419025 PMCid:PMC4236721 |
||||
Ary D, Jacobs LC, and Razavieh A (1996). Introduction to research in education. Harcourt Brace College Publishers, Florida, USA. | ||||
Bennett RE (1999). Using new technologies to improve assessment. Educational Measurement: Issues and Practice, 18(3): 5-12. https://doi.org/10.1111/j.1745-3992.1999.tb00266.x |
||||
Bennett RE, Morley M, and Quardt D (2000). Three response types for broadening the conception of mathematical problem solving in computerized tests. Applied Psychological Measurement, 24(4): 294-309. https://doi.org/10.1177/01466210022031769 |
||||
Brennan RL (2000). Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24(4): 339-353. https://doi.org/10.1177/01466210022031796 |
||||
Brennan RL (2010). Generalizability theory and classical test theory. Applied Measurement in Education, 24(1): 1-21. https://doi.org/10.1080/08957347.2011.532417 |
||||
Büyükkidik S and Anil D (2015). Investigation of reliability in generalizability theory with different designs on performance-based assessment. Education and Science, 40(177): 285-296. https://doi.org/10.15390/eb.2015.2454 |
||||
Cardinet J, Johnson S, and Pini G (2009). Applying generalizability theory using EduG: Quantitative methodology series. Routledge, New York, USA. PMCid:PMC2698847 |
||||
Clauser BE, Swanson DB, and Clyman SG (1999). A comparison of the generalizability of scores produced by expert raters and automated scoring systems. Applied Measurement in Education, 12(3): 281-299. https://doi.org/10.1207/S15324818AME1203_4 |
||||
Dezhi C, Bi YH, Xitao F, and Kejian L (2014). Measurement quality of the Chinese early childhood program rating scale: An investigation using multivariate generalizability theory. Journal of Psychoeducational Assessment, 32(3): 236-248. https://doi.org/10.1177/0734282913504813 |
||||
Fan CH and Hansmann PR (2015). Applying generalizability theory for making quantitative RTI progress-monitoring decisions. Assessment for Effective Intervention, 40(4): 205-215. https://doi.org/10.1177/1534508415573299 |
||||
Gardner H (1993). Multiple intelligences: The theory in practice. Basic Book, New York, USA. | ||||
Harrington HL, Meisels SJ, McMahon P, Dichtelmiller ML, and Jablon JR (1997). Observing, documenting, and assessing learning: The work sampling system handbook for teacher educators. Rebus, Michigan, USA. | ||||
Hills TW (1993). Assessment in context: Teachers and children at work. Young Children, 48(5): 20-28. | ||||
Kleinert H, Greene P, and Harte M (2002). Creating and using meaningful alternative assessments. Teaching Exceptional Children, 34(4): 40-47. https://doi.org/10.1177/004005990203400406 |
||||
Nancy JR (2001). Using authentic assessment to document the emerging literacy skills of young children. Childhood Education, 78(2): 66-69. https://doi.org/10.1080/00094056.2002.10522706 |
||||
Ralph B and Geoffrey N (2012). Generalizability theory for the perplexed: A practical introduction and guide: AMEE Guide No. 68. Medical Teacher, 34(11): 960-992. https://doi.org/10.3109/0142159X.2012.703791 PMid:23140303 |
||||
Shavelson R and Webb N (1991). Generalizability theory: A primer. SAGE, California, USA. | ||||
Wehlage GG, Newmann FM and Secada WG (1996). Standards for authentic achievement and pedagogy. In: Fred MN (Ed.), Authentic Achievement: Restructuring Schools for Intellectual Quality: 21-48. Jossey-Bass, San Francisco, USA. |