Using G-theory in the development of performance assessment of the socio-emotional domain of children

Mashitah, M.R. Nor; Mariani, M.N.; Jamaluddin, Shahrir; Rahman, Mohd Nazri Abdul

International Journal of Advanced and Applied Sciences

Int. j. adv. appl. sci.

EISSN: 2313-3724

Print ISSN: 2313-626X

Volume 4, Issue 3 (March 2017), Pages: 141-147

Title: Using G-theory in the development of performance assessment of the socio-emotional domain of children

Author(s): M.R. Nor Mashitah ^{1, 2,} *, M.N. Mariani ¹, Shahrir Jamaluddin ¹, Mohd Nazri Abdul Rahman ¹

Affiliation(s):

¹Department of Educational Psychology and Counseling, Faculty of Education, University of Malaya, Kuala Lumpur, Malaysia
²Department of Early Childhood and Education, Faculty of Education and Human Development, Sultan Idris Education University, Perak, Malaysia

https://doi.org/10.21833/ijaas.2017.03.022

Full Text - PDF XML

Abstract:

Performance assessment socio-emotional domain, through the ability of children in the context of the performance criteria. This study, investigates potential applications of Generalizability theory (G-theory) in the development of such a performance-based assessment procedure. 77 kindergarten children were assessed as participants in this study. Firstly, analysis of variance showed that nested rater variance component in person and item (r:pi) component accounted for the highest percentage of the total variance, 0.24942; 42.2% and the smallest, variance of items 0.04232; 7.2%. Secondly, through analysis in G-study, 94% of the overall variance can be explained by the design. Next, based on optimization analysis in D-study that the overall absolute Coefficient G reading remains at 0.97, which was an acceptable value. Lastly, for reliability test from G-facets analysis, the overall cognitive domain reliability was recorded at 0.96 as the reliability of the 38 items was ranging to 0.96. This study base on Theory-G had an impact on minimizing the error of measurement and determining the appropriateness use of items in the administration of the assessment.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Keywords: Socio-emotional domain performancebased assessment, G-Theory

Article History: Received 11 November 2016, Received in revised form 17 January 2017, Accepted 21 January 2017

Digital Object Identifier:

https://doi.org/10.21833/ijaas.2017.03.022

Citation:

Mashitah MRN, Mariani MN, Jamaluddin S, and Rahman MNA (2017). Using G-theory in the development of performance assessment of the socio-emotional domain of children. International Journal of Advanced and Applied Sciences, 4(3): 141-147

http://www.science-gate.com/IJAAS/V4I3/Mashitah.html

References:

Arterberry BJ, Martens MP, Cadigan JM, and Rohrer D (2014). Application of generalizability theory to the big five inventory. Personality and Individual Differences, 69: 98-103. https://doi.org/10.1016/j.paid.2014.05.015 PMid:25419025 PMCid:PMC4236721

Ary D, Jacobs LC, and Razavieh A (1996). Introduction to research in education. Harcourt Brace College Publishers, Florida, USA.

Bennett RE (1999). Using new technologies to improve assessment. Educational Measurement: Issues and Practice, 18(3): 5-12. https://doi.org/10.1111/j.1745-3992.1999.tb00266.x

Bennett RE, Morley M, and Quardt D (2000). Three response types for broadening the conception of mathematical problem solving in computerized tests. Applied Psychological Measurement, 24(4): 294-309. https://doi.org/10.1177/01466210022031769

Brennan RL (2000). Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24(4): 339-353. https://doi.org/10.1177/01466210022031796

Brennan RL (2010). Generalizability theory and classical test theory. Applied Measurement in Education, 24(1): 1-21. https://doi.org/10.1080/08957347.2011.532417

Büyükkidik S and Anil D (2015). Investigation of reliability in generalizability theory with different designs on performance-based assessment. Education and Science, 40(177): 285-296. https://doi.org/10.15390/eb.2015.2454

Cardinet J, Johnson S, and Pini G (2009). Applying generalizability theory using EduG: Quantitative methodology series. Routledge, New York, USA. PMCid:PMC2698847

Clauser BE, Swanson DB, and Clyman SG (1999). A comparison of the generalizability of scores produced by expert raters and automated scoring systems. Applied Measurement in Education, 12(3): 281-299. https://doi.org/10.1207/S15324818AME1203_4

Dezhi C, Bi YH, Xitao F, and Kejian L (2014). Measurement quality of the Chinese early childhood program rating scale: An investigation using multivariate generalizability theory. Journal of Psychoeducational Assessment, 32(3): 236-248. https://doi.org/10.1177/0734282913504813

Fan CH and Hansmann PR (2015). Applying generalizability theory for making quantitative RTI progress-monitoring decisions. Assessment for Effective Intervention, 40(4): 205-215. https://doi.org/10.1177/1534508415573299

Gardner H (1993). Multiple intelligences: The theory in practice. Basic Book, New York, USA.

Harrington HL, Meisels SJ, McMahon P, Dichtelmiller ML, and Jablon JR (1997). Observing, documenting, and assessing learning: The work sampling system handbook for teacher educators. Rebus, Michigan, USA.

Hills TW (1993). Assessment in context: Teachers and children at work. Young Children, 48(5): 20-28.

Kleinert H, Greene P, and Harte M (2002). Creating and using meaningful alternative assessments. Teaching Exceptional Children, 34(4): 40-47. https://doi.org/10.1177/004005990203400406

Nancy JR (2001). Using authentic assessment to document the emerging literacy skills of young children. Childhood Education, 78(2): 66-69. https://doi.org/10.1080/00094056.2002.10522706

Ralph B and Geoffrey N (2012). Generalizability theory for the perplexed: A practical introduction and guide: AMEE Guide No. 68. Medical Teacher, 34(11): 960-992. https://doi.org/10.3109/0142159X.2012.703791 PMid:23140303

Shavelson R and Webb N (1991). Generalizability theory: A primer. SAGE, California, USA.

Wehlage GG, Newmann FM and Secada WG (1996). Standards for authentic achievement and pedagogy. In: Fred MN (Ed.), Authentic Achievement: Restructuring Schools for Intellectual Quality: 21-48. Jossey-Bass, San Francisco, USA.