Print ISSN: 2204-1990

Online ISSN: 1323-6903

Keywords : performance-based assessment

Rater Errors in Rating Processandthe Needsto Be Identified Among Student Raters

NurulFarienaAsli, Mohd Effendi Ewan MohdMatore, MelorMdYunos

Journal of Contemporary Issues in Business and Government, 2021, Volume 27, Issue 2, Pages 6140-6158
DOI: 10.47750/cibg.2021.27.02.608

In assessing performance-based language assessment, rater behaviour is one of the
contributing factors in measurement error which is derived from rater error that become a threat
in a rating process. The emphasis on students’ self-directed learning after the implementation of
CEFR in the Malaysian English language syllabus has required students to be able to assess their
progress where the validity and reliability of the scores should be unquestionable. However,
since it is a new practice, there is a lack of awareness of the need to identify rater behaviour
among students. Therefore, this paper aims to discuss the different types of rater errors that occur
in a rating process which will highlight the importance of the errors to be identified among
students in secondary schools in Malaysia. Other than that, it is also aimed to propose a
conceptual framework of rater-mediated assessment using the Many Facet Rasch Model
(MFRM) that can be used in understanding the rater errors. The implication of this conceptual
paper is teachers will gain insights into the factors that will become a threat for students to be
good rater. Apart from that, the conceptual framework of rater mediated-assessment using
MFRM will assist teachers to understand the relationship between three facets which are task,
examinee, and rater with the outputs produced by MFRM. Future research should delve into
factors that contribute to student’s rater errors which undoubtedly affecting their judging in a
rating process based on the conceptual framework of rater mediated assessment using MFRM.