Interrater Agreement

定义 Definition

评定者一致性（也称“评分者间一致性/编码者间一致性”）：指多个评定者（raters）对同一对象进行评分、分类或判断时，其结果一致的程度。常用于心理测量、医学诊断、教育评估、内容分析与机器学习标注质量控制中。（也常与 interrater reliability 相关联。）

发音 Pronunciation (IPA)

/ˌɪn.tərˈreɪ.tər əˈɡriː.mənt/

例句 Examples

The interrater agreement was high after the training session.
培训后，评定者一致性很高。

To evaluate the coding scheme, the researchers reported interrater agreement using Cohen’s kappa across three independent raters.
为评估编码方案，研究人员在三位独立评定者之间用 Cohen’s kappa 报告了评定者一致性。

词源 Etymology

inter- 表示“在……之间”，rater 源自 rate（评分、评估），agreement 表示“一致、同意”。合起来字面意思就是“评定者之间的一致”，在学术语境中专指多名评分/标注者对同一材料判断结果的一致程度。

相关词 Related Words

• Reliability
• Validity
• Consistency
• Rater
• Consensus
• Cohen's Kappa
• Fleiss' Kappa
• Intraclass Correlation
• Calibration
• Measurement Error

文献与著作中的用例 Notable Works

• Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics.
• Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement.
• Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin.
• Krippendorff, K. (2018/早期版多次再版). Content Analysis: An Introduction to Its Methodology. (讨论评定者一致性与 Krippendorff’s alpha)
• McHugh, M. L. (2012). Interrater reliability: the kappa statistic. Biochemia Medica.