Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Interrater Agreement and Reliability

Interrater Agreement and Reliability The distinction between interrater (or interobserver, interjudge, interscorer) "agreement" and "reliability" is discussed. A total of 3 approaches or techniques for the estimation of interrater agreement and reliability are illustrated and compared, using data from a hypothetical study. The 3 approaches are (a) simple percentage of agreement and kappa, (b) simple correlational techniques, and (c) generalizability (G) theory techniques. In discussing the relative advantages and disadvantages of the various approaches, the G theory techniques are emphasized-because they are the most comprehensive and flexible, allowing the researcher to isolate multiple sources of measurement error in one study. Some recommendations regarding the "method of choice" for various situations are offered. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Measurement in Physical Education and Exercise Science Taylor & Francis

Interrater Agreement and Reliability

Interrater Agreement and Reliability

Measurement in Physical Education and Exercise Science , Volume 5 (1): 22 – Mar 1, 2001

Abstract

The distinction between interrater (or interobserver, interjudge, interscorer) "agreement" and "reliability" is discussed. A total of 3 approaches or techniques for the estimation of interrater agreement and reliability are illustrated and compared, using data from a hypothetical study. The 3 approaches are (a) simple percentage of agreement and kappa, (b) simple correlational techniques, and (c) generalizability (G) theory techniques. In discussing the relative advantages and disadvantages of the various approaches, the G theory techniques are emphasized-because they are the most comprehensive and flexible, allowing the researcher to isolate multiple sources of measurement error in one study. Some recommendations regarding the "method of choice" for various situations are offered.

Loading next page...
 
/lp/taylor-francis/interrater-agreement-and-reliability-MNdkZhMzXS

References (83)

Publisher
Taylor & Francis
Copyright
Copyright Taylor & Francis Group, LLC
ISSN
1532-7841
eISSN
1091-367X
DOI
10.1207/S15327841MPEE0501_2
Publisher site
See Article on Publisher Site

Abstract

The distinction between interrater (or interobserver, interjudge, interscorer) "agreement" and "reliability" is discussed. A total of 3 approaches or techniques for the estimation of interrater agreement and reliability are illustrated and compared, using data from a hypothetical study. The 3 approaches are (a) simple percentage of agreement and kappa, (b) simple correlational techniques, and (c) generalizability (G) theory techniques. In discussing the relative advantages and disadvantages of the various approaches, the G theory techniques are emphasized-because they are the most comprehensive and flexible, allowing the researcher to isolate multiple sources of measurement error in one study. Some recommendations regarding the "method of choice" for various situations are offered.

Journal

Measurement in Physical Education and Exercise ScienceTaylor & Francis

Published: Mar 1, 2001

There are no references for this article.