---
title: "Cohen's kappa (two-rater categorical agreement)"
method_id: SM.RLB.AGR.2RAT.KAPPA
family: Reliability
version: 1.1.0
date_modified: 2026-06-18
canonical: https://clarus.ofrencber.com/library/cohen_kappa.md
source: Clarus method library
---

# Cohen's kappa (two-rater categorical agreement)

> A classical statistical method in the Clarus library (Reliability family).
> Clarus selects, assumption-checks and runs this method on your data with a
> deterministic rule engine and real numerical libraries (scipy / statsmodels);
> it never generates numbers, citations or results.

**Also known as:** Cohen kappa; kappa coefficient; Cohen's coefficient of agreement; kappa.

## Hypotheses

- **Null (H0):** kappa = 0 (observed agreement is no greater than chance-expected agreement)
- **Alternative (H1):** kappa != 0 (agreement differs from chance; in practice interest is in kappa > 0)

## When to use it

Use this method when your goal is to:
- quantify inter rater agreement for two raters
- assess categorical classification agreement beyond chance
- evaluate coding reliability between two coders

## Data it expects

- **dependent:** not_applicable (agreement between two ratings of the same categorical variable)
- **independent:** 2 raters/coders rating the same set of subjects on one nominal (or fixed-set ordinal-treated-as-nominal) category scheme
- **pairing:** paired (each subject is rated by both raters)

## Assumptions Clarus checks

- **categories nominal and shared** (severity: error; on violation: warn_only)
- **independent subjects** (severity: warning; on violation: warn_only)
- **balanced marginals** (severity: warning; on violation: warn_only)

## Effect size reported

- **kappa** (Cohen (1960))

## Honest limitations

- Kappa measures agreement, not accuracy — high agreement does not mean either rater is correct against a true gold standard.
- When one category dominates (skewed marginals), kappa can be low even though raters agree on most subjects (the kappa paradox); always read observed percent agreement alongside kappa.
- Verbal labels (slight/fair/moderate/substantial/almost perfect) are arbitrary conventions; do not treat a band boundary as a hard pass/fail criterion.
- Cohen's kappa assumes two fixed raters; for three or more raters use Fleiss' kappa, and for ordered categories where some disagreements are worse than others use weighted kappa.
- Kappa describes this set of raters and subjects; it does not generalise automatically to other raters or populations.

## Primary sources

- Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37-46. https://doi.org/10.1177/001316446002000104
- Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310
- Fleiss, J. L., Levin, B., & Paik, M. C. (2003). Statistical Methods for Rates and Proportions (3rd ed.), Ch.18.
- Byrt, T., Bishop, J., & Carlin, J. B. (1993). Bias, prevalence and kappa. Journal of Clinical Epidemiology, 46(5), 423-429. https://doi.org/10.1016/0895-4356(93)90018-V
- Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of Clinical Epidemiology, 43(6), 543-549. https://doi.org/10.1016/0895-4356(90)90158-L

## How to cite this

To cite this Clarus method page (the page itself — for the method's own primary sources, see above):

**Plain text**

Clarus, "Cohen's kappa (two-rater categorical agreement)", version 1.1.0, Clarus method library, 2026. https://clarus.ofrencber.com/library/cohen_kappa.md

**APA**

Clarus. (2026). Cohen's kappa (two-rater categorical agreement) (Version 1.1.0) [Statistical method, Clarus method library]. Retrieved from https://clarus.ofrencber.com/library/cohen_kappa.md

**BibTeX**

```bibtex
@misc{clarus-cohen-kappa,
  author       = {Clarus},
  title        = {Cohen's kappa (two-rater categorical agreement)},
  howpublished = {Clarus method library},
  version      = {1.1.0},
  year         = {2026},
  url          = {https://clarus.ofrencber.com/library/cohen_kappa.md}
}
```

_No DOI is minted for Clarus method pages yet; this citation uses the canonical URL, version and date. Cite the primary sources above for the method's scientific provenance._

---

METHOD SM.RLB.AGR.2RAT.KAPPA · VERSION 1.1.0 · UPDATED 2026-06-18 · SOURCE Clarus method library