Expected kappa lower bound with a fixed sample size
Source:vignettes/08-kappasizefixedn-comprehensive.Rmd
08-kappasizefixedn-comprehensive.RmdWhat the calculation answers
kappaSizeFixedN() reports the expected lower bound of a
one-sided 100(1−alpha)% goodness-of-fit confidence interval around an
anticipated kappa, for a fixed number of subjects. In
jamovi choose Power > Power Analysis by meddecide > Lowest
Expected Kappa for a Fixed Sample Size.
This is a confidence-limit calculation. It does not calculate achieved power, a minimum detectable effect, or a guaranteed minimum observed kappa. The fact that the analysis appears in the Power menu does not change its estimand.
Use kappaSizePower() to plan a hypothesis test and
kappaSizeCI() to size a study for expected confidence
limits.
Example
Suppose 100 independent subjects will each be rated by two raters. The anticipated kappa is 0.60 and the expected category proportions are 0.30 and 0.70.
r <- kappaSizeFixedN(
outcome = "2", kappa0 = 0.60, props = "0.30, 0.70",
raters = "2", alpha = 0.05, n = 100,
showExplanations = TRUE
)
r$text1
#> A total of 100 subjects can produce an expected one-sided 95% lower limit for kappa of 0.440.kappa0 is the agreement anticipated in the study. In
kappaSizePower(), the same argument name instead denotes
the null kappa. Always report which calculation was
used.
The returned lower bound is the smallest kappa compatible with the expected table at the chosen confidence level. Values below it are excluded by that interval. A future study has random counts, so its lower confidence limit can be higher or lower. The expected bound does not guarantee that any particular study will clear a clinical requirement.
Parameters
| Argument | Meaning | Supported values |
|---|---|---|
outcome |
Number of categories |
"2", "3", "4",
"5"
|
kappa0 |
Anticipated kappa | 0.01–0.99 |
props |
Common category proportions | String, summing to one; one prevalence is sufficient for binary outcomes |
raters |
Raters who each rate every subject |
"2" through "6"
|
alpha |
One-sided interval error level | 0.001–0.20; 0.05 requests a 95% lower bound |
n |
Independent subjects | Integer from 11 to 1,000,000 |
showExplanations |
Methodology and study explanation |
TRUE or FALSE
|
showSummary |
Upstream package summary for comparison |
TRUE or FALSE
|
Use a decimal point in proportions. Commas, semicolons, pipes or whitespace can separate values. Category proportions describe the intended population; enriching rare categories changes the agreement study’s target population.
multi <- kappaSizeFixedN(
outcome = "3", kappa0 = 0.70, props = "0.50, 0.30, 0.20",
raters = "3", alpha = 0.05, n = 200
)
multi$text1
#> A total of 200 subjects can produce an expected one-sided 95% lower limit for kappa of 0.639.Model and inference
The calculation uses the common-correlation goodness-of-fit models
underlying kappaSize. The kappa is an
intraclass/Fleiss-type agreement parameter. With two raters and equal
marginal frequencies it coincides with Cohen’s kappa. It is not a
planner for weighted kappa, ICC, unequal rater margins, or a design in
which raters score different subsets of cases.
The module solves the expected-table lower root directly. Except for a binary outcome with two raters, it takes the lower of the fixed-proportion bound and the restricted-maximum-likelihood bound allowing the proportions to be estimated. The optional upstream summary can differ because its algorithm walks kappa in steps of 0.001 and does not necessarily apply the same adjustment.
At alpha 0.05, the one-sided calculation uses a one-degree-of-freedom chi-square cutoff at 1−2alpha. A two-sided 95% interval uses a different cutoff. Do not compare the limits as though the confidence statements were identical.
The sparse-cell diagnostics concern agreement patterns, not merely the number of cases in each category. Several raters and rare findings can leave agreement-pattern cells sparse even when the category counts appear adequate. The asymptotic calculation can then have poor coverage. Follow the warning and simulate the planned analysis; increasing alpha just to suppress a warning changes the inferential question.
Read the kappaSize manual for the underlying model family. A Wald or bootstrap interval produced by an agreement analysis can have different expected limits from this goodness-of-fit interval.
Reporting
A suitable planning statement is:
With 100 independent subjects, two raters, anticipated kappa 0.60, common category proportions 0.30 and 0.70, and alpha 0.05, we calculated the expected one-sided 95% goodness-of-fit lower confidence bound using
kappaSizeFixedN(). The reported bound is a planning quantity, not achieved power or an assurance that the realized study interval will exceed it.
Include the returned bound, module version and all model/sparse-cell warnings in the actual report. Define acceptable disagreement from clinical consequences; Landis–Koch labels are descriptive conventions, not clinical validation criteria.
For broader assumptions see Choosing and interpreting an analysis.