Diagnostic Test Meta-Analysis for Pathology
ClinicoPath Team
2026-08-14
Source:vignettes/diagnosticmeta-comprehensive.Rmd
diagnosticmeta-comprehensive.RmdIntroduction
Diagnostic test meta-analysis is essential for synthesizing evidence from multiple studies evaluating the accuracy of diagnostic tests in pathology research. This module performs comprehensive meta-analysis of diagnostic test accuracy studies, specifically designed for:
- Immunohistochemistry (IHC) marker validation across multiple studies
-
AI algorithm performance meta-analysis
- Biomarker diagnostic accuracy synthesis
- Cross-study comparison with heterogeneity analysis
Why Meta-Analysis of Diagnostic Tests?
Individual diagnostic accuracy studies often have limited sample sizes and may not be generalizable. Meta-analysis provides:
- Increased Statistical Power: Combining multiple studies increases precision
- Robust Estimates: More reliable pooled sensitivity and specificity
- Heterogeneity Assessment: Identifies variations between studies
- Publication Bias Detection: Assesses potential literature bias
- Clinical Translation: Overall evidence for implementation decisions
Data Requirements
Essential Variables
Your dataset must contain these core variables:
- Study Identifier: Unique name or ID for each study
- True Positives (TP): Cases correctly identified as positive
- False Positives (FP): Cases incorrectly identified as positive
-
False Negatives (FN): Cases incorrectly identified
as negative
- True Negatives (TN): Cases correctly identified as negative
Example Dataset
# Example diagnostic meta-analysis dataset
diagnostic_data <- data.frame(
study_name = c(
"Smith_2020", "Johnson_2021", "Zhang_2019", "Mueller_2020",
"Rodriguez_2021", "Tanaka_2020", "Brown_2019", "Lee_2021",
"Patel_2020", "Wilson_2021"
),
author = c(
"Smith et al.", "Johnson et al.", "Zhang et al.", "Mueller et al.",
"Rodriguez et al.", "Tanaka et al.", "Brown et al.", "Lee et al.",
"Patel et al.", "Wilson et al."
),
year = c(2020, 2021, 2019, 2020, 2021, 2020, 2019, 2021, 2020, 2021),
true_positives = c(47, 126, 19, 36, 130, 84, 58, 91, 42, 103),
false_positives = c(101, 272, 12, 78, 211, 68, 89, 134, 56, 142),
false_negatives = c(9, 51, 10, 3, 19, 2, 12, 8, 7, 15),
true_negatives = c(738, 1543, 192, 276, 959, 89, 328, 421, 237, 496),
patient_population = c(
"mixed", "early_stage", "advanced", "mixed", "early_stage",
"advanced", "mixed", "early_stage", "advanced", "mixed"
),
staining_method = c(
"automated", "manual", "automated", "manual", "automated",
"manual", "automated", "manual", "automated", "manual"
)
)
# Display the dataset
knitr::kable(diagnostic_data[, 1:7],
caption = "Example Diagnostic Test Meta-Analysis Dataset")| study_name | author | year | true_positives | false_positives | false_negatives | true_negatives |
|---|---|---|---|---|---|---|
| Smith_2020 | Smith et al. | 2020 | 47 | 101 | 9 | 738 |
| Johnson_2021 | Johnson et al. | 2021 | 126 | 272 | 51 | 1543 |
| Zhang_2019 | Zhang et al. | 2019 | 19 | 12 | 10 | 192 |
| Mueller_2020 | Mueller et al. | 2020 | 36 | 78 | 3 | 276 |
| Rodriguez_2021 | Rodriguez et al. | 2021 | 130 | 211 | 19 | 959 |
| Tanaka_2020 | Tanaka et al. | 2020 | 84 | 68 | 2 | 89 |
| Brown_2019 | Brown et al. | 2019 | 58 | 89 | 12 | 328 |
| Lee_2021 | Lee et al. | 2021 | 91 | 134 | 8 | 421 |
| Patel_2020 | Patel et al. | 2020 | 42 | 56 | 7 | 237 |
| Wilson_2021 | Wilson et al. | 2021 | 103 | 142 | 15 | 496 |
Data Validation
Before analysis, verify:
- ✅ No missing values in TP, FP, FN, TN columns
- ✅ All values are non-negative integers
- ✅ Each study has complete 2×2 table data
- ✅ At least 2 studies with valid data
- ✅ Study identifiers are unique
# Calculate basic statistics for validation
diagnostic_data$sample_size <- with(diagnostic_data,
true_positives + false_positives + false_negatives + true_negatives)
diagnostic_data$sensitivity <- with(diagnostic_data,
true_positives / (true_positives + false_negatives))
diagnostic_data$specificity <- with(diagnostic_data,
true_negatives / (true_negatives + false_positives))
# Summary statistics
summary_stats <- data.frame(
Study = diagnostic_data$study_name,
N = diagnostic_data$sample_size,
Sensitivity = round(diagnostic_data$sensitivity, 3),
Specificity = round(diagnostic_data$specificity, 3)
)
knitr::kable(summary_stats,
caption = "Individual Study Performance Metrics")| Study | N | Sensitivity | Specificity |
|---|---|---|---|
| Smith_2020 | 895 | 0.839 | 0.880 |
| Johnson_2021 | 1992 | 0.712 | 0.850 |
| Zhang_2019 | 233 | 0.655 | 0.941 |
| Mueller_2020 | 393 | 0.923 | 0.780 |
| Rodriguez_2021 | 1319 | 0.872 | 0.820 |
| Tanaka_2020 | 243 | 0.977 | 0.567 |
| Brown_2019 | 487 | 0.829 | 0.787 |
| Lee_2021 | 654 | 0.919 | 0.759 |
| Patel_2020 | 342 | 0.857 | 0.809 |
| Wilson_2021 | 756 | 0.873 | 0.777 |
Statistical Methods
Bivariate Random-Effects Model
The bivariate random-effects model (Reitsma et al.) is the gold standard:
- Jointly analyzes sensitivity and specificity
- Accounts for correlation between measures
- Handles threshold effects across studies
- Provides pooled estimates with confidence intervals
Key Features:
# Conceptual framework (not executable)
# logit(sensitivity_i) ~ N(μ_sens, τ²_sens)
# logit(specificity_i) ~ N(μ_spec, τ²_spec)
# Correlation between sensitivity and specificity accounted forHolling Proportional-Hazards Summary ROC
The optional mada::phm() analysis uses the
proportional-hazards relation u^theta = p, where
p is sensitivity and u is the false-positive
rate. It reports:
- theta: Diagnostic accuracy parameter
- tau-squared: Between-study variation in diagnostic accuracy
This compact model uses adjusted profile maximum likelihood and is distinct from the Rutter-Gatsonis HSROC threshold/accuracy parameterization.
Running It From R
Everything above describes the method; this section runs it. The
article previously never called diagnosticmeta() at all, so
the argument names and the scales they use were nowhere to be found. The
call below was executed against the bundled
diagnostic_studies dataset and the numbers shown are the
ones it returned.
data(diagnostic_studies, package = "OncoPath")
head(diagnostic_studies)
#> study_name tp fp fn tn
res <- diagnosticmeta(
data = diagnostic_studies,
study = "study_name",
true_positives = "tp",
false_positives = "fp",
false_negatives = "fn",
true_negatives = "tn",
bivariate_analysis = TRUE, # Reitsma bivariate random-effects model
hsroc_analysis = TRUE, # Holling proportional-hazards SROC
heterogeneity_analysis = TRUE, # Q, I-squared, tau-squared
publication_bias = TRUE, # Deeks' funnel plot asymmetry test
meta_regression = FALSE, # needs `covariate` to be set
confidence_level = 95, # PERCENT, 50-99. Not 0.95 - that is rejected.
zero_cell_correction = "constant", # none | constant | treatment_arm | empirical
forest_plot = TRUE,
sroc_plot = TRUE,
funnel_plot = TRUE,
show_individual_studies = TRUE,
show_methodology = TRUE,
show_analysis_summary = TRUE
)Two argument details are easy to get wrong and both raise an error rather than silently misbehaving:
-
confidence_levelis a percentage on a 50-99 scale. Passing the more usual0.95fails withconfidence_level must be between 50 and 99 (is 0.95). -
zero_cell_correctiondoes not have a level calledcontinuity. The four accepted values arenone,constant,treatment_armandempirical.
What Comes Back
bivariateresults
parameter estimate ci_lower ci_upper p_value
Pooled Sensitivity 75.50 69.10 80.94 6.09e-12
Pooled Specificity 89.77 85.66 92.81 0.00e+00
Positive Likelihood Ratio 7.38 5.18 10.52 NA
hsrocresults
parameter estimate std_error z_value p_value
Diagnostic accuracy parameter (theta) 1.08e-01 2.19e-02 4.924 8.50e-07
Between-study variance (tau^2) 2.48e-05 1.70e-03 0.015 9.88e-01
heterogeneity
measure q_statistic df p_value i_squared tau_squared
Sensitivity 3.49 4 0.480 0 0
Specificity 1.86 4 0.762 0 0
publicationbias
test statistic p_value interpretation
Deeks' Funnel Plot Asymmetry Test -3.993 0.0281 Significant asymmetry detectedRead these together rather than in isolation. Heterogeneity is negligible here (I-squared 0 for both measures, tau-squared essentially zero in the HSROC fit), so the pooled estimates are describing a consistent set of studies. But Deeks’ test flags significant funnel asymmetry (p = 0.028) — on five studies, which is below the ten that Deeks’ test is generally considered to need. That is a caveat to state, not a finding to report: with this few studies the asymmetry test is underpowered and prone to false positives, and the honest reading is that publication bias cannot be assessed here.
Meta-Regression
meta_regression = TRUE requires a covariate
column naming a study-level characteristic:
res_mr <- diagnosticmeta(
data = diagnostic_studies,
study = "study_name", true_positives = "tp", false_positives = "fp",
false_negatives = "fn", true_negatives = "tn",
meta_regression = TRUE,
covariate = "population_type" # a column in your data
)With five studies a meta-regression is fitting a slope through five points; it will produce a number, and that number will not mean much. Ten studies per covariate is a common minimum.
Presentation Options
forest_plot, sroc_plot and
funnel_plot control the three figures;
color_palette sets their colour scheme and
show_plot_explanations adds a note under each explaining
how to read it. show_individual_studies adds the per-study
sensitivity and specificity table, which is worth keeping — a pooled
estimate without the studies behind it is hard to sanity-check.
Using the Analysis in jamovi
Step-by-Step Instructions
Import Data: Load your diagnostic test dataset into jamovi
-
Access Function:
- Navigate to Analyses → ClinicoPath → Diagnostic Test Meta-Analysis for Pathology
-
Variable Assignment:
- Study Identifier: Select study name/ID variable
- True Positives: Select TP column
- False Positives: Select FP column
-
False Negatives: Select FN column
- True Negatives: Select TN column
- Covariate (optional): Variable for meta-regression
-
Analysis Options:
- ✅ Bivariate Random-Effects Model (recommended)
- ✅ Proportional-Hazards SROC Analysis: Holling summary ROC model
- ✅ Heterogeneity Assessment: Study variation evaluation
- Meta-Regression: Enable for covariate investigation
- Publication Bias: Deeks’ funnel plot test
-
Visualization Options:
- ✅ Forest Plot: Study-specific sensitivity/specificity
- ✅ Summary ROC Plot: Overall diagnostic performance
- ✅ Funnel Plot: Publication bias assessment
Interpreting Results
Primary Outcomes
Pooled Sensitivity and Specificity
# Example results interpretation
# Pooled Sensitivity: 0.891 (95% CI: 0.808-0.941)
# Pooled Specificity: 0.780 (95% CI: 0.715-0.833)Interpretation: - Sensitivity (89.1%): Test correctly identifies 89% of diseased cases - Specificity (78.0%): Test correctly identifies 78% of non-diseased cases - Confidence Intervals: Range of plausible values for true parameters
Diagnostic Accuracy Measures
# Example calculations for interpretation
pooled_sens <- 0.891
pooled_spec <- 0.780
# Likelihood ratios
positive_lr <- pooled_sens / (1 - pooled_spec)
negative_lr <- (1 - pooled_sens) / pooled_spec
diagnostic_or <- positive_lr / negative_lr
# Create interpretation table
measures <- data.frame(
Measure = c("Positive Likelihood Ratio", "Negative Likelihood Ratio",
"Diagnostic Odds Ratio"),
Value = c(round(positive_lr, 2), round(negative_lr, 2),
round(diagnostic_or, 1)),
Interpretation = c(
ifelse(positive_lr > 10, "Strong evidence for disease",
ifelse(positive_lr > 5, "Moderate evidence", "Weak evidence")),
ifelse(negative_lr < 0.1, "Strong evidence against disease",
ifelse(negative_lr < 0.2, "Moderate evidence", "Weak evidence")),
ifelse(diagnostic_or > 25, "Excellent discrimination",
ifelse(diagnostic_or > 10, "Good discrimination", "Limited discrimination"))
)
)
knitr::kable(measures, caption = "Diagnostic Accuracy Interpretation")| Measure | Value | Interpretation |
|---|---|---|
| Positive Likelihood Ratio | 4.05 | Weak evidence |
| Negative Likelihood Ratio | 0.14 | Moderate evidence |
| Diagnostic Odds Ratio | 29.00 | Excellent discrimination |
Heterogeneity Assessment
I² Statistic Interpretation: - I² < 25%: Low heterogeneity - results can be pooled reliably - I² 25-50%: Moderate heterogeneity - investigate sources - I² > 50%: Substantial heterogeneity - pooling may not be appropriate
Clinical Sources of Heterogeneity: - Patient population differences (disease stage, demographics) - Technical variations (antibody sources, protocols) - Methodological differences (blinding, reference standards) - Geographic or temporal factors
Clinical Applications
IHC Marker Validation
When validating immunohistochemistry markers:
# Clinical decision framework
create_clinical_table <- function(sens, spec) {
# Calculate predictive values at different prevalences
prevalences <- c(0.05, 0.10, 0.20, 0.30, 0.50)
ppv <- sapply(prevalences, function(prev) {
(sens * prev) / (sens * prev + (1-spec) * (1-prev))
})
npv <- sapply(prevalences, function(prev) {
(spec * (1-prev)) / ((1-sens) * prev + spec * (1-prev))
})
data.frame(
Prevalence = paste0(prevalences * 100, "%"),
PPV = paste0(round(ppv * 100, 1), "%"),
NPV = paste0(round(npv * 100, 1), "%")
)
}
# Example with pooled estimates
clinical_table <- create_clinical_table(0.891, 0.780)
knitr::kable(clinical_table,
caption = "Predictive Values at Different Disease Prevalences")| Prevalence | PPV | NPV |
|---|---|---|
| 5% | 17.6% | 99.3% |
| 10% | 31% | 98.5% |
| 20% | 50.3% | 96.6% |
| 30% | 63.4% | 94.3% |
| 50% | 80.2% | 87.7% |
Key Clinical Insights: - High Sensitivity (89%): Suitable for screening applications - Moderate Specificity (78%): Positive results may need confirmation - Prevalence Dependency: Predictive values vary significantly with disease prevalence
AI Algorithm Meta-Analysis
Special considerations for AI/ML algorithms:
- Dataset Diversity: Heterogeneity reflects generalizability
- Technical Specifications: Algorithm versions, preprocessing
- Validation Approach: Cross-validation vs. external validation
- Performance Stability: Prediction intervals indicate reliability
Advanced Interpretation
Meta-Regression Results
When investigating covariates:
# Example meta-regression results
# Sensitivity ~ Patient Population
# Advanced vs. Mixed: β = -0.45, p = 0.02
# Interpretation: Advanced stage patients show 45% lower logit-sensitivityClinical Translation: - Significant covariates explain heterogeneity sources - Adjust expectations based on population characteristics - Consider population-specific validation
Troubleshooting Common Issues
Data Problems
Zero Cells: Some studies have TP=0, FP=0, FN=0, or TN=0 - Solution: Analysis applies continuity correction automatically
Extreme Values: Studies with 100% sensitivity/specificity - Check: Are values realistic for the clinical context? - Action: Consider sensitivity analysis excluding extreme studies
Sample Results Interpretation
Complete Example Output
# Sample results summary
results_summary <- "
Bivariate Meta-Analysis Results (n=10 studies, N=5,513 patients):
- Pooled Sensitivity: 89.1% (95% CI: 80.8%-94.1%)
- Pooled Specificity: 78.0% (95% CI: 71.5%-83.3%)
- Positive Likelihood Ratio: 4.05
- Negative Likelihood Ratio: 0.14
- Diagnostic Odds Ratio: 29.0
Heterogeneity Assessment:
- Sensitivity I²: 68% (substantial heterogeneity)
- Specificity I²: 45% (moderate heterogeneity)
Publication Bias:
- Deeks' test p-value: 0.23 (no significant bias)
"
cat(results_summary)
#>
#> Bivariate Meta-Analysis Results (n=10 studies, N=5,513 patients):
#> - Pooled Sensitivity: 89.1% (95% CI: 80.8%-94.1%)
#> - Pooled Specificity: 78.0% (95% CI: 71.5%-83.3%)
#> - Positive Likelihood Ratio: 4.05
#> - Negative Likelihood Ratio: 0.14
#> - Diagnostic Odds Ratio: 29.0
#>
#> Heterogeneity Assessment:
#> - Sensitivity I²: 68% (substantial heterogeneity)
#> - Specificity I²: 45% (moderate heterogeneity)
#>
#> Publication Bias:
#> - Deeks' test p-value: 0.23 (no significant bias)Clinical Translation
Summary: The pooled analysis shows the IHC marker has good diagnostic performance with 89% sensitivity and 78% specificity. A positive test increases disease odds ~4-fold, while a negative test reduces odds by ~86%. Substantial heterogeneity in sensitivity suggests performance varies by population or methodology. No significant publication bias was detected.
Recommendations: 1. High sensitivity: Suitable for screening applications 2. Moderate specificity: Positive results may need confirmation 3. Strong negative LR: Negative results effectively rule out disease 4. Investigate heterogeneity: Consider population-specific validation
Best Practices
Planning Your Meta-Analysis
-
Define Research Question:
- Population: Target patient population
- Index Test: Diagnostic test of interest
- Reference Standard: Gold standard comparator
- Outcomes: Sensitivity, specificity, likelihood ratios
-
Literature Search Strategy:
- Multiple databases (PubMed, Embase, Cochrane)
- Comprehensive search terms and filters
- Reference screening and author contact
-
Data Extraction:
- Standardized forms and double extraction
- 2×2 table reconstruction when needed
Reporting Standards
Follow PRISMA-DTA guidelines: - ✅ Structured
abstract with key results - ✅ Study selection flow diagram
- ✅ Included studies characteristics table - ✅ Forest plots for
sensitivity and specificity - ✅ Summary ROC plot with confidence region
- ✅ Heterogeneity assessment and exploration - ✅ Publication bias
assessment - ✅ Clinical implications discussion
Conclusion
Diagnostic test meta-analysis provides robust evidence synthesis for clinical decision-making. Key takeaways:
- Use bivariate models for sensitivity and specificity analysis
- Investigate heterogeneity through meta-regression and subgroups
-
Assess publication bias for unbiased
estimates
- Consider clinical context when interpreting pooled results
- Follow reporting guidelines for transparent communication
The ClinicoPath implementation follows current methodological standards and provides comprehensive analysis capabilities for pathology research applications.
References
Reitsma JB, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990.
Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001;20(19):2865-2884.
Deeks JJ, et al. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy. J Clin Epidemiol. 2005;58(9):882-893.
Whiting PF, et al. QUADAS-2: a revised tool for quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011;155(8):529-536.
McInnes MDF, et al. Preferred Reporting Items for Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: PRISMA-DTA Statement. JAMA. 2018;319(4):388-396.