Skip to contents

Clinical ROC Analysis (enhancedROC)

Overview

The enhancedROC module provides comprehensive Receiver Operating Characteristic (ROC) curve analysis for clinical diagnostic performance evaluation. It includes ROC curve analysis, Youden Index optimization, sensitivity/specificity analysis, optimal cutoff determination, comparative ROC analysis, calibration assessment, multi-class ROC, clinical impact metrics, and class imbalance detection.

All examples below use the test datasets in the data-raw/non-rda/ folder of the ClinicoPath repository (.omv for jamovi, .csv for R). On this website the code is shown without output.


Datasets Used in This Guide

Dataset N Outcome Prevalence Key Features Primary Use
enhancedroc_biomarker 300 Disease/Healthy 30% 3 biomarkers + clinical risk score Single biomarker validation
enhancedroc_comparative 400 Cancer/No Cancer 25% 5 markers (varying AUCs) Compare diagnostic tests
enhancedroc_imbalanced 500 Positive/Negative 5% Rare disease, screening/confirmatory markers Imbalanced data handling
enhancedroc_multiclass 350 4 severity levels Mixed 3 biomarkers Multi-class severity grading
enhancedroc_calibration 300 Event/No Event 35% Predicted probs + risk score Calibration assessment
enhancedroc_screening 600 Screen Pos/Neg 8% High-sensitivity markers Screening context
enhancedroc_confirmatory 250 Confirmed/Not 45% High-specificity markers Confirmatory testing
enhancedroc_small 60 Positive/Negative 30% Single simple marker Small sample testing
enhancedroc_validation 350 Positive/Negative 30% Probability predictors, biomarker CROC / convex hull
enhancedroc_tiedscores 200 Disease/No Disease 35% Ordinal scores (many ties) Scores with many ties

1. Single Biomarker Analysis (Default)

The default analysis computes a single ROC curve with AUC, optimal cutoff via Youden Index, and diagnostic performance metrics.

Basic single ROC

biomarker_data <- read.csv(paste0(data_path, "enhancedroc_biomarker.csv"))

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  customCutoffs = NULL
)

With cutoff table and clinical metrics

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  cutoffTable = TRUE,
  clinicalMetrics = TRUE,
  prevalence = 0.30,
  useObservedPrevalence = TRUE,
  customCutoffs = NULL
)

Direction control and custom cutoffs

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  direction = "higher",
  customCutoffs = "5.0,7.5,10.0"
)

2. Comparative ROC Analysis (Multiple Predictors)

Compare diagnostic performance across multiple biomarkers using DeLong’s test.

Five-marker comparison with DeLong

comparative_data <- read.csv(paste0(data_path, "enhancedroc_comparative.csv"))

enhancedROC(
  data = comparative_data,
  outcome = "cancer_status",
  positiveClass = "Cancer",
  validationNegative = NULL,
  predictors = vars(established_marker, novel_marker1, novel_marker2,
                    imaging_score, genetic_risk_score),
  analysisType = "comparative",
  pairwiseComparisons = TRUE,
  comparisonMethod = "delong",
  customCutoffs = NULL
)

Bootstrap comparison with metrics differences

enhancedROC(
  data = comparative_data,
  outcome = "cancer_status",
  positiveClass = "Cancer",
  validationNegative = NULL,
  predictors = vars(established_marker, novel_marker1),
  analysisType = "comparative",
  comparisonMethod = "bootstrap",
  bootstrapSamples = 100,
  showMetricsDiff = TRUE,
  statisticalComparison = TRUE,
  customCutoffs = NULL
)

Comprehensive analysis with clinical interpretation

enhancedROC(
  data = comparative_data,
  outcome = "cancer_status",
  positiveClass = "Cancer",
  validationNegative = NULL,
  predictors = vars(established_marker, novel_marker1, novel_marker2,
                    imaging_score, genetic_risk_score),
  analysisType = "comprehensive",
  comprehensive_output = TRUE,
  clinical_interpretation = TRUE,
  customCutoffs = NULL
)

3. Bootstrap & Confidence Intervals

BCa bootstrap confidence intervals

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  useBootstrap = TRUE,
  bootstrapSamples = 100,
  bootstrapMethod = "bca",
  customCutoffs = NULL
)

Percentile bootstrap with stratification

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  useBootstrap = TRUE,
  bootstrapSamples = 100,
  bootstrapMethod = "percentile",
  stratifiedBootstrap = TRUE,
  customCutoffs = NULL
)

90% CI with a bootstrap CI for the partial AUC

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  confidenceLevel = 90,
  useBootstrap = TRUE,
  bootstrapSamples = 100,
  bootstrapPartialAUC = TRUE,
  partialAuc = TRUE,
  customCutoffs = NULL
)

4. ROC Curve Plot Options

Clinical theme with cutoff points and confidence bands

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  rocCurve = TRUE,
  showCutoffPoints = TRUE,
  showConfidenceBands = TRUE,
  plotTheme = "clinical",
  customCutoffs = NULL
)

Classic theme with custom dimensions

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  rocCurve = TRUE,
  plotTheme = "classic",
  plotWidth = 800,
  plotHeight = 800,
  customCutoffs = NULL
)

Comparative ROC overlay plot

enhancedROC(
  data = comparative_data,
  outcome = "cancer_status",
  positiveClass = "Cancer",
  validationNegative = NULL,
  predictors = vars(established_marker, novel_marker1),
  analysisType = "comparative",
  rocCurve = TRUE,
  customCutoffs = NULL
)

5. Youden Index & Cutoff Analysis

Youden Index plot

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  youdenOptimization = TRUE,
  customCutoffs = NULL
)

Cutoff analysis plot

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  cutoffTable = TRUE,
  customCutoffs = NULL
)

Clinical decision plot

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  clinicalMetrics = TRUE,
  customCutoffs = NULL
)

6. Advanced ROC Methods

Binormal smoothing

validation_data <- read.csv(paste0(data_path2, "enhancedroc_validation.csv"))

enhancedROC(
  data = validation_data,
  outcome = "outcome",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "biomarker",
  smoothMethod = "binormal",
  customCutoffs = NULL
)

Partial AUC (high specificity region)

enhancedROC(
  data = validation_data,
  outcome = "outcome",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "biomarker",
  partialAuc = TRUE,
  partialAucType = "specificity",
  partialAucFrom = 0.8,
  partialAucTo = 1.0,
  customCutoffs = NULL
)

Partial AUC (high sensitivity region)

enhancedROC(
  data = validation_data,
  outcome = "outcome",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "biomarker",
  partialAuc = TRUE,
  partialAucType = "sensitivity",
  partialAucFrom = 0.9,
  partialAucTo = 1.0,
  customCutoffs = NULL
)

CROC analysis (Concentrated ROC)

enhancedROC(
  data = validation_data,
  outcome = "outcome",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "biomarker",
  crocAnalysis = TRUE,
  crocAlpha = 7.0,
  customCutoffs = NULL
)

Convex hull analysis

enhancedROC(
  data = validation_data,
  outcome = "outcome",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "biomarker",
  convexHull = TRUE,
  customCutoffs = NULL
)

Scores with many ties

Ordinal scores and rounded lab values produce many ties. A tied pair of patients (one positive, one negative) always counts one half in the AUC, and a note under the AUC table reports how many ties there are and the bounds the AUC could take if they were broken either way. No option is needed.

tied_data <- read.csv(paste0(data_path2, "enhancedroc_tiedscores.csv"))

enhancedROC(
  data = tied_data,
  outcome = "disease",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = vars(ordinal_score, rounded_lab, composite_score),
  customCutoffs = NULL
)

7. Sensitivity & Specificity Thresholds

Screening context: high sensitivity threshold

screening_data <- read.csv(paste0(data_path, "enhancedroc_screening.csv"))

enhancedROC(
  data = screening_data,
  outcome = "screening_indication",
  positiveClass = "Screen Positive",
  validationNegative = NULL,
  predictors = "sensitive_marker",
  sensitivityThreshold = 0.95,
  specificityThreshold = 0.5,
  customCutoffs = NULL
)

Confirmatory context: high specificity threshold

confirmatory_data <- read.csv(paste0(data_path, "enhancedroc_confirmatory.csv"))

enhancedROC(
  data = confirmatory_data,
  outcome = "confirmed_diagnosis",
  positiveClass = "Confirmed",
  validationNegative = NULL,
  predictors = "specific_marker",
  sensitivityThreshold = 0.5,
  specificityThreshold = 0.95,
  customCutoffs = NULL
)

8. Class Imbalance Detection

Imbalance detection with PRC recommendation

imbalanced_data <- read.csv(paste0(data_path, "enhancedroc_imbalanced.csv"))

enhancedROC(
  data = imbalanced_data,
  outcome = "rare_disease",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = vars(screening_marker, confirmatory_marker),
  detectImbalance = TRUE,
  imbalanceThreshold = 3.0,
  showImbalanceWarning = TRUE,
  recommendPRC = TRUE,
  customCutoffs = NULL
)

Lower imbalance threshold (more sensitive detection)

enhancedROC(
  data = imbalanced_data,
  outcome = "rare_disease",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "screening_marker",
  detectImbalance = TRUE,
  imbalanceThreshold = 1.5,
  customCutoffs = NULL
)

9. Clinical Context & Presets

Biomarker screening preset

enhancedROC(
  data = screening_data,
  outcome = "screening_indication",
  positiveClass = "Screen Positive",
  validationNegative = NULL,
  predictors = "sensitive_marker",
  clinicalContext = "screening",
  clinicalPresets = "biomarker_screening",
  customCutoffs = NULL
)

Confirmatory testing preset

enhancedROC(
  data = confirmatory_data,
  outcome = "confirmed_diagnosis",
  positiveClass = "Confirmed",
  validationNegative = NULL,
  predictors = "specific_marker",
  clinicalContext = "diagnosis",
  clinicalPresets = "confirmatory_testing",
  customCutoffs = NULL
)

Diagnostic validation preset (balanced)

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  clinicalContext = "prognosis",
  clinicalPresets = "diagnostic_validation",
  customCutoffs = NULL
)

Research comprehensive preset

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = vars(biomarker1, biomarker2, clinical_risk_score),
  clinicalPresets = "research_comprehensive",
  customCutoffs = NULL
)

Monitoring context with custom settings

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  clinicalContext = "monitoring",
  clinicalPresets = "custom",
  customCutoffs = NULL
)

10. Calibration Analysis

Full calibration assessment

calibration_data <- read.csv(paste0(data_path, "enhancedroc_calibration.csv"))

enhancedROC(
  data = calibration_data,
  outcome = "outcome",
  positiveClass = "Event",
  validationNegative = NULL,
  predictors = vars(predicted_prob, risk_score),
  calibrationAnalysis = TRUE,
  calibrationPlot = TRUE,
  brierScore = TRUE,
  calibrationMetrics = TRUE,
  customCutoffs = NULL
)

Hosmer-Lemeshow test

enhancedROC(
  data = calibration_data,
  outcome = "outcome",
  positiveClass = "Event",
  validationNegative = NULL,
  predictors = "predicted_prob",
  hosmerLemeshow = TRUE,
  hlGroups = 10,
  customCutoffs = NULL
)

Hosmer-Lemeshow with fewer groups

enhancedROC(
  data = calibration_data,
  outcome = "outcome",
  positiveClass = "Event",
  validationNegative = NULL,
  predictors = "predicted_prob",
  hosmerLemeshow = TRUE,
  hlGroups = 5,
  customCutoffs = NULL
)

11. Multi-Class ROC

One-vs-Rest with macro averaging

multiclass_data <- read.csv(paste0(data_path, "enhancedroc_multiclass.csv"))

enhancedROC(
  data = multiclass_data,
  outcome = "disease_severity",
  positiveClass = "Severe",
  validationNegative = NULL,
  predictors = vars(biomarker_A, biomarker_B, imaging_severity_score),
  multiClassROC = TRUE,
  multiClassStrategy = "ovr",
  multiClassAveraging = "macro",
  customCutoffs = NULL
)

One-vs-One with weighted averaging

enhancedROC(
  data = multiclass_data,
  outcome = "disease_severity",
  positiveClass = "Severe",
  validationNegative = NULL,
  predictors = "biomarker_A",
  multiClassROC = TRUE,
  multiClassStrategy = "ovo",
  multiClassAveraging = "weighted",
  customCutoffs = NULL
)

12. Clinical Impact & Utility

Clinical impact with NNT calculation

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  clinicalImpact = TRUE,
  nntCalculation = TRUE,
  decisionImpactTable = TRUE,
  customCutoffs = NULL
)

Clinical utility curve

enhancedROC(
  data = biomarker_data,
  outcome = "disease_status",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "biomarker1",
  clinicalUtilityCurve = TRUE,
  customCutoffs = NULL
)

13. Edge Cases

Small sample (n=60)

small_data <- read.csv(paste0(data_path, "enhancedroc_small.csv"))

enhancedROC(
  data = small_data,
  outcome = "disease",
  positiveClass = "Positive",
  validationNegative = NULL,
  predictors = "marker",
  useBootstrap = TRUE,
  bootstrapSamples = 100,
  customCutoffs = NULL
)

Low prevalence screening data

enhancedROC(
  data = screening_data,
  outcome = "screening_indication",
  positiveClass = "Screen Positive",
  validationNegative = NULL,
  predictors = "sensitive_marker",
  customCutoffs = NULL
)

Tied scores

enhancedROC(
  data = tied_data,
  outcome = "disease",
  positiveClass = "Disease",
  validationNegative = NULL,
  predictors = "ordinal_score",
  customCutoffs = NULL
)

References

  • DeLong ER, DeLong DM, Clarke-Pearson DL (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44(3), 837-845.
  • Youden WJ (1950). Index for rating diagnostic tests. Cancer, 3(1), 32-35.
  • Hanley JA, McNeil BJ (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29-36.
  • Robin X, Turck N, Hainard A, et al. (2011). pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics, 12, 77.
  • Hosmer DW, Lemeshow S (2000). Applied Logistic Regression. 2nd ed. Wiley.
  • Brier GW (1950). Verification of forecasts expressed in terms of probability. Monthly Weather Review, 78(1), 1-3.