Receiver Operating Characteristic (ROC) curve analysis with optimal cutpoint determination.
psychopdaROC(
manualRun = FALSE,
run = FALSE,
clinicalMode = "basic",
data,
dependentVars,
classVar,
positiveClass,
subGroup = NULL,
clinicalPreset = "none",
method = "maximize_metric",
metric = "youden",
direction = ">=",
specifyCutScore = "",
tol_metric = 1e-06,
break_ties = "mean",
allObserved = FALSE,
boot_runs = 0,
seed = 123,
usePriorPrev = FALSE,
priorPrev = 0.5,
costratioFP = 1,
sensSpecTable = FALSE,
showThresholdTable = FALSE,
maxThresholds = 20,
delongTest = FALSE,
plotROC = TRUE,
combinePlots = TRUE,
cleanPlot = FALSE,
showOptimalPoint = TRUE,
displaySE = FALSE,
smoothing = FALSE,
showConfidenceBands = FALSE,
legendPosition = "right",
directLabel = FALSE,
interactiveROC = FALSE,
showCriterionPlot = FALSE,
showPrevalencePlot = FALSE,
showDotPlot = FALSE,
precisionRecallCurve = FALSE,
partialAUC = FALSE,
partialAUCfrom = 0.8,
partialAUCto = 1,
rocSmoothingMethod = "none",
bootstrapCI = FALSE,
bootstrapReps = 2000,
quantileCIs = FALSE,
quantiles = "0.1,0.25,0.5,0.75,0.9",
compareClassifiers = FALSE,
calculateIDI = FALSE,
calculateNRI = FALSE,
refVar,
nriThresholds = "",
idiNriBootRuns = 1000,
effectSizeAnalysis = FALSE,
powerAnalysis = FALSE,
powerAnalysisType = "post_hoc",
expectedAUCDifference = 0.1,
targetPower = 0.8,
significanceLevel = 0.05,
correlationROCs = 0.5,
bayesianAnalysis = FALSE,
priorAUC = 0.7,
priorPrecision = 10,
clinicalUtilityAnalysis = FALSE,
treatmentThreshold = "0.05,0.5,0.05",
harmBenefitRatio = 0.25,
interventionCost = FALSE,
fixedSensSpecAnalysis = FALSE,
fixedAnalysisType = "sensitivity",
fixedSensitivityValue = 0.9,
fixedSpecificityValue = 0.9,
showFixedROC = TRUE,
fixedInterpolation = "linear",
showFixedExplanation = TRUE,
metaAnalysis = FALSE,
metaAnalysisMethod = "both",
heterogeneityTest = TRUE,
forestPlot = FALSE,
overrideMetaAnalysisWarning = FALSE
)
manualRun:
When TRUE, results are only computed after clicking the Run button. Useful for skipping intermediate recomputes while adjusting options on slow bootstrap or cutpoint analyses.
run:
.
clinicalMode:
Select the complexity level of analysis: Basic - Essential ROC metrics for clinical decision making Advanced - Additional statistical comparisons and metrics Comprehensive - Full research-grade analysis with all options
data:
The data as a data frame.
dependentVars:
Test variable(s) to be evaluated for classification performance. Multiple variables can be selected for comparison.
classVar:
Binary classification variable representing the true class (gold standard). Must have exactly two levels.
positiveClass:
Specifies which level of the class variable should be treated as the positive class.
subGroup:
Optional grouping variable for stratified analysis. ROC curves will be calculated separately for each group.
clinicalPreset:
Choose a preset configuration optimized for specific clinical scenarios: Screening - High sensitivity to avoid missing cases Confirmation - High specificity to avoid false positives Balanced - Equal weight to sensitivity and specificity Research - Comprehensive analysis for publication
method:
Method for determining the optimal cutpoint. Different methods optimize different aspects of classifier performance.
metric:
Metric to optimize when determining the cutpoint. Only applies to maximize/minimize methods.
direction:
Direction of classification relative to the cutpoint. Use ‘>=’ when higher test values indicate the positive class.
specifyCutScore:
Specific cutpoint value to use when method is set to ‘Manual cutpoint’.
tol_metric:
Cutpoints whose metric is within this tolerance of the best value are treated as equivalent and averaged together. BEHAVIOUR CHANGE (1.0.4) - the default was 0.05, which is 50000 times wider than the 1e-06 used by the underlying cutpointr package and meant the reported “optimal” cutpoint was usually not the one maximising the metric. On the bundled example data 39 of 200 candidate thresholds, spanning 52.1 to 67.7, fell within 0.05 Youden of the optimum and were averaged, giving a cutpoint with 84.5 percent sensitivity in place of the 94.4 percent available at the true optimum. Raise it deliberately if you want that averaging as a guard against over-fitting one threshold.
break_ties:
Method for handling ties when multiple cutpoints achieve the same metric value.
allObserved:
Display performance metrics for all observed test values as potential cutpoints, not just the optimal cutpoint.
boot_runs:
Number of bootstrap iterations for methods using bootstrapping. Set to 0 to disable bootstrapping.
seed:
Random seed for reproducibility of bootstrap and permutation tests.
usePriorPrev:
Use a specified prior prevalence instead of the sample prevalence for calculating predictive values.
priorPrev:
Population prevalence to use for predictive value calculations. Only used when ‘Use Prior Prevalence’ is checked.
costratioFP:
Relative cost of false positives compared to false negatives. Values > 1 penalize false positives more heavily.
sensSpecTable:
Display detailed confusion matrices at optimal cutpoints.
showThresholdTable:
Display detailed table with performance metrics at multiple thresholds.
maxThresholds:
Maximum number of threshold values to show in the threshold table.
delongTest:
Test whether the diagnostic performance differs significantly between multiple tests. Uses DeLong’s method to compare Area Under the Curve (AUC) values. Requires at least two test variables.
plotROC:
Display ROC curves for visual assessment of classifier performance.
combinePlots:
When multiple test variables are selected, combine all ROC curves in a single plot.
cleanPlot:
Create clean ROC curves without annotations, suitable for publications.
showOptimalPoint:
Display the optimal cutpoint on the ROC curve.
displaySE:
Display standard error bands on ROC curves (when LOESS smoothing is applied).
smoothing:
Apply LOESS smoothing to ROC curves for visualization.
showConfidenceBands:
Display confidence bands around the ROC curve.
legendPosition:
Position of the legend in plots with multiple ROC curves.
directLabel:
Label curves directly on the plot instead of using a legend.
interactiveROC:
Create an interactive HTML ROC plot (requires plotROC package).
showCriterionPlot:
Plot showing how sensitivity and specificity change across different thresholds.
showPrevalencePlot:
Plot showing how PPV and NPV change with disease prevalence.
showDotPlot:
Dot plot showing the distribution of test values by class.
precisionRecallCurve:
Display precision-recall curves alongside ROC curves.
partialAUC:
Calculate AUC for a specific region of the ROC curve.
partialAUCfrom:
Lower bound of specificity range for partial AUC calculation.
partialAUCto:
Upper bound of specificity range for partial AUC calculation.
rocSmoothingMethod:
Method for smoothing the ROC curve (requires pROC package).
bootstrapCI:
Calculate bootstrap confidence intervals for AUC and optimal cutpoints.
bootstrapReps:
Number of bootstrap replications for confidence interval calculation.
quantileCIs:
Display confidence intervals at specific quantiles of the test variable.
quantiles:
Comma-separated list of quantiles (0-1) at which to display confidence intervals.
compareClassifiers:
Perform comprehensive comparison of classifier performance metrics.
calculateIDI:
Calculate how much better one test is at discriminating between diseased and healthy patients. IDI (Integrated Discrimination Improvement) measures the average improvement in predicted probabilities.
calculateNRI:
Calculate how many patients are correctly reclassified when using a new test. NRI (Net Reclassification Index) measures the net improvement in patient classification.
refVar:
Reference test variable for IDI and NRI calculations. Other variables will be compared against this reference.
nriThresholds:
Comma-separated probability thresholds (0-1) defining risk categories for NRI. Leave empty for continuous NRI.
idiNriBootRuns:
Number of bootstrap iterations for IDI and NRI confidence intervals.
effectSizeAnalysis:
Calculate effect sizes for ROC curve differences using Cohen’s conventions and standardized mean differences between AUC values.
powerAnalysis:
Perform statistical power analysis for ROC curve comparisons including sample size estimation and power calculations for detecting AUC differences.
powerAnalysisType:
Type of power analysis to perform.
expectedAUCDifference:
Expected difference in AUC values for power calculations and sample size estimation.
targetPower:
Target statistical power for sample size calculations (typically 0.8 or 0.9).
significanceLevel:
Type I error rate for power calculations (typically 0.05).
correlationROCs:
Expected correlation between paired ROC curves for power calculations. Use 0.5 for moderate correlation, 0.0 for independent samples.
bayesianAnalysis:
Perform bootstrap-based ROC analysis with optional prior weighting to estimate uncertainty in AUC. Uses bootstrap resampling to create an empirical distribution. NOTE: This is NOT full Bayesian MCMC inference; it uses bootstrap simulation with prior parameters as weights. Interpret “credible intervals” as bootstrap percentile confidence intervals.
priorAUC:
Prior belief about AUC value for Bayesian analysis (center of prior distribution).
priorPrecision:
Precision of prior belief (higher values = more confident prior).
clinicalUtilityAnalysis:
Perform clinical utility analysis including net benefit curves, decision curve analysis, and clinical impact assessment.
treatmentThreshold:
Treatment threshold range for decision curve analysis (min,max,step). Example: “0.05,0.5,0.05” creates thresholds from 5 percent to 50 percent in 5 percent steps.
harmBenefitRatio:
Ratio of harm from unnecessary treatment to benefit from necessary treatment. Lower values favor more aggressive treatment policies.
interventionCost:
Include cost-effectiveness considerations in clinical utility analysis.
fixedSensSpecAnalysis:
Determine cutoffs based on fixed sensitivity or specificity values and display corresponding performance metrics.
fixedAnalysisType:
Choose whether to fix sensitivity or specificity value for cutoff determination.
fixedSensitivityValue:
Target sensitivity value (0-1) for determining the corresponding cutoff and specificity.
fixedSpecificityValue:
Target specificity value (0-1) for determining the corresponding cutoff and sensitivity.
showFixedROC:
Display separate ROC curve highlighting the fixed sensitivity/specificity point.
fixedInterpolation:
Method for interpolating between observed points to achieve target sensitivity/specificity.
showFixedExplanation:
Display explanatory guide for fixed sensitivity/specificity analysis including clinical interpretation, interpolation methods, and usage recommendations.
metaAnalysis:
Perform meta-analysis of AUC values across multiple test variables. Requires at least 3 test variables to enable pooled effect estimation.
metaAnalysisMethod:
Statistical method for combining AUC estimates across studies/variables.
heterogeneityTest:
Perform Cochran’s Q test and calculate I² statistic to assess heterogeneity between AUC estimates.
forestPlot:
Create forest plot visualization of individual and pooled AUC estimates with confidence intervals.
overrideMetaAnalysisWarning:
ADVANCED OPTION: Bypass the independence assumption check for meta-analysis. WARNING - Only use if you fully understand the statistical implications. Meta-analysis on non-independent data produces invalid results and should NOT be used for formal inference. Use DeLong’s test instead for within-study comparisons of multiple markers.
A results object containing:
results$instructions |
a html | ||||
results$procedureNotes |
a html | ||||
results$runSummary |
a html | ||||
results$simpleResultsTable |
a table | ||||
results$clinicalInterpretationTable |
a table | ||||
results$resultsTable |
an array of tables | ||||
results$sensSpecTable |
an array of htmls | ||||
results$thresholdTable |
a table | ||||
results$fixedSensSpecTable |
a table | ||||
results$fixedSensSpecExplanation |
a html | ||||
results$aucSummaryTable |
a table | ||||
results$delongComparisonTable |
a table | ||||
results$delongTest |
a preformatted | ||||
results$plotROC |
an array of images | ||||
results$interactivePlot |
an image | ||||
results$fixedSensSpecROC |
an array of images | ||||
results$criterionPlot |
an array of images | ||||
results$prevalencePlot |
an array of images | ||||
results$dotPlot |
an array of images | ||||
results$dotPlotMessage |
a html | ||||
results$precisionRecallPlot |
an array of images | ||||
results$idiTable |
a table | ||||
results$nriTable |
a table | ||||
results$effectSizeTable |
a table | ||||
results$powerAnalysisTable |
a table | ||||
results$bayesianROCTable |
a table | ||||
results$clinicalUtilityTable |
a table | ||||
results$metaAnalysisWarning |
a html | ||||
results$metaAnalysisTable |
a table | ||||
results$decisionCurveTable |
a table | ||||
results$partialAUCTable |
a table | ||||
results$bootstrapCITable |
a table | ||||
results$rocComparisonTable |
a table | ||||
results$effectSizePlot |
an array of images | ||||
results$powerCurvePlot |
an array of images | ||||
results$bayesianTracePlot |
an array of images | ||||
results$decisionCurvePlot |
an array of images | ||||
results$metaAnalysisForestPlot |
an array of images |
Tables can be converted to data frames with asDF or
as.data.frame. For
example:
results$simpleResultsTable$asDF
as.data.frame(results$simpleResultsTable)