Diagnostic Metrics, Anchoring Perspectives, and Curve Interpretations #

This guide summarizes the core diagnostic metrics based on anchoring logic (condition vs. prediction), and how these metrics relate to ROC and PR curves — especially under balanced vs. imbalanced class distributions.

🔹 Test-Centric Metrics (Anchored on Actual Condition) #

• Evaluates test performance, independent of disease prevalence.
• Anchor: Ground truth label (actual condition).

Positive-Focused (Sensitivity) #

• Fix actual Positive label.
• Incorrect prediction: False Negative (FN).
• Pair: (TP, FN)
• Sensitivity = TP / (TP + FN)

Negative-Focused (Specificity) #

• Fix actual Negative label.
• Incorrect prediction: False Positive (FP).
• Pair: (TN, FP)
• Specificity = TN / (TN + FP)

🔸 Outcome-Centric Metrics (Anchored on Prediction) #

• Evaluates usefulness of test result, dependent on both test performance and prevalence.
• Anchor: Test result (prediction output).

Positive-Focused (PPV / Precision) #

• Fix Positive prediction.
• Incorrect prediction: False Positive (FP).
• Pair: (TP, FP)
• Positive Predictive Value (PPV) = TP / (TP + FP)

Negative-Focused (NPV) #

• Fix Negative prediction.
• Incorrect prediction: False Negative (FN).
• Pair: (TN, FN)
• Negative Predictive Value (NPV) = TN / (TN + FN)

📊 Extension to ROC and PR Curves #

🎯 ROC Curve (Receiver Operating Characteristic) #

What it does: #

• Plots True Positive Rate (TPR) vs. False Positive Rate (FPR) across thresholds.
  • TPR = Sensitivity = TP / (TP + FN)
  • FPR = 1 − Specificity = FP / (FP + TN)
• These metrics are calculated by conditioning on the actual class labels, not predictions.

Anchoring View: #

• ✅ Test-Centric / Condition-Anchored
• Starts from actual condition and evaluates how well the test distinguishes between classes.
• Independent of class imbalance in its calculation.

Use Case: #

• Suitable when both positive and negative classes are equally important.
• Can be misleading in highly imbalanced datasets (e.g., rare disease).

📈 Precision-Recall (PR) Curve #

What it does: #

• Plots Precision (PPV) vs. Recall (Sensitivity) across thresholds.
  • Precision = TP / (TP + FP)
  • Recall = TP / (TP + FN)

Anchoring View: #

• ✅ Outcome-Centric / Prediction-Anchored
• Focuses on the model’s positive predictions and how often they are correct.
• Particularly useful for evaluating performance on the positive class in imbalanced datasets.

Use Case: #

• Ideal for problems with class imbalance, where the positive class is rare but important (e.g., cancer detection, fraud, anomaly detection).
• Answers: “When the model says positive, can I trust it?”

🧠 Summary of Metric Anchors and Curve Use #

💡 Takeaway: #

• ROC Curve is a test-centric (condition-anchored) tool: great for balanced data, focuses on test performance across thresholds.
• PR Curve is an outcome-centric (prediction-anchored) tool: best for imbalanced data, reflects how reliable positive predictions are.