The ROC curve compares true positive rate against false positive rate as the threshold moves. Points closer to the top-left mean more sick cases are caught with fewer false alarms. The dot shows the current slider threshold.
The precision-recall curve compares how many positive calls are correct against how many sick cases are found. It is especially sensitive to prevalence. The dot shows the current threshold's precision and recall.
Prevalence is the fraction of people in the population who actually have the disease.
Changes: PPV (precision), NPV, accuracy, and the PR curve — these depend on how many sick vs. healthy cases are in the pool.
Unchanged: Sensitivity, specificity, and the ROC curve — calculated within each group separately, unaffected by disease frequency.
A test with high sensitivity and specificity can still have poor PPV if the disease is rare, because most positives will be false alarms.
A disease affects 1 in 5 people. A model gives higher scores to patients who are more likely to be sick.
What threshold would you choose if the clinic wants to catch most sick people, even if some healthy people get flagged?
A hospital can only follow up with 100 patients per day. The model scores 1,000 patients each morning.
Should the cutoff be fixed, like "flag everyone above 80," or capacity-based, like "flag the top 100 scores"?
A model was trained in a hospital where 30% of patients had the disease. Now it is used in a primary care clinic where only 5% have it.
Should the threshold stay the same?
The healthy or sick score is the model's confidence that a patient is healthy or sick. Accuracy and AUC/ROC are two metrics used to grade that model.
Accuracy changes with the threshold because it measures how many classifications are correct at one chosen threshold. It is useful for picking or evaluating a cutoff, but it can be misleading if prevalence or false-positive and false-negative costs are uneven.
AUC/ROC is constant across thresholds because it measures how well the score ranks sick cases above healthy cases across all possible thresholds. It is a discrimination metric.