In this blog post, we explain how to understand and evaluate the effectiveness of biometric systems using ROC curves.

Why Is Biometric Authentication Necessary?

As the digital age has progressed, many industries—including finance—have integrated with IT, leading to the widespread adoption of services such as mobile banking and app-based cards. Strong security is essential for these financial services, and the demand for personal data protection has grown due to a series of data breaches.
While complex passwords combining numbers, letters, and special characters, or login methods based on digital certificates, are effective from a security perspective, they have limitations in terms of usability (speed and convenience). With the widespread adoption of wearables such as smartwatches, biometric technology—which utilizes “features unique to the individual”—is gaining attention.

Key Characteristics of Biometrics

The fundamental properties expected of biometrics are universality (everyone possesses these features), uniqueness (they differ from person to person), and permanence (they do not change significantly over time). Fingerprints, irises, and blood vessels meet these conditions and are widely used.
However, setting recognition criteria too strictly can lead to false negatives (rejecting the actual user), while setting them too loosely increases false positives (incorrectly identifying someone else as the user). The ROC curve is useful for evaluating this balance.

The Meaning of Sensitivity and Specificity (or TPR and FPR)

Sensitivity (TPR) indicates how well the system recognizes the actual person as “themselves.” For example, if the fingerprint recognition criteria are too loose so that anyone is recognized as the person, the sensitivity is high.
Specificity (FPR) indicates how well the system distinguishes actual strangers as “strangers.” In the previous example, if everyone is recognized as the owner, the system fails to identify strangers, resulting in low specificity. Sensitivity and specificity are distinct concepts from accuracy.

What is an ROC curve?

An ROC (Receiver Operating Characteristic) curve is a graph that plots sensitivity (TPR) on the y-axis and the false positive rate (FPR = 1 − specificity) on the x-axis, showing how performance changes as the threshold is adjusted. Both TPR and FPR take values between 0 and 1.
Lowering the threshold results in more samples being classified as “genuine,” causing both TPR and FPR to increase; conversely, raising the threshold tends to decrease both, resulting in a curve that reflects these changes in the threshold.

Creating a ROC Curve Using a Real-World Example

The following are the results of 10 simulated fingerprint matching experiments. Each run shows the fingerprint match rate (%) and the actual status (true positive O or false negative X): 35% (X), 41% (X), 47% (X), 53% (X), 62% (O), 67% (X), 74% (O), 83% (X), 91% (O), 95% (O).

In reality, the true owner was identified 4 times (O), and a non-owner was identified 6 times (X).
Now, let’s generate prediction results by applying various thresholds and calculate the TPR and FPR for each threshold. Here, a “predicted positive” refers to a match rate that meets or exceeds the threshold.
Threshold ≥35%: Predicted positives are all samples (10) → TP=4, FP=6 → TPR=4/4=1.0, FPR=6/6=1.0
Threshold ≥41%: Predicted positives are 9 (excluding 35%) → TP=4, FP=5 → TPR=1.0, FPR=5/6≈0.833
Threshold ≥47%: 8 predicted positives → TP=4, FP=4 → TPR=1.0, FPR=4/6≈0.667
Threshold ≥53%: 7 predicted positives → TP=4, FP=3 → TPR=1.0, FPR=3/6=0.5
Threshold ≥62%: 6 predicted positives → TP=4, FP=2 → TPR=1.0, FPR=2/6 ≈ 0.333
Threshold ≥67%: 5 predicted positives → TP=3, FP=2 → TPR=3/4=0.75, FPR=2/6 ≈ 0.333
Threshold ≥74%: 4 predicted positives → TP=3, FP=1 → TPR=0.75, FPR=1/6 ≈ 0.167
Threshold ≥83%: 3 predicted positives → TP=2, FP=1 → TPR=0.5, FPR ≈ 0.167
Threshold ≥91%: 2 predicted positives → TP=2, FP=0 → TPR=0.5, FPR=0
Threshold ≥95%: 1 predicted positive → TP=1, FP=0 → TPR=0.25, FPR=0
Connecting these (FPR, TPR) points yields an approximate ROC curve. If the sample size is large enough, this curve appears smoother.

ROC Interpretation and AUC

In ROC analysis, the ideal performance is FPR=0, TPR=1—that is, the top-left point (0,1). The closer the point is to this location, the more it signifies “detecting all true positives and never misclassifying a negative as a positive.”
AUC (Area Under the Curve) refers to the area under the ROC curve; the larger the value, the better the classifier’s performance. An AUC of 0.5 corresponds to random guessing, and the closer the AUC is to 1, the closer the performance is to perfection.
The general interpretation of AUC is as follows: non-informative (AUC = 0.5), poor accuracy (0.5 < AUC ≤ 0.7), moderate accuracy (0.7 < AUC ≤ 0.9), very accurate (0.9 < AUC ≤ 1), and perfect (AUC = 1).

Other Considerations Regarding Biometric Effectiveness and Conclusion

ROC and AUC are very useful for comparing different biometric techniques or settings, as they provide a clear, at-a-glance view of the balance between sensitivity and specificity at a given threshold. However, the effectiveness of a system is determined by various other factors, including the probability of correctly acquiring biometric data, the performance of algorithms that accurately process the acquired data, processing costs and speed, and user convenience.
In addition to fingerprints, irises, and blood vessels, features that do not rely on specific body parts—such as behavior-based biometrics—can also be utilized for biometric authentication. When comparing various candidate technologies, using ROC curves and AUC can help in selecting the relatively more accurate technology.