A new metric now mandatory for PAD product certification highlights the impact of false spoof alarms on biometric system performance
Presentation attack detection (PAD) determines whether a biometric sample is of a live and present human or a spoof. It is used in facial recognition systems to prevent bad actors from representing a printed photo or an image on a screen as the face of a live person to commit fraud.
The ISO/IEC 30107 standard specifies various aspects of biometric PAD technology, such as reference architectures and terminology. Part 3 of the standard specifies “testing and reporting”, with methods and metrics for evaluation of PAD product performance used by accredited labs to perform product certifications. Many companies highlight the “certified” status of their PAD products.
A product being certified helps buyers know that it will reliably detect spoofs without overburdening users with false positives. The friction caused by false positives can have a significant impact on customer satisfaction and ultimately the bottom line for service providers, as illustrated in a recent case study showing the financial impact of replacing active liveness with passive liveness on a bank’s customer onboarding processes.
The 2023 revision of ISO 30107 includes new metrics that provide better insight into the real-world performance of a complete biometric system. One new metric is called “RIAPAR”1, used to measure how well a biometric system detects attacks without interrupting legitimate users. RIAPAR is essentially a sum of two error rates: 1) false accepts that make the system more vulnerable to attack, and 2) false rejects that make the system less convenient for legitimate users. The change is notable because it will make it harder to certify products as secure without revealing the impact of false rejects. This article explains RIAPAR and how it relates to other metrics.
PAD metrics refresh: APCER and BPCER, security vs. convenience
The PAD standard provides metrics called APCER and BPCER2, which are used to measure the security and convenience of PAD. They are analogous to false match and false non-match rates of a biometric matcher. A low APCER rate means that security is strong; fewer attacks go undetected. A low BPCER means a low rate of false positives that cause inconvenience for legitimate users.
APCER and BPCER are core to the PAD standard, but either one measured on their own can be misleading. This is because they are dependent on a decision threshold that can be set to optimize for one or the other in a way that may not be viable in a real deployment. A threshold that is too high can deliver strong security but lots of false alarms for legitimate users. A threshold that is too low might be convenient but lacks sufficient security. Setting the threshold to optimize for either APCER or BPCER in testing makes it difficult to assess how the technology will work in a real-world environment, where APCER and BPCER must both be optimized. The 2023 revision recognizes this by introducing a single metric that takes both into account for a given threshold.
Measuring performance of a biometric system: IAPAR and RIAPAR
IAPAR3 is a metric that measures the likelihood of a successful attack on a complete biometric system comprised of 1) data capture, 2) biometric matcher and/or 3) PAD subsystems. This is important because while implementers need to make decisions on sourcing and deployment of the individual components of a system, it is useful to understand how components behave in concert when under attack. How do the behaviors of the subsystems correlate to affect the overall performance?
RIAPAR is a new mandatory metric introduced with the 2023 version of the 30107 standard that reflects how a biometric system behaves in terms of both security and convenience. It does so by adding IAPAR (security) to the false reject rate, or FRR (convenience). Adding RIAPAR as a mandatory metric is a positive development for buyers and implementers of biometric solutions by creating an incentive to perform testing at a threshold that optimizes for both security and convenience for a target use case. The same way that APCER and BPCER should always be considered together (for the same decision threshold) in order to understand both the security and convenience of the system, RIAPAR provides a way to capture this concept in a single metric.
Figure: Metrics to evaluate a biometric system and its subcomponents
It is helpful to recognize that for biometric systems in production, the FRR is typically much larger than IAPAR by design. That is to say that the “inconvenience rate”—the frequency with which legitimate user are rejected—is far larger than the frequency with which attackers are able to spoof the system.
Furthermore, the FRR is significantly impacted by the BPCER. That is to say that the false spoof alarms are a significant contributor to false rejects of the whole biometric system as compared to the rate of false non-matches caused by biometric comparison.
Consequently, the RIAPAR metric is impacted significantly by the BPCER of the PAD algorithm, and so the update encourages the use of PAD subsystems with lower BPCER as well as data capture subsystems with lower FTE (failure to enroll) metrics.
The importance of low BPCER
A frictionless approach to liveness detection—in contrast to an “active” approach that interrupts the user experience or demands the user to perform tasks—has been shown to significantly decrease BPCER. By providing a passive liveness detection solution, it eliminates the need for user instructions and allows for seamless user interactions, leading to a more convenient and reliable biometric recognition system. It also avoids providing clues to fraudsters as to how to defeat the mechanism.
An illustration of RIAPAR calculation
The following figure illustrates the evaluation of a biometric system using 10,000 mated bona fide biometric samples and 1,000 impostor presentation attacks that are purposefully matched (mated) with their corresponding targets.
- For this example, we assume perfect data capture with no failures to acquire (FTA).
- The PAD subsystem rejects 420 of our 10,000 bona fide samples for a BPCER of 4.20%, and accepts 4 of 1,000 impostor attacks as live for an APCER of 0.4%.
- Of the 9,580 remaining bona fide samples that pass to the matching system, 5 are rejected as non-biometric matches for a false non-match rate (FNMR) of 0.052%. 3 of our 4 impostors successfully match for an IAPAR of 0.3%.
- The false reject rate considers false rejections from both systems, which in this case is 420 + 5, or 425 of 10,000 for a FRR of 4.25%
- The RIAPAR considers the rates of successful impostors and unsuccessful bona fides, adding 0.3% to 4.25% for a RIAPAR of 4.55%.
Using the RIAPAR metric helps ensure that the artificial optimization of a PAD system–achieving “perfect” security at the expense of convenience–will be reflected in the RIAPAR, which includes the BPCER.
FRR = BPCER + FNMR = (420+5)/10,000 = 4.25%
IAPAR = 3/1,000 = 0.3%
RIAPAR = FRR + IAPAR = 4.25% + 0.3% = 4.55%
Certification example: ID R&D’s IDLive Face
ID R&D previously tested its IDLive Face product for presentation attack detection against the ISO 30107 standard through iBeta, an accredited testing laboratory. ID R&D achieved perfect scores at Level 2, the highest level, rejecting zero genuine users and allowing zero impostors to get through. Level 2 includes 3D masks. ID R&D also allowed zero impostors through when tested at Level 1. Level 1 includes photos and video spoofing attacks.
End-to-end systems that use IDLive Face will score very well against this new ISO testing standard. This is because IDLive Face requires the least amount of user effort to check liveness; simply take a selfie. This leads to the least amount of genuine rejections. Experience shows that systems using active liveness will likely not perform as well when measured against this new standard.
_________________________________________________
1 RIAPAR stands for relative impostor attack presentation accept rate.
2 APCER and BPCER stand for attack presentation classification rate and bona fide presentation classification error rate, respectively.
3 IAPAR stands for imposter attack presentation accept rate (formerly IAPMR – impostor attack presentation match rate)