ROC SDK v1.20 delivers top-tier accuracy and unparalleled efficiency in latest NIST 1:N report

The ROC SDK v1.20 was released in Nov. 2019, and delivered wide ranging improvements, including major accuracy improvements. This algorithm delivers top-tier accuracy alongside industry leading efficiency according to the latest NIST FRVT 1:N report released a few weeks ago. Though we are only a few weeks away from the release of our next ROC SDK algorithm (which is poised to deliver another round of meaningful accuracy improvements) it is worth analyzing the value of the combined accuracy and efficiency of the ROC SDK v1.20 algorithm. 

While the frequently updated FRVT Ongoing 1:1 report measures accuracy when performing identity comparison (i.e., answering whether two face images are the same person or different persons), the FRVT 1:N report measures accuracy when performing identification (i.e., given a photo of a person and an identity database, answering who the person is). The analysis in this article is related more specifically to open-set identification, where an automated threshold is applied to the search results. 

The following table compares the accuracy and efficiency of the highest accuracy algorithm submitted by leading Western face recognition vendors benchmarked in the Feb. 2020 NIST FRVT 1:N report:

Screen Shot 2020-03-20 at 3.32.24 PM

Rank One scores as one of the most accurate algorithms available and delivers a level of efficiency that is multiple factors better than the vendors such as the Nippon Electric Company (NEC), Microsoft, EverAI / Paravision, and Idemia. 

This combination of top-tier accuracy and unparalleled efficiency makes the Rank One algorithm uniquely qualified to power certain applications, such as real-time screening, video surveillance, mobile identification, and enterprise search, where other algorithms would be onerous, if not impossible, to deploy. 

Real-time Screening / Video Surveillance

Real-time screening and video surveillance applications both involve constant template generation from real-time feeds of streaming video. The relative difference in enrollment speed directly scales to the difference in CPU hardware required. Thus, the slower the template generation speed the more hardware that will be required, which means most of the other vendors require 2x to 3x more CPU hardware to process video streams.

Mobile and Embedded Identification

Mobile ARM processors generate templates roughly 3x slower than an x64 server processor (such as the one used by NIST). For example, an algorithm with a 700ms template generation speed on a server would take over 2s on a mobile processor. Mobile enrollment for nearly all of Rank One’s competitors’ would be slower than 1s, which is too slow for embedded identification and access control applications.

For mobile and embedded applications that involve identification (searching a probe template against a gallery of templates), template size plays a critical factor as templates need to be loaded into computer memory in order to be compared against during search. While sublinear search strategies could remove the need to load all of the templates into RAM, none of the Western friendly vendors analyzed above offer sublinear search. Because template size is directly proportional to the amount of RAM required for each algorithm to search a database, most of Rank One’s competitors require over 10x the amount RAM to enable identification applications. In the context of 1:N search applications on mobile processors, this additional RAM may simply not exist.

Enterprise Search

Similar to mobile search, template size directly impacts the amount of RAM required to host an enterprise search application, such as a forensic face recognition search system. When hosting large search databases as required in most enterprise systems, the hardware cost could surge when using inefficient algorithms. 

Another critical consideration for enterprise search systems is template generation speed. This consideration is especially important given the staggering rate of improvement in face recognition accuracy. For example, the following chart illustrates the 30x reduction in error rate of the Rank One algorithm on unconstrained imagery over the last 4 years (from 60% to 2% error), which is similar to the reduction in error rate from other other industries leaders over this period of time.Face recognition error rates on unconstrained imagery vs. time (3).pngWith such rapid decreases in error rate, large scale search systems should be updated on a regular basis, which requires regenerating all of the gallery templates. Algorithms that have an onerous template generation speed will make this process expensive and difficult. Furthermore, certain vendors do not provide new algorithm versions without purchasing a new license, meaning their customers do not benefit from the rapid increases in facial recognition accuracy without an additional license purchase. It is critical, therefore, to obtain evergreen licensing rights when procuring a face recognition system or it may be too expensive to upgrade despite the cutting edge improvements steadily being delivered across the industry.

Circling back to efficiency, let’s consider the hardware requirements and throughput for a 12M template search system:

Screen Shot 2020-03-24 at 4.08.30 PM

As shown in the above table, both higher and lower accuracy algorithms often have onerous hardware requirements. Contrast these with the Rank One algorithm, which requires nearly 25x less RAM and 3x less CPUs than, for example, the EverAI / Paravision algorithm while having a 1% difference in error rate on a 12M image database. It is clear that significantly different value propositions exist for these algorithms. And, while top-tier accuracy is already being delivered with ROC SDK v1.20 using a fraction of the hardware as other solutions, the next version of the ROC SDK is only a few weeks away, and, because of its efficiency, Rank One’s partners will be able to rapidly upgrade their search database and unlock the enhanced accuracy of this soon-to-be-released version. 

Summary 

The latest NIST FRVT 1:N report demonstrates the unique combination of accuracy and efficiency delivered by the ROC SDK. This versatility has resulted in Rank One powering a wide range of applications and systems across different industries such as defense, law enforcement, ID proofing, finance, medical, security, retail, and consumer electronics. Couple these performance metrics with transparent licensing agreements and the trust that comes from an employee-own company based in Denver, Colorado, U.S.A., and it is clear why Rank One’s user community continues to grow at a rapid pace. 

Contact us to learn more about the ROC SDK v1.20, as well as our other solutions and services, and stay tuned in the weeks ahead for the improvements that will be delivered in the new ROC SDK version.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s