Facial recognition is rapidly becoming ubiquitous in our daily lives. From mobile devices to health kiosks, more and more consumers are interacting with facial recognition technology. As the technology becomes more readily available through rapid breakthroughs in AI technology, the use cases continue to expand and you may find yourself exploring your own implementation of a facial recognition solution.

When choosing a facial recognition solution, there are many different factors you have to consider. While each implementation is unique and requires a comprehensive approach, the factors for a successful implementation largely remain unchanged regardless of the implementation. Below, you will find our guide to ensuring a successful implementation of a facial recognition solution.

Facial Recognition is Having Its Moment

In recent years, biometric models, especially facial recognition, have benefited greatly of scientific advancements from Artificial Intelligence (AI), dramatically increasing their accuracy while addressing multiple sources of bias from factors such as skin complexion, age, gender, ethnicity, and even inadequate lighting. Significant improvements on power, cost and size of edge-based hardware have made the technology available to a broadening range of use cases, across multiple industries and for customers of all sizes. With many choices and more options available each month, the question is not: “Can facial recognition be used in your case?” but “What is the best facial recognition solution for your use case?”

Recent Breakthrough and Outlook of Facial Recognition

The recent advances in facial recognition expertise and productization have fueled a fertile innovation cycle across the entire tech industry. Far from slowing down, progress has been accelerating over the past couple of years.

Increased Performance

The advancement of both facial recognition software and available hardware have amplified the speed and power at which facial recognition engines can operate today. Now capable of detecting and identifying tens and sometimes hundreds of faces simultaneously, facial recognition has become a critical security asset for large public spaces such as malls, department stores, airports and hospitals, where many individuals may enter simultaneously.

Decreased Cost and Power Requirements

As this technology has continued to be innovated and refined, the costs of purchasing and implementing facial recognition has decreased notably. Thanks to a variety of market offerings with software and hardware bundles tailored to specific use cases, businesses and individuals interested in employing facial recognition have many options available. Similarly, as the technology has advanced, power requirements for operation have lessened, making it available in more settings and with lower recurring costs.

Smaller Form

The hardware components needed to run facial recognition are not only seeing dramatic performance improvements but their size keeps shrinking, relaxing an important constraint. Now, facial recognition systems are available in forms as small as an individual electronic door lock.

Public Health Assisted Integrations

As a result of the COVID-19 health crisis, the better facial recognition offerings now come equipped with health tech integrations such as mask detection, facial recognition for individuals wearing masks and social distance monitoring.

The most recent case studies:

Watch Health add-on intro video:

Multiplication of Facial Recognition Use Cases

Advances that have allowed the technology to become better performing and more affordable have boosted its applicability and allowed for use cases for facial recognition to grow immensely. Where performance has been tested and the value to end-users demonstrated, hesitation quickly dissipates and we see adoption increase . Think of the iPhone Face ID, but also the CLEAR kiosks that are now making their way beyond airport security, into train stations and granting express access to spectator sport and concert venues.

Security and Access Control

As the most accurate & secure biometric technology available on the market, facial recognition has become a critical access security tool, being used to protect secure spaces such as offices, hospitals, banks, warehouses, smart homes and more. Facial recognition provides numerous advantages to legacy access control systems which traditionally use keycards or codes to facilitate entry to restricted areas, by eliminating any possibility of lost, stolen or traded access credentials.

User Experience Enhancements and Personalization

The retail and hospitality industries have become early adopters of facial recognition, leveraging the technology to provide more seamless experiences for both customers and employees. At hotels using facial recognition, guests have the ability to gain access to their rooms and other on-site amenities without plastic keycards, and employees can easily clock in and out simply by entering or exiting facilities through doors equipped with facial recognition cameras. In retail settings, stores can actively monitor for any block-listed individuals or shoplifters while also providing seamless loyalty reward experiences for members without the need for verbally communicating personal information such as phone numbers or email addresses at the register. Similarly, personalized digital signage and marketing opportunities are arising as a result of growing facial recognition adoption.

Identity Verification

The Banking, Financial Services and Insurance (BFSI) sector has similarly begun integrating facial recognition into its online banking portals, mobile apps and bank kiosks to provide a high level of security for customers prior to authenticating transactions and other restricted account activities. The high accuracy & security afforded by facial recognition, combined with adjacent technologies that enable the validation of photo ID (e.g., driver’s license or passports) allows any bank or financial service employee to validate their customers’ identity instantly. Furthermore, customers can also validate their identity on their own when using an automated kiosk or even at home through their mobile device. The benefits are obvious, when compared to the traditional tedious checks that typically accompany manual identity validation, such as requesting driver’s license, social security information or confirming numerous security questions which may easily be sidestepped. Even the dual authentication measures often in place today to validate online customers’ identity, such as captchas, tokens or authenticator apps don’t come close to the precision and streamlined experience delivered by facial recognition.

The multiplication of use cases, each with their own sets of needs and constraints, lead to a slightly different question than the one proposed in the opening paragraph: “What is the best facial recognition solution for your unique needs?” As implied in the question, there is not one right answer, but as many answers as there are specific use cases. The remainder of this article will provide a framework aimed at helping you finding YOUR answer.

The Three Facial Recognition Configurations

In its simplest expression, facial recognition typically operates in one of three potential models:

Edge-Based / Single Location
The edge-based, single location model assumes a local server or database (not on the cloud) with which the facial recognition system communicates to verify identification. The edge operation allows for a fast, highly secure process, as the system does not have to communicate with the cloud or any external device. This model is typically used by smaller businesses or, at the other end of the spectrum, to create a highly secure closed-loop system and protect highly valuable assets, for example a bank vault.
Edge-Based / Multi Locations
Facial recognition systems can also operate at the edge while communicating with a central cloud server. This workflow optimizes speed for operators with systems covering multiple locations where a cloud database is necessary.
Cloud-Based workflow
The cloud-based model requires the facial recognition engine to communicate with the cloud to verify identification. This can incrementally decrease processing speed and security as the system is forced to send data back and forth to the cloud continuously.

Key Components

An operational facial recognition system consists of the following components:

The Decision Tree – Navigate the Best Facial Recognition Solution

To build the best facial recognition solution for your unique scenario, you need to consider these key factors:

  1. Use case – what is the primary job you’re implementing facial recognition to do?
  2. Needs and constraints – what specific characteristics does it require? Capacity to recognize people wearing a mask? Instantaneous liveness detection? What are the constraints? Must fit inside a camera casing?
  3. In-house expertise – what is the primary job you’re implementing facial recognition to do?

It’s important to define these factors up-front and create the decision tree that will help navigate the 7 success factors presented below. Addressing the most critical aspects up-front will help discard irrelevant options and focus only on those that matter.

A well-defined decision tree also helps address dependencies between the decisions. Here’s how that can work:

The 7 Success Factors for Choosing Your Own Facial Recognition Solutions

1. Precision/Accuracy

There are a several levels to think about when considering accuracy in facial recognition – these include:

When Accuracy is a Decisive Factor:

Accuracy is always a critical aspect of the success of a facial recognition system, and for this reason operators are recommended to consider solutions from vendors whose regularly updated algorithms are vetted and ranked highly in industry tests such as the FRVT. Accuracy can be the most critical factor in situations where facial recognition is used to protect access to secure facilities, highly confidential data, controlled or hazardous substances, and other highly restricted tangible or intangible assets. Obviously, the most accurate algorithms, even in cases where they might not be inherently more expensive, require more storage and processing power, which can drive significantly the total cost of deployment as we will discuss later.

Some of the most popular use cases for facial recognition would likely not be economically viable if they used the most precise models available in the market. Other constraints such as form factor, size, weight or energy requirement might force the exclusion of the most accurate models. In practice, most use cases don’t need a 99% or higher level of accuracy. The top vendors in the market likely also offer models that can address all these constraints without losing more than 1 or 2%.

In the below scenarios where accuracy is less dependent, it is still recommended to utilize a solution performing above the 95th percentile.

Use Cases:

Verticals:

Size of the deployment:

2. Features

Each facial recognition solution offers specific features . The three basic features that should be expected from any solution include:

The most advanced facial recognition solutions such as FaceMe®, also include enhanced features, such as:

When Features are a Decisive Factor:

Use Cases:

Verticals:

Size:

3. Performance

As with accuracy, many components play into the performance ability of a facial recognition system. Let’s break them down here:

Why Performance Matters:

Performance can be a dependent factor to enabling a facial recognition system in multiple use cases. For example, deployments in large facilities often need to handle hundreds of video channels concurrently, some in high traffic areas. A vendor offering high performing facial recognition models can significantly reduce the number of expensive workstations required to monitor such facilities.

Performance and Edge vs Cloud Architecture:

In the next decision section, we’ll be looking at edge vs cloud architecture for your facial recognition system. It’s important to consider that edge systems, while not having a inherent performance advantage over cloud-based processing units, generally deliver much faster facial recognition as sending images or video to be processed on the cloud increases response time from milliseconds to several seconds.

Benchmarks for Best Performance:

Optimized Chipsets:

Optimized Software:

When Performance is a Decisive Factor:

Use Cases:

Verticals:

Size:

4. Edge vs Cloud:

Edge vs cloud architecture can impact the security and performance of your facial recognition system as a whole and can be an important consideration for operators seeking maximum speed. Edge-based systems operate more quickly because information does not have to be sent back and forth to the cloud, potentially adding several seconds for upload transmission.

Edge-based systems offer several additional benefits and thus have become popular choices:

However, cloud can be a better option to consider in use cases with:

5. Device and Hardware Support

The hardware environment is sometimes a constraining factor when selecting your facial recognition system. Here’s what to take into account:

Operating System Support

Windows and Android are the most common operating systems supporting by facial recognition software. Better solutions offer a wider range of OS support such as iOS, Linux versions, Jetson, etc. For example, FaceMe® offers one of the market's most comprehensive support, with more than 10 OSs: Windows, Android, iOS, Linux variants, Ubuntu x64, Ubuntu ARM, RedHat, JetPack (mainly for NVIDIA Jetson family), CentOS, Yocto ARM.

Due to their reduced size and capabilities, iOS and Android have greater restraints in terms of processing power as compared to Windows or Linux which are much more powerful.

Large facilities, for example petro-chemical plants which house a variety of security systems and existing IoT infrastructure may be running multiple OS across these systems. In this case, it’s important to consider support for interoperability between multiple operating systems.

When Device & Operating System Support is a Decisive Factor:

Use Cases:

Verticals:

Size:

Hardware Support

Thanks to fast evolving hardware and chipset technology innovation, there are ever increasing options on the market to best address speed, power, form factor and cost constraints, driving the introduction of new use cases previously not possible.

Hardware Options to Run Facial Recognition

Chipset Options to Run Facial Recognition

CPU
CPUs are typically a simpler, cheaper option.
GPU
Facial recognition systems can also operate at the edge while communicating with a central cloud server. This workflow optimizes speed for operators with systems covering multiple locations where a cloud database is necessary.GPUs are typically more expensive and higher performing.
VPU
GPUs are typically more expensive and higher performing.
Combo
Combinations of different chipsets are designed to power complete yet affordable solutions.

When Hardware Support is a Decisive Factor:

Use Cases:

Verticals:

Size:

6. Software and SDK Flexibility

Facial recognition software is the actual program which processes information extracted from video feeds to determine a match or detect faces. There are a few formats in which the software component of your facial recognition system can be incorporated – let’s walk through them further:

When Software Format is a Decisive Factor:

Use Cases:

Verticals:

Size:

In many cases size is not a barrier to using a plug-and-play solution, as long as you are looking to address a relatively standard set of functionalities or use cases. Good software solutions are scalable. Unique customized solutions integrating an SDK generally come with significant costs of integration and are typically reserved to large organizations. However, with the fast decrease of hardware cost and the exponential improvements in chipset performance, Facial recognition SDKs can now be integrated at a reasonable cost in mass-market AIoT products that can sometimes retail below $100.

7. Cost of Ownership

It’s important to consider cost of ownership for the full life of your facial recognition system prior to integration. Here are some of the core components to consider:

Here are some examples of cost relative to deployment size:

Designing the Best Facial Recognition System for You

There are many options available when it comes to designing a facial recognition solution for your unique scenario. Each core component and decision element described above will impact the effectiveness of the solution for your needs. A safe approach starts by familiarizing yourself with each decision and the range of options that exist today, consider how fast the technology and solution offerings have improved in the recent past and inquire about trends and how expected breakthrough might inform your testing and deployment timeline. Then focus on the elements that are more crucial for the success of your deployment (such as performance, features, hardware, etc.).

It can also be useful to learn about successful deployments and failures in your industry. Many industries have associations and task forces monitoring and analyzing facial recognition and how it could benefit their members.

Finally, use the culmination of this information to build your unique blueprint or decision tree that will guide the right decisions and ultimately result in the best solution that fits your unique needs.

For a full overview of facial recognition, how it works and how it can be deployed, read Edge-based Facial Recognition - The Ultimate Guide.