Facial Recognition at the Edge - The Ultimate Guide 2022
FaceMe®
< All Articles

Facial Recognition at the Edge - The Ultimate Guide 2022

2022/04/01

Biometric technology continues to develop rapidly, offering safety and security to all facets of life. By using an individual’s physical characteristics, which are almost impossible to replicate, biometric technology proves a better choice than traditional security methods that are easily lost or stolen, such as personal identification cards, passwords/PINs, and keys. Though it has been around for decades, primarily through fingerprints, biological samples (DNA, blood, saliva, etc.), and even iris scanning, one biometric technology deserves a closer look: facial recognition.

Facial recognition has grown in popularity in recent years. One of the most ubiquitous use cases is mobile phones. Many consumers worldwide interact with this technology daily to secure and unlock their phones.

However, the possibilities extend far beyond personal mobile devices and include significant benefits in safety, security, and efficiency across a variety of industries. As a compelling, comprehensive, and rewarding technology, it is vital to understand how facial recognition works, how it can be deployed and optimized, technical considerations and specifications, varied use cases, and of course, its potential.

1. What is Facial Recognition?

Facial recognition is a biometric technology that identifies facial vectors and features, matching them with pre-enrolled individuals. Recent advancements in AI technologies, based on deep neural networks (DNN), have dramatically improved precision, unlocking a wealth of new use cases.

The technology leverages proprietary AI algorithms and mathematical equations to create an individual's template by measuring facial variables – nose depth and width, forehead length, and eye shape. Facial recognition then compares the generated template with existing templates in a database. If there is a match, it can confirm an individual’s identity.

Building on its facial recognition expertise, CyberLink leveraged deep learning and neural networks to create FaceMe®, an AI-based facial recognition engine. CyberLink continues to push the boundaries of these technologies to enhance its AI-based models, resulting in one of the world’s most accurate, secure, and flexible edge-based solutions.

1.1 How Does Facial Recognition Work?

Facial recognition is by far the most powerful and relevant AI biometric technology. It has vast abilities and can carry out a number of tasks beyond just face detection and face recognition. The more robust and feature-forward a facial recognition platform, like FaceMe, the more benefits and fewer biases it brings.

The key features of a facial recognition engine are:

Face detection

Face detection is the first step the technology takes to detect a face. In this step, the technology scans the whole image to see if any area contains full or even partial human faces. Fast and precise face detection is a critical first step to ensure the performance of the entire facial recognition process. FaceMe can detect more than one face simultaneously, count how many faces are present, and perform detection on each of them individually.

Face feature extraction

After face detection, feature extraction is the next step. The engine first extracts an n-dimensional vector set (a template) from the facial image. Achieving very high precision requires a high “n” value. Next, the template extracted from an individual’s face is used for matching or searching.

1:1 Face match

If the goal is to verify a person’s identity and answer the question, “Is this person who they say they are?” then the engine will perform a face match. This means using a 1:1 method to take the extracted facial template and match it with the same facial template from the pre-enrolled database. It could also involve checking the facial template against the face on a piece of identification to verify that both belong to the same person. Some of the most common uses of face match happen on mobile phones, such as Apple’s Face ID which unlocks the phone, or mobile banking services where you log in to a financial service portal with your face.

1:N Face search

During this step, FaceMe will answer the question, “Who is this person?” The engine will complete a 1:N search by comparing an individual’s template against pre-enrolled faces in the database to find the best match and confirm the person’s identity. FaceMe only stores encrypted template data. To fully ensure privacy, no actual images of faces are stored on our platform. The most common use of face search is a security and surveillance system using a camera to verify that a person belongs to a particular company – if their face was pre-enrolled in the company’s facial database, face search will confirm it is them and grant them access.

1.2 More than Recognizing Faces

gender, age and mood facial recognition

Some key use cases require additional features, such as the following:

Facial attribute detection (face analysis)

Face attribute detection, similarly known as face analysis, identifies and analyzes characteristics such as age, gender, mood, and head orientation or movements (e.g., nodding, shaking). This feature is a crucial enabler of smart retail and digital signage for use cases like pushing customized ads and messaging to targeted audiences or collecting detailed visitor statistics.

Mask detection

Mask detection is one of the newest and most valuable features for ensuring health and safety in public or private spaces as a result of the Covid-19 pandemic. CyberLink’s FaceMe SDK is a cross-platform facial recognition SDK (Software Development Kit) that offers optimized mask detection and facial recognition when wearing a mask. It recognizes health-compliant masks and verifies if the nose and mouth are properly covered while performing highly accurate face detection and recognition. Additionally, CyberLink’s FaceMe Security helps existing security solutions upgrade their mask detection features so that employees can enter their offices without having to remove their masks.

Enjoying what you're reading so far? Subscribe to our newsletter and get the latest insights straight to your inbox!
Subscribe

Anti-spoofing

For protection against biometric fraud, such as holding someone else’s photo or video in front of a camera, anti-spoofing technology provides secure and accurate live-detection with 3D or 2D cameras.

2D cameras (e.g., USB webcams) catch fraud through interactive and non-interactive measures. Interactive measures detect natural and precise head or facial movements to confirm the presence of a live person. Non-interactive measures are unique to each solution provider and its AI algorithm for face detection and recognition.

3D cameras perform depth detection, allowing quasi-instantaneous anti-spoofing. There is no need for separate interactive detection or recognition measures. 3D cameras generally provide a superior experience, but they are costlier. 2D alternatives can provide accurate anti-spoofing at a fraction of the cost. FaceMe supports both 2D and 3D cameras. A few of the 3D camera options compatible with FaceMe are Intel RealSense, 3D cameras on iPads and iPhones, Orbbec, Himax, Altek, and eYs3D.

CyberLink’s FaceMe facial recognition technology passed iBeta’s spoofing attack detection test by detecting and rejecting all impersonation and substitution attempts using the test’s photos and videos.

Anti-spoofing technologies at ATM

1.3 Provides Remarkable Accuracy

The most precise facial recognition engines are characterized by a low false non-match rate (FNMR) and an extremely low false match rate (FMR). A false non-match is the failure to match two facial templates from the same person, whereas a false match is when one person’s face is matched with someone else’s.

The National Institution of Standards and Technology (NIST) in the U.S.A. is the governing body that determines how well a given facial recognition algorithm achieves a set of common tasks. NIST’s Facial Recognition Vendor Test (FRVT) uses different datasets to evaluate the performance of an algorithm. For example, the VISA category tests a facial recognition algorithm’s ability to correctly identify an individual based on a passport photo, while the WILD test uses random, non-constrained photojournalism-style images.

FaceMe has achieved one of the highest NIST VISA-Border test ratings, with 99.48% at 1E-6 accuracy. 1E-6 means the False Match Rate (FMR) is 1 in 1,000,000 (0.000001). FaceMe has also demonstrated top-tier accuracy levels for WILD (97.00% at 1E-5). By comparison, smartphones’ Face ID offers about 96% at 1E-4 accuracy.

Beyond the algorithm, some of the main factors affecting accuracy are camera resolution, camera positioning, lighting, cleanliness, and camera type. Facial recognition engines generally work adequately with 720p cameras but a 1080p resolution is generally recommended.

Find out how to integrate FaceMe SDK, a cross-platform AI facial recognition engine, into edge-based AIoT/IoT devices for all business scenarios.

2. Why You Should Deploy Edge-Based Facial Recognition

Facial recognition can be deployed using cloud-based solutions, such as Microsoft or Amazon, or by integrating SDKs or software onto edge devices. While each approach offers distinct benefits, edge-based facial recognition is generally deemed superior, offering better speed, security, affordability, flexibility, and versatility.

2.1 What is Edge-based Facial Recognition?

Facial recognition SDK built for Windows, Linux, Android, and iOS.

When deploying facial recognition at the edge, the technology is embedded in local devices: a smart lock, mobile phone, point-of-sale (POS) system, interactive kiosk, digital signage, or others. Edge devices run facial recognition quickly and with extreme precision, from cloud processing to large file transmissions. There is no inherent need for a network or cloud connection except to access the face database if it is not stored on the edge device. Even then, only a small, encrypted, template is transmitted for validation. The database’s encrypted templates are often hosted on a remote server, resulting in a secure operation completed within milliseconds.

Businesses that implement facial recognition from the ground up will benefit from an edge-based approach with IoT devices carrying out the required tasks. Financial institutions make a compelling case for edge-based facial recognition systems, as many banks do not allow internet connections for security reasons.

The low cost, flexibility, and scalability of edge-based facial recognition make it the best option for most end users.

CyberLink’s FaceMe SDK perfectly exemplifies top edge-based facial recognition solutions. This highly competitive and flexible face recognition SDK is easy to integrate across a wide range of edge devices. FaceMe leads the way by supporting a variety of chipsets and OSes, and its highly accurate AI engine is ranked one of the best in the NIST Face Recognition Vendor Test (FRVT). Through constant innovation, the technology meets the highest accuracy and security standards for deployments across a wide range of industries and use cases, including security, access control, public safety, smart banking, smart retail, smart city, and home protection.

Contact us to get an evaluation version and price quote today!

2.2 The Future of Facial Recognition Technology is on Edge Devices

Let's examine trends for (1) cost, (2) response time and (3) service availability:

2.2.1 Cost of Ownership

AI is very demanding on computation, and cloud computing is not cheap, so choosing between the edge or cloud is a key design decision. Edge devices have a cost advantage over the cloud, which typically charges an incremental amount for each face recognized. Cloud-based computing might be cheaper on a very small scale, not exceeding a few faces per hour. As AI chips have become more inexpensive, edge-based solutions have had a sustainable and widening cost advantage.

2.2.2 Response Time

When it comes to speed, the best facial recognition algorithms operate in milliseconds. Edge-based solutions rank supreme, outperforming their cloud-based counterparts by several orders of magnitude. Regardless of why a business seeks to implement facial recognition, response time matters. Across a growing number of use cases, the cloud cannot compete. For instance, delays in identifying a block-listed individual could cause irreparable harm to the company.

2.2.3 Service Availability

No internet service is immune from interruptions or unexplainable low bandwidth issues. Imagine if the lock on your front door at home stops working because it depends on a cloud-based facial access solution. Edge-based facial recognition doesn’t have such vulnerabilities to internet issues.

With its inherent superiority and huge momentum in innovation, edge-based technology will be a vital driver of the future success of facial recognition. Therefore, we will focus on edge-based facial recognition for the remainder of this article.

3. How to Integrate Facial Recognition in Edge Devices

When building a facial recognition edge device, choosing the right chipset based on the specific use case is the most consequential decision in regards to cost and performance. For example, a high-end NVIDIA, GPU chip has a higher upfront cost but can handle hundreds of video channels concurrently, thus reducing the number of costly workstations required to monitor a large facility. At the other end, a low-cost SoC from MediaTek or Broadcom will offer limited performance at speeds of about five frames per second and only frontal face recognition. Still, it will likely be powerful enough, and more affordable, for smaller use cases such as door access.

3.1 Standalone GPU

GPU chips are powerful hardware designed for facial recognition with outstanding performance. In general, substantial memory, high memory bandwidth, and a considerable amount of floating-point computation capability make GPUs the best option for complex, computation-hungry AI algorithms such as facial recognition. GPUs are also suitable for enabling facial recognition in surveillance systems, which requires simultaneously applying facial recognition across multiple video channels. A separate CPU chip is needed to build a system using these GPUs.

This year we recommend two GPUs from NVIDIA. They are extremely competent and provide a good balance between computational power and price.

Vendor
Product
Highlights
NVIDIA
RTX A5000
RTX A5000 was first introduced in April 2021 and is now generally available. It replaces the Quadro 5000, providing processing speeds 2.5 times faster. The A5000 is a great option for workstation GPUs with solid computing power at an outstanding price.

Device: Workstation
Performance: Very High
NVIDIA
RTX A6000
RTX A6000 was announced in Oct. 2020, featuring the new and powerful Ampere architecture. Compared to the previously mentioned RTX A5000, A6000 has higher processing power for AI facial recognition. If you have performance requirements, A6000 is one of the best options.

Device: Workstation
Performance: Very High

GPU acceleration cards are most appropriate for workstations or on-premises servers. Our article How to Build a Proper Workstation for Facial Recognition provides more details.

3.2 CPU/SoC for AIoT Devices

These chips are complete yet affordable solutions for enabling facial recognition in small, mass-market IoT devices. They all integrate CPUs, resulting in simpler, cheaper options. SoCs may seem less capable than GPU options, with lower power consumption and a smaller form factor, but several FaceMe facial recognition models (detailed in section 3.4) can enable these AIoT devices while still providing highly accurate facial recognition results.

Vendor
Product
Highlights
NVIDIA
Jetson Xavier NX
The Xavier NX is the latest Jetson model. Xavier NX was first introduced in March 2020 and uses Volta architecture as its core. Xavier NX balances performance, power consumption, form factor, and price – an excellent option for powering robots, AI cameras, or even small-sized AI workstations.

Device: IPCs, kiosks, POS systems, robots, etc.
Performance: High
NVIDIA
Jetson Orin NX
Jetson AGX Orin
In 2021, NVIDIA announced Jetson Orin NX and Jetson AGX Orin, which both use the latest Ampere architecture and are five to six times faster than the previous generation. At the time of publication, they are still not available.

Device: IPCs, kiosks, POS systems, robots, smart displays, etc.
Performance: High to very high
Intel
Celeron / Core i3
Intel Celeron and Core i3 are considered entry-level CPUs for consumer PCs. Yet they are more than sufficient to handle various tasks and facial recognition algorithms in industrial PCs. Intel CPUs are compatible with both Windows and Ubuntu OS, making them a worthy option if your existing applications or systems rely on these OSs.

Device: IPCs, kiosks, POS systems, small-size workstations
Performance: Medium to high
Qualcomm
Snapdragon 400/600 series
QCS 410/610
The Snapdragon 400/600 series is commonly used in entry to mid-level smart devices (tablets, mobile computers), and power-efficient wearables.
Comparatively, QCS 410/610 are application processors that offer high performance and power efficiency for performing on-device AI. They are designed for edge applications with the SNPE framework, which accelerates AI applications such as facial recognition algorithms.

Device: kiosks, mobile computers (with barcode scanners), smart displays, etc.
Performance: Medium
MediaTek
i350
i350 incorporates a dedicated APU (AI processor) to enable vision edge AI, such as facial and object recognition, with better performance and power efficiency.

Device: kiosks, mobile computers (with barcode scanners), smart displays, etc.
Performance: Medium
NXP
i.MX8M Plus
With its high reliability, the i.MX8M Plus family focuses on machine learning and vision, advanced multimedia, and industrial IoT. NXP’s Plus models add a powerful NPU to the SoC to significantly enhance AI algorithms' performance.

Device: kiosks, mobile computers (with barcode scanners), smart displays, etc.
Performance: Medium
Broadcom
BCM2711
This chip is used in the Raspberry Pi 4. It’s very affordable and suitable for early development, testing, and proof-of-concept purposes.

Device: AIoT devices, early development test devices
Performance: Acceptable for some use cases

More details on building ARM-based AIoT devices can be found in this article: How to Build a Proper Workstation for Facial Recognition.

3.3 Operating Systems

Chipsets are often designed to run on specific operating systems (OS). An excellent facial recognition engine should support as many chipset+OS combinations as possible. FaceMe supports one of the market’s most comprehensive lists of chipsets and more than 10 OSes:

  • Windows
  • Android
  • iOS
  • Linux variants: Ubuntu x64, Ubuntu ARM, RedHat, JetPack (mainly for NVIDIA Jetson family), Debian ARM, CentOS, Yocto ARM

CyberLink designed FaceMe to be highly versatile, allowing flexible customization of multiple combinations of hardware chips and OSes that match end users’ unique needs. FaceMe’s multi-OS support is ideal for cross-platform solutions. Developers have access to several GPU acceleration options, harnessing OpenVINO, NVIDIA CUDA/TensorRT, Intel Movidius, NVIDIA Jetson, Qualcomm SNPE, MediaTek NeuroPilot, and more, to speed up deep learning algorithms and further optimize performance.

3.4 System Architecture Optimization for Best Performance

Designing an excellent facial recognition system for a high-performance workstation or PC with GPU (or VPU) is never easy, because dozens of concurrent video streams run between the CPU, GPU, and memory over the system bus. Even an excellent facial recognition algorithm will be slow if implemented improperly on a system architecture level. The system architecture design should minimize the data flow between CPU, GPU, and memory.

FaceMe has an optimized system architecture that allows it to deliver the best performance. For example, on a single workstation FaceMe with NVIDIA RTX A6000 can handle 256 to 416 frames per second (the exact number may vary depending on which FaceMe facial recognition model is used). This is equivalent to handling 25 to 41 concurrent video channels (each with 10 frames per second) per workstation — an outstanding cost-performance offering.

3.5 Lightweight AI Model for Facial Recognition

Several cost-constrained use cases don’t require more than basic frontal face recognition (e.g., smart door locks). These devices are driving market demand for lightweight AI models that can achieve reasonable facial recognition at a lower cost. FaceMe provides three models for this purpose:

  • Ultra High (UH) Model: This model can recognize both VISA and WILD types of faces with market-leading precision. However, it demands very high computation power from the GPU or high-end Intel CPU.
  • Very High (VH) Model: This model can recognize both VISA and WILD types of faces with slightly less precision than that of the UH Model, but the required computational power is significantly lower. It unlocks a wide range of use cases requiring superior accuracy that wouldn’t be economically viable otherwise.
  • High (H) Model: This model can run on low-cost chips with low computation power. The precision is still good enough for VISA-type frontal faces, enabling several use cases that weren’t remotely possible before.
Are you enjoying this article? Subscribe to our newsletter and get the latest insights straight to your inbox!
Subscribe

4. Edge-based Facial Recognition: On-premises Devices and Workstations

One important attribute of leading facial recognition solutions like FaceMe is its flexibility for all relevant types of hardware. FaceMe can be deployed across workstations, computers, mobile, and IoT devices. Let’s examine a few examples.

Facial recognition on workstations

Organizations looking to deploy a facial recognition SDK over tens, or even hundreds, of video channels across large facilities will benefit from a workstation powered by a high-end GPU capable of simultaneously handling multiple IP camera video feeds. Different properties, including department stores, airports, industrial plants, and hospitals, have dozens or even hundreds of cameras for varied purposes: from security and access control to visitor behavior analysis, crowd management, and VIP customer identification. These are all use cases enabled or optimized by facial recognition. Connecting all the cameras to one or a few central workstations running facial recognition is the easiest, most robust, and likely most economical solution.

To learn more about the benefits of FaceMe for workstations, check out our partnership with VIVOTEK and the successful integration in their facial recognition solution.

Facial recognition on PCs

Facial recognition is most commonly used on PCs for smaller operations or single uses. Take a store or restaurant that wants to identify VIPs, automatically clock in employees, or get alerts for block-listed people. In the case of the Covid-19 pandemic, it even has the ability to ensure everyone who enters the venue – employees and customers – wears a mask and does not have a high temperature. Store or restaurant management can install IP or USB cameras at the front and back doors and simply connect them to a PC that runs robust integrated facial recognition software. A value-priced, ready-to-deploy software solution that includes all these features is FaceMe Security. Learn more about crucial considerations for building top-of-the-line industrial PCs.

Facial recognition on mobile devices

The potential of facial recognition technology on mobile devices goes way beyond unlocking a cell phone. One compelling fintech use case is the integration of eKYC (electronic know your customer) facial recognition technologies on mobile phones to strengthen identity verification for online banking, loan applications, insurance, and more.

Facial recognition on smart IoT devices

The current fast-paced innovation in edge computing, driving better performance while cutting costs, also opens the door to endless IoT device use cases powered by facial recognition. Smart kiosks offer a compelling example. Frequent travelers are familiar with Global Entry and Clear kiosks, which use facial recognition. Now fast-food restaurants, hospitals, and hotels are deploying smart kiosks and integrating facial recognition. Major hotel chains have eagerly introduced self-check-in kiosks to cut wait times. Adding a facial recognition engine like FaceMe provides exciting, personalized experiences for guests by using their opted-in faces as the only ID they need through their stay. Take a look at our extended article on smart IoT devices.

Find out how to integrate FaceMe SDK, a cross-platform AI facial recognition engine, into edge-based AIoT/IoT devices for all business scenarios.

5. Other Facial Recognition Design Factors: Security, Encryption & Privacy

As reviewed earlier, edge-based facial recognition is more secure than cloud-based options that require individuals’ pictures and videos to be sent through the internet to a cloud server; a process inherently vulnerable to attacks and leakage. Edge-based computing avoids most risks as the only data captured and stored takes the form of encrypted face templates, and the entire process can run without any cloud connection.

FaceMe uses AES-256 bit encryption to secure all data before it is stored in any database. AES is one of the best symmetric encryption algorithms, and 256-bit is the highest security confirmation. The face template is protected by storing the encrypted file (with a secret key) outside the source or platform server, keeping it secure even if physical devices are compromised or stolen.

Individuals must first opt-in to any facial recognition program requiring face enrollment. In edge-based solutions, the captured information will consist of template data (a mathematical number in a very high dimension) for future matching and identification purposes. The template doesn’t contain an actual face image, it can’t be used to recompose someone’s face, and it is kept separate from any personal information that could identify a person. The encrypted data that is captured when performing facial recognition is only used to establish a match with a pre-enrolled template stored in a secure database. Many data privacy laws and regulations (such as GDPR, CCPA, BIPA, and LGPD) count biometric data as personal information. Therefore, any business looking to employ face recognition must always obtain the user’s consent.

When evaluating various facial recognition providers it is important to locate their headquarters and essential facilities. For instance, the U.S. government has expressed fair concerns around surveillance technologies from companies in China and Russia – as they might not have adequate opt-in or data protection requirements. Most facial recognition solutions are safe and rigorously apply strict data and privacy protection standards. But as an end-user, you should be able to fully trust your service provider, especially when it comes to security, privacy, and protection of human rights.

6. Top Use Cases of Facial Recognition Technology

While debates on the technology continue, there are many success stories where facial recognition deployment has increased safety, resulting in positive user experiences. A recent survey conducted by the Security Industry Association (SIA) found that the majority of Americans (68%) believe facial recognition can make society safer, with particular support for the technology in airports (75% for airlines; 69% for TSA), office buildings (70%), and banks (68%).

Key relevant use cases fall under five major categories:

  1. Access control, e.g., entrance access, medical cabinets, smart locks
  2. Surveillance and security, e.g., detecting unauthorized people in warehouse areas
  3. Authentication, e.g., eKYC for BFSI (Banking, Financial Services and Insurance)
  4. Smart retail, e.g., collecting statistics on shoppers’ demographics
  5. Health control during a pandemic, e.g., detecting whether a face mask is worn properly

Example: FaceMe Security offers a comprehensive facial recognition application in surveillance and security.

AI biometrics can power many use cases in specific vertical segments. For more on these use cases, read our article Top 7 Use Cases for Facial Recognition in 2022.

7. Final Thoughts: How Facial Recognition Will Transform Biometrics

Facial recognition technology is poised to make our world a better place. But to do that, individuals everywhere need broader levels of education on ethical implementation to feel more comfortable with and accepting of businesses that have openly adopted this AI biometric technology as a new, safe standard.

Facial recognition and the potential it holds are more than what the fear-mongering makes it. It’s businesses keeping their employees safe by automating secure access control in the office. It’s retailers enhancing customer experiences in their stores. It’s manufacturers simplifying access to their many restricted areas. It’s banks and fintech companies introducing much stronger authentication and cutting-edge security controls. And that’s just the tip of the iceberg.

Facial recognition is the future of AI biometric technology. The industry must better educate consumers and debunk the many falsehoods circulated about this technology while explaining its positive value and potential for good. Facial recognition also needs to be regulated appropriately not to hinder innovation but to bring forth its many benefits.

FaceMe®: CyberLink’s Complete Facial Recognition Solution

Contact
Our Sales Team

For further information on FaceMe®

Our team of experts will be happy to answer your questions and schedule a demo. Free evaluation versions of the FaceMe® are available to qualified contacts.

Contact Our Sales Team

Subscribe to FaceMe® Newsletter