Biometric technology continues to develop rapidly, offering safety and security to all facets of life. By using an individual’s physical characteristics, which are almost impossible to replicate, biometric technology proves a better choice than traditional security methods that are easily lost or stolen, such as personal identification cards, passwords/PINs, and keys. Though it has been around for decades, primarily through fingerprints, biological samples (DNA, blood, saliva, etc.), and even iris scanning, one biometric technology deserves a closer look: facial recognition.
Facial recognition has grown in popularity in recent years. One of the most ubiquitous use cases is mobile phones. Many consumers worldwide interact with this technology daily to secure and unlock their phones.
However, the possibilities extend far beyond personal mobile devices and include significant benefits in safety, security, and efficiency across a variety of industries. As a compelling, comprehensive, and rewarding technology, it is vital to understand how facial recognition works, how it can be deployed and optimized, technical considerations and specifications, varied use cases, and of course, its potential.
Facial recognition is a biometric technology that identifies facial vectors and features, matching them with pre-enrolled individuals. Recent advancements in AI technologies, based on deep neural networks (DNN), have dramatically improved precision, unlocking a wealth of new use cases.
The technology leverages proprietary AI algorithms and mathematical equations to create an individual's template by measuring facial variables – nose depth and width, forehead length, and eye shape. Facial recognition then compares the generated template with existing templates in a database. If there is a match, it can confirm an individual’s identity.
Building on its facial recognition expertise, CyberLink leveraged deep learning and neural networks to create FaceMe®, an AI-based facial recognition engine. CyberLink continues to push the boundaries of these technologies to enhance its AI-based models, resulting in one of the world’s most accurate, secure, and flexible edge-based solutions.
Facial recognition is by far the most powerful and relevant AI biometric technology. It has vast abilities and can carry out a number of tasks beyond just face detection and face recognition. The more robust and feature-forward a facial recognition platform, like FaceMe, the more benefits and fewer biases it brings.
The key features of a facial recognition engine are:
Face detection is the first step the technology takes to detect a face. In this step, the technology scans the whole image to see if any area contains full or even partial human faces. Fast and precise face detection is a critical first step to ensure the performance of the entire facial recognition process. FaceMe can detect more than one face simultaneously, count how many faces are present, and perform detection on each of them individually.
After face detection, feature extraction is the next step. The engine first extracts an n-dimensional vector set (a template) from the facial image. Achieving very high precision requires a high “n” value. Next, the template extracted from an individual’s face is used for matching or searching.
If the goal is to verify a person’s identity and answer the question, “Is this person who they say they are?” then the engine will perform a face match. This means using a 1:1 method to take the extracted facial template and match it with the same facial template from the pre-enrolled database. It could also involve checking the facial template against the face on a piece of identification to verify that both belong to the same person. Some of the most common uses of face match happen on mobile phones, such as Apple’s Face ID which unlocks the phone, or mobile banking services where you log in to a financial service portal with your face.
During this step, FaceMe will answer the question, “Who is this person?” The engine will complete a 1:N search by comparing an individual’s template against pre-enrolled faces in the database to find the best match and confirm the person’s identity. FaceMe only stores encrypted template data. To fully ensure privacy, no actual images of faces are stored on our platform. The most common use of face search is a security and surveillance system using a camera to verify that a person belongs to a particular company – if their face was pre-enrolled in the company’s facial database, face search will confirm it is them and grant them access.
Some key use cases require additional features, such as the following:
Face attribute detection, similarly known as face analysis, identifies and analyzes characteristics such as age, gender, mood, and head orientation or movements (e.g., nodding, shaking). This feature is a crucial enabler of smart retail and digital signage for use cases like pushing customized ads and messaging to targeted audiences or collecting detailed visitor statistics.
Mask detection is one of the newest and most valuable features for ensuring health and safety in public or private spaces as a result of the Covid-19 pandemic. CyberLink’s FaceMe SDK is a cross-platform facial recognition SDK (Software Development Kit) that offers optimized mask detection and facial recognition when wearing a mask. It recognizes health-compliant masks and verifies if the nose and mouth are properly covered while performing highly accurate face detection and recognition. Additionally, CyberLink’s FaceMe Security helps existing security solutions upgrade their mask detection features so that employees can enter their offices without having to remove their masks.
For protection against biometric fraud, such as holding someone else’s photo or video in front of a camera, anti-spoofing technology provides secure and accurate live-detection with 3D or 2D cameras.
2D cameras (e.g., USB webcams) catch fraud through interactive and non-interactive measures. Interactive measures detect natural and precise head or facial movements to confirm the presence of a live person. Non-interactive measures are unique to each solution provider and its AI algorithm for face detection and recognition.
3D cameras perform depth detection, allowing quasi-instantaneous anti-spoofing. There is no need for separate interactive detection or recognition measures. 3D cameras generally provide a superior experience, but they are costlier. 2D alternatives can provide accurate anti-spoofing at a fraction of the cost. FaceMe supports both 2D and 3D cameras. A few of the 3D camera options compatible with FaceMe are Intel RealSense, 3D cameras on iPads and iPhones, Orbbec, Himax, Altek, and eYs3D.
CyberLink’s FaceMe facial recognition technology passed iBeta’s spoofing attack detection test by detecting and rejecting all impersonation and substitution attempts using the test’s photos and videos.
The most precise facial recognition engines are characterized by a low false non-match rate (FNMR) and an extremely low false match rate (FMR). A false non-match is the failure to match two facial templates from the same person, whereas a false match is when one person’s face is matched with someone else’s.
The National Institution of Standards and Technology (NIST) in the U.S.A. is the governing body that determines how well a given facial recognition algorithm achieves a set of common tasks. NIST’s Facial Recognition Vendor Test (FRVT) uses different datasets to evaluate the performance of an algorithm. For example, the VISA category tests a facial recognition algorithm’s ability to correctly identify an individual based on a passport photo, while the WILD test uses random, non-constrained photojournalism-style images.
FaceMe has achieved one of the highest NIST VISA-Border test ratings, with 99.48% at 1E-6 accuracy. 1E-6 means the False Match Rate (FMR) is 1 in 1,000,000 (0.000001). FaceMe has also demonstrated top-tier accuracy levels for WILD (97.00% at 1E-5). By comparison, smartphones’ Face ID offers about 96% at 1E-4 accuracy.
Beyond the algorithm, some of the main factors affecting accuracy are camera resolution, camera positioning, lighting, cleanliness, and camera type. Facial recognition engines generally work adequately with 720p cameras but a 1080p resolution is generally recommended.
Facial recognition can be deployed using cloud-based solutions, such as Microsoft or Amazon, or by integrating SDKs or software onto edge devices. While each approach offers distinct benefits, edge-based facial recognition is generally deemed superior, offering better speed, security, affordability, flexibility, and versatility.
When deploying facial recognition at the edge, the technology is embedded in local devices: a smart lock, mobile phone, point-of-sale (POS) system, interactive kiosk, digital signage, or others. Edge devices run facial recognition quickly and with extreme precision, from cloud processing to large file transmissions. There is no inherent need for a network or cloud connection except to access the face database if it is not stored on the edge device. Even then, only a small, encrypted, template is transmitted for validation. The database’s encrypted templates are often hosted on a remote server, resulting in a secure operation completed within milliseconds.
Businesses that implement facial recognition from the ground up will benefit from an edge-based approach with IoT devices carrying out the required tasks. Financial institutions make a compelling case for edge-based facial recognition systems, as many banks do not allow internet connections for security reasons.
The low cost, flexibility, and scalability of edge-based facial recognition make it the best option for most end users.
CyberLink’s FaceMe SDK perfectly exemplifies top edge-based facial recognition solutions. This highly competitive and flexible face recognition SDK is easy to integrate across a wide range of edge devices. FaceMe leads the way by supporting a variety of chipsets and OSes, and its highly accurate AI engine is ranked one of the best in the NIST Face Recognition Vendor Test (FRVT). Through constant innovation, the technology meets the highest accuracy and security standards for deployments across a wide range of industries and use cases, including security, access control, public safety, smart banking, smart retail, smart city, and home protection.
Let's examine trends for (1) cost, (2) response time and (3) service availability:
AI is very demanding on computation, and cloud computing is not cheap, so choosing between the edge or cloud is a key design decision. Edge devices have a cost advantage over the cloud, which typically charges an incremental amount for each face recognized. Cloud-based computing might be cheaper on a very small scale, not exceeding a few faces per hour. As AI chips have become more inexpensive, edge-based solutions have had a sustainable and widening cost advantage.
When it comes to speed, the best facial recognition algorithms operate in milliseconds. Edge-based solutions rank supreme, outperforming their cloud-based counterparts by several orders of magnitude. Regardless of why a business seeks to implement facial recognition, response time matters. Across a growing number of use cases, the cloud cannot compete. For instance, delays in identifying a block-listed individual could cause irreparable harm to the company.
No internet service is immune from interruptions or unexplainable low bandwidth issues. Imagine if the lock on your front door at home stops working because it depends on a cloud-based facial access solution. Edge-based facial recognition doesn’t have such vulnerabilities to internet issues.
With its inherent superiority and huge momentum in innovation, edge-based technology will be a vital driver of the future success of facial recognition. Therefore, we will focus on edge-based facial recognition for the remainder of this article.
When building a facial recognition edge device, choosing the right chipset based on the specific use case is the most consequential decision in regards to cost and performance. For example, a high-end NVIDIA, GPU chip has a higher upfront cost but can handle hundreds of video channels concurrently, thus reducing the number of costly workstations required to monitor a large facility. At the other end, a low-cost SoC from MediaTek or Broadcom will offer limited performance at speeds of about five frames per second and only frontal face recognition. Still, it will likely be powerful enough, and more affordable, for smaller use cases such as door access.
GPU chips are powerful hardware designed for facial recognition with outstanding performance. In general, substantial memory, high memory bandwidth, and a considerable amount of floating-point computation capability make GPUs the best option for complex, computation-hungry AI algorithms such as facial recognition. GPUs are also suitable for enabling facial recognition in surveillance systems, which requires simultaneously applying facial recognition across multiple video channels. A separate CPU chip is needed to build a system using these GPUs.
This year we recommend two GPUs from NVIDIA. They are extremely competent and provide a good balance between computational power and price.
GPU acceleration cards are most appropriate for workstations or on-premises servers. Our article How to Build a Proper Workstation for Facial Recognition provides more details.
These chips are complete yet affordable solutions for enabling facial recognition in small, mass-market IoT devices. They all integrate CPUs, resulting in simpler, cheaper options. SoCs may seem less capable than GPU options, with lower power consumption and a smaller form factor, but several FaceMe facial recognition models (detailed in section 3.4) can enable these AIoT devices while still providing highly accurate facial recognition results.
More details on building ARM-based AIoT devices can be found in this article: How to Build a Proper Workstation for Facial Recognition.
Chipsets are often designed to run on specific operating systems (OS). An excellent facial recognition engine should support as many chipset+OS combinations as possible. FaceMe supports one of the market’s most comprehensive lists of chipsets and more than 10 OSes:
CyberLink designed FaceMe to be highly versatile, allowing flexible customization of multiple combinations of hardware chips and OSes that match end users’ unique needs. FaceMe’s multi-OS support is ideal for cross-platform solutions. Developers have access to several GPU acceleration options, harnessing OpenVINO, NVIDIA CUDA/TensorRT, Intel Movidius, NVIDIA Jetson, Qualcomm SNPE, MediaTek NeuroPilot, and more, to speed up deep learning algorithms and further optimize performance.
Designing an excellent facial recognition system for a high-performance workstation or PC with GPU (or VPU) is never easy, because dozens of concurrent video streams run between the CPU, GPU, and memory over the system bus. Even an excellent facial recognition algorithm will be slow if implemented improperly on a system architecture level. The system architecture design should minimize the data flow between CPU, GPU, and memory.
FaceMe has an optimized system architecture that allows it to deliver the best performance. For example, on a single workstation FaceMe with NVIDIA RTX A6000 can handle 256 to 416 frames per second (the exact number may vary depending on which FaceMe facial recognition model is used). This is equivalent to handling 25 to 41 concurrent video channels (each with 10 frames per second) per workstation — an outstanding cost-performance offering.
Several cost-constrained use cases don’t require more than basic frontal face recognition (e.g., smart door locks). These devices are driving market demand for lightweight AI models that can achieve reasonable facial recognition at a lower cost. FaceMe provides three models for this purpose:
One important attribute of leading facial recognition solutions like FaceMe is its flexibility for all relevant types of hardware. FaceMe can be deployed across workstations, computers, mobile, and IoT devices. Let’s examine a few examples.
Organizations looking to deploy a facial recognition SDK over tens, or even hundreds, of video channels across large facilities will benefit from a workstation powered by a high-end GPU capable of simultaneously handling multiple IP camera video feeds. Different properties, including department stores, airports, industrial plants, and hospitals, have dozens or even hundreds of cameras for varied purposes: from security and access control to visitor behavior analysis, crowd management, and VIP customer identification. These are all use cases enabled or optimized by facial recognition. Connecting all the cameras to one or a few central workstations running facial recognition is the easiest, most robust, and likely most economical solution.
To learn more about the benefits of FaceMe for workstations, check out our partnership with VIVOTEK and the successful integration in their facial recognition solution.
Facial recognition is most commonly used on PCs for smaller operations or single uses. Take a store or restaurant that wants to identify VIPs, automatically clock in employees, or get alerts for block-listed people. In the case of the Covid-19 pandemic, it even has the ability to ensure everyone who enters the venue – employees and customers – wears a mask and does not have a high temperature. Store or restaurant management can install IP or USB cameras at the front and back doors and simply connect them to a PC that runs robust integrated facial recognition software. A value-priced, ready-to-deploy software solution that includes all these features is FaceMe Security. Learn more about crucial considerations for building top-of-the-line industrial PCs.
The potential of facial recognition technology on mobile devices goes way beyond unlocking a cell phone. One compelling fintech use case is the integration of eKYC (electronic know your customer) facial recognition technologies on mobile phones to strengthen identity verification for online banking, loan applications, insurance, and more.
The current fast-paced innovation in edge computing, driving better performance while cutting costs, also opens the door to endless IoT device use cases powered by facial recognition. Smart kiosks offer a compelling example. Frequent travelers are familiar with Global Entry and Clear kiosks, which use facial recognition. Now fast-food restaurants, hospitals, and hotels are deploying smart kiosks and integrating facial recognition. Major hotel chains have eagerly introduced self-check-in kiosks to cut wait times. Adding a facial recognition engine like FaceMe provides exciting, personalized experiences for guests by using their opted-in faces as the only ID they need through their stay. Take a look at our extended article on smart IoT devices.
As reviewed earlier, edge-based facial recognition is more secure than cloud-based options that require individuals’ pictures and videos to be sent through the internet to a cloud server; a process inherently vulnerable to attacks and leakage. Edge-based computing avoids most risks as the only data captured and stored takes the form of encrypted face templates, and the entire process can run without any cloud connection.
FaceMe uses AES-256 bit encryption to secure all data before it is stored in any database. AES is one of the best symmetric encryption algorithms, and 256-bit is the highest security confirmation. The face template is protected by storing the encrypted file (with a secret key) outside the source or platform server, keeping it secure even if physical devices are compromised or stolen.
Individuals must first opt-in to any facial recognition program requiring face enrollment. In edge-based solutions, the captured information will consist of template data (a mathematical number in a very high dimension) for future matching and identification purposes. The template doesn’t contain an actual face image, it can’t be used to recompose someone’s face, and it is kept separate from any personal information that could identify a person. The encrypted data that is captured when performing facial recognition is only used to establish a match with a pre-enrolled template stored in a secure database. Many data privacy laws and regulations (such as GDPR, CCPA, BIPA, and LGPD) count biometric data as personal information. Therefore, any business looking to employ face recognition must always obtain the user’s consent.
When evaluating various facial recognition providers it is important to locate their headquarters and essential facilities. For instance, the U.S. government has expressed fair concerns around surveillance technologies from companies in China and Russia – as they might not have adequate opt-in or data protection requirements. Most facial recognition solutions are safe and rigorously apply strict data and privacy protection standards. But as an end-user, you should be able to fully trust your service provider, especially when it comes to security, privacy, and protection of human rights.
While debates on the technology continue, there are many success stories where facial recognition deployment has increased safety, resulting in positive user experiences. A recent survey conducted by the Security Industry Association (SIA) found that the majority of Americans (68%) believe facial recognition can make society safer, with particular support for the technology in airports (75% for airlines; 69% for TSA), office buildings (70%), and banks (68%).
Key relevant use cases fall under five major categories:
Example: FaceMe Security offers a comprehensive facial recognition application in surveillance and security.
Facial recognition technology is poised to make our world a better place. But to do that, individuals everywhere need broader levels of education on ethical implementation to feel more comfortable with and accepting of businesses that have openly adopted this AI biometric technology as a new, safe standard.
Facial recognition and the potential it holds are more than what the fear-mongering makes it. It’s businesses keeping their employees safe by automating secure access control in the office. It’s retailers enhancing customer experiences in their stores. It’s manufacturers simplifying access to their many restricted areas. It’s banks and fintech companies introducing much stronger authentication and cutting-edge security controls. And that’s just the tip of the iceberg.
Facial recognition is the future of AI biometric technology. The industry must better educate consumers and debunk the many falsehoods circulated about this technology while explaining its positive value and potential for good. Facial recognition also needs to be regulated appropriately not to hinder innovation but to bring forth its many benefits.
ContactOur team of experts will be happy to answer your questions and schedule a demo. Free evaluation versions of the FaceMe® are available to qualified contacts.
