Biometric technology continues to develop rapidly, offering safety and security to all facets of life. By using an individual’s physical characteristics, which are almost impossible to replicate, biometric technology proves a better choice than traditional security methods that are easily lost or stolen, such as personal identification cards, passwords/PINs, and keys. Though it has been around for decades, primarily through fingerprints, biological samples (DNA, blood, saliva, etc.), and even iris scanning, one biometric technology deserves a closer look: facial recognition.
Facial recognition solutions have grown in popularity in recent years. One of the most ubiquitous use cases is mobile phones. Many consumers worldwide interact with this biometric technology daily, to secure and unlock their phones.
However, the possibilities extend far beyond personal mobile devices and provide significant benefits in safety, security, and efficiency across a variety of industries. As a compelling, comprehensive, and rewarding technology, it is vital to understand how facial recognition works, how it can be deployed and optimized, its technical considerations and specifications, the varied use cases, and of course, its potential.
Facial recognition biometrically identifies facial vectors and features, matching them with pre-enrolled individuals. Recent advancements in AI technologies, based on deep neural networks (DNNs), have dramatically improved precision, unlocking a wealth of new use cases.
The technology leverages proprietary AI algorithms and mathematical equations to create an individual's template by measuring facial variables – nose depth and width, forehead length, and eye shape. Facial recognition then compares the generated template with existing templates in a database. If there is a match, it can confirm an individual’s identity.
Building on its facial recognition expertise, CyberLink leveraged deep learning and neural networks to create FaceMe®, an AI-based facial recognition engine. CyberLink continues to push the boundaries of these technologies to enhance its AI-based models, resulting in one of the world’s most accurate, secure, and flexible edge-based solutions.
Facial recognition is by far the most powerful and relevant AI biometric technology. It has vast abilities and can carry out several tasks beyond face detection and recognition. The more robust and feature-forward a facial recognition platform, like FaceMe, the more benefits and fewer biases it brings.
The key features of a facial recognition engine are:
Face detection is the first step the engine takes to confirm the presence of faces as they appear on a live camera feed, a video recording or as it scans still image captures. The whole field of view is scanned for any area containing full or even partial human faces. Fast and precise face detection is a critical first step to ensure the performance of the entire facial recognition process. FaceMe can detect more than one face simultaneously, count how many faces are present, and perform detection on each of them individually.
After face detection, feature extraction is the next step. The engine first extracts an n-dimensional vector set (a template) from the facial image. Achieving very high precision requires a high “n” value. FaceMe only stores encrypted template data. To fully ensure privacy, no actual images of faces are stored on our platform. Next, the template extracted from an individual’s face is used for matching or searching.
If the goal is to verify a person’s identity, in other words to answer the question, “Is this person who they say they are?” then the facial recognition engine performs a 1:1 face match. If they are already enrolled, the engine extracts a facial template from the camera feed’s live view and checks if it matches the template on file for this person.
When the person is not pre-enrolled or when the process requires a verification to an officially recognized identity document, the engine can be connected to an ID scanner to extract a template from the face on the picture ID and use the same 1:1 method to check if it matches the template extracted from the live view of this person’s face. The most common use of 1:1 face match is arguably to control smartphone or computer access, as well as to login to an app or online service. iPhone users have enthusiastically embraced Apple’s Face ID, which performs 1:1 face match, essentially as described here.
If the goal is to answer the question, “Who is this person?” the engine completes a 1:N face search. It compares the individual’s facial template to the pre-enrolled faces in the database and confirms the person’s identity if it finds a match. The most common use of face search is a security and surveillance system using a camera to verify that a person belongs to a particular company – if their face was pre-enrolled in the company’s facial database, face search will confirm it is them and grant them access.
Some key use cases require additional features, such as the following:
Face attribute detection, also known as face analysis, identifies and analyzes characteristics such as age, gender, facial expression, and head orientation or movements (e.g., nodding, shaking). This feature is a crucial enabler of smart retail and digital signage for use cases like delivering customized ads and messaging to targeted audiences or collecting detailed visitor statistics.
While other technologies allow the collection of data such as time spent by an individual at a location, the path taken by that individual and if they make a stop at the cash register before leaving, FaceMe’s technology adds for the first time precise demographic data and sentiment analysis to these individuals. By default, to ensure privacy compliance, FaceMe aggregates the facial attribute data without generating a facial template for individuals who have not expressly given their consent to biometric identification. Its analytical tools can generate, in real time, a precise detailed distribution for each attribute, for any time interval, for each camera, for any cross-tabulation. It can also aggregate the data to any level of abstraction.
Depending on the workflow and business practices, it can either generate reports or directly enhance the visitor’s experience, by sending alerts to the staff or directly feeding data to signage’s content management system, even adjusting the product mix to be sent from the warehouse to the point of sale.
Mask detection is one of the newest and most valuable features to ensure health and safety in public or private spaces during the pandemic. CyberLink’s FaceMe SDK is a cross-platform facial recognition SDK that offers optimized mask detection and facial recognition when wearing a mask. It recognizes health-compliant masks and verifies if the nose and mouth are properly covered while performing highly accurate face detection and recognition. Additionally, CyberLink’s FaceMe Security helps existing security solutions upgrade their mask detection features so that employees can enter the office while still wearing masks.Learn More : Toyota Motor Corporation adopts ITOCHU Techno-Solutions’ “Vehicle Inspection Information System” integrated with FaceMe Facial Recognition.
Liveness detection, often referred to as anti-spoofing technology, is used for biometric fraud protection against a range of techniques that have been used to impersonate authorized people and gain access illegally.
There are three main ways imposters attempt to spoof facial recognition systems. The least sophisticated approach consists in placing an authorized individual’s photo in front of the camera. Almost any recent solution will detect the absence of facial movement. Showing video instead of a still picture seems the logical solution. But depth-sensing cameras easily detect the subterfuge, which is why we occasionally see the use of sophisticated 3D masks that show the facial features of the authorized individual in a realistic shape.
One advantage over other biometric identification technologies is that precise facial recognition can be performed with simple 2D webcams or security IP cams, just as well as when using sophisticated 3D cameras equipped with the latest laser or infrared technologies. Specific use cases or constraints as varied as cost, location, form factor and speed inform camera decisions that directly impact the availability of anti-spoofing options and their performance.
2D cameras typically catch spoofers through interactive measures such as performing a set of motions randomly chosen by the access computer before being granted access, as well as non-interactive measures which are unique to each solution provider and its AI algorithm.
3D cameras on the other hand, perform depth detection, precisely identifying a spoofing attempt within a few milliseconds. 3D cameras generally provide a superior experience, but they are costlier, while 2D alternatives can also provide accurate anti-spoofing at a fraction of the cost, wherever a small delay is acceptable. Costs and size of 3D cameras are rapidly going down, and a new generation of time-of-flight sensors can be attached to 2D installations, adding depth detection at a fraction of the cost of new 3D devices. The development and availability of 3D solutions has grown massively in recent years and has made this technology very accessible.
The more advanced attack method, use of a physical 3D mask, requires a software-based defense. Advanced facial recognition systems like FaceMe leverage AI anti-spoofing technology to analyze natural facial movements and determine whether the face in front of the camera is displaying expressions or emotions as a natural human face would. Facial recognition engines process such a massive amount of data, that even a well-made 3D mask has many tells in the form of highly specific facial vector data that contrasts to that of a real human face.
In this relatively new field, industry beacon iBeta offers tests like the Presentation Attack Detection Test which holds facial recognition software to rigorous anti-spoofing standards and aids in choosing the right solution. Only a few facial recognition solutions have passed both iBeta anti-spoofing tests. FaceMe achieved a near-perfect score in iBeta's Level 1 test (photos and videos) and detected 100% of spoofing attempts using 3D masks in the advanced Level 2 Anti-Spoofing Test, attesting that it complies with ISO 30107-3 testing and reporting requirements.
The most precise facial recognition engines are characterized by a low false non-match rate (FNMR) and an extremely low false match rate (FMR). A false non-match is the failure to match two facial templates from the same person, whereas a false match is when one person’s face is matched with someone else’s.
The National Institution of Standards and Technology (NIST) in the U.S.A. is the governing body that determines how well a given facial recognition algorithm achieves a set of common tasks. NIST’s Facial Recognition Vendor Test (FRVT) uses different datasets to evaluate the performance of an algorithm. For example, the VISA category tests a facial recognition algorithm’s ability to correctly identify an individual based on a passport photo, while the WILD test uses random, non-constrained photojournalism-style images.
FaceMe has achieved one of the highest NIST VISA test ratings, with 99.83% at 1E-6 accuracy. 1E-6 means the False Match Rate (FMR) is 1 in 1,000,000 (0.000001). By comparison, contemporary smartphone Face ID technology offers an FMR of about 96% at 1E-4 accuracy.
Beyond the algorithm, some of the main factors affecting accuracy are camera resolution, camera positioning, lighting, clarity, and camera type. Facial recognition engines generally work adequately with 720p cameras, but a 1080p resolution device is generally recommended.
Facial recognition can be deployed using cloud-based technology, often using popular Microsoft or Amazon solutions, or by integrating SDKs or software onto edge devices. While each approach offers distinct benefits, edge-based facial recognition is generally deemed superior, offering better speed, security, affordability, flexibility, and versatility.
When deploying facial recognition at the edge, the technology is embedded in local devices: a smart lock, mobile phone, point-of-sale (POS) system, interactive kiosk, digital signage, or similar. Edge devices run facial recognition solutions quickly, securely and with extreme precision. There is no inherent need for a network or cloud connection except to access the facial template database if it is not stored on the edge device. Even then, only a small, encrypted, template is transmitted for validation. The database’s encrypted templates are often hosted on a remote server, resulting in a secure operation completed within milliseconds.
Businesses that implement facial recognition from the ground-up will benefit from an edge-based approach with IoT devices carrying out the required tasks. Financial institutions make a compelling case for edge-based facial recognition systems, as many banks do not allow internet connections for security reasons.
The low cost, flexibility, and scalability of edge-based facial recognition make it the best option for most end users.
CyberLink’s FaceMe SDK perfectly exemplifies a best of breed edge-based facial recognition solution. This highly competitive and flexible face recognition SDK is easy to integrate across a wide range of edge devices. FaceMe leads the way by supporting a variety of chipsets and OSes, and its highly accurate AI engine is ranked one of the best in the NIST Face Recognition Vendor Test (FRVT). Through constant innovation, the technology meets the highest accuracy and security standards for deployments across a wide range of industries and use cases including security, access control, public safety, smart banking, smart retail, smart city, and home protection.
Let's examine trends for (1) cost, (2) response time, and (3) service availability:
AI is very computationally demanding, and cloud computing is not cheap, so choosing between the edge or cloud is a key design decision. Edge devices have a cost advantage over the cloud, which typically charges an incremental amount for each face recognized, which can quickly add-up. Meanwhile, cloud-based computing might be cheaper on a very small scale, in applications not exceeding more than a few faces per hour. As AI-acceleration processor pricing has come down, the cost advantage offered by edge-based solutions has become increasingly apparent.
No internet service is immune from interruptions or unexplainable low bandwidth issues. Imagine if the lock on your front door at home stopped working because a cloud-based facial access solution lost connectivity. Edge-based facial recognition isn’t as vulnerable to internet issues.
With its inherent superiority and huge momentum in innovation, edge-based technology will be a vital driver of the future success of facial recognition. Therefore, we will focus on edge-based facial recognition for the remainder of this article.
No internet service is immune from interruptions or unexplainable low bandwidth issues. Imagine if the lock on your front door at home stops working because it depends on a cloud-based facial access solution. Edge-based facial recognition doesn’t have such vulnerabilities to internet issues.
With its inherent superiority and huge momentum in innovation, edge-based technology will be a vital driver of the future success of facial recognition. Therefore, we will focus on edge-based facial recognition for the remainder of this article.
When building a facial recognition edge device, choosing the right chipset based on the specific use case will be the most consequential decision regarding cost and performance. For example, a high-end NVIDIA GPU chip may have a higher upfront cost, but can handle hundreds of video channels concurrently, thus reducing the number of costly workstations required to monitor a large facility.
On the other hand, a low-cost SoC from MediaTek or Broadcom may offer limited performance, with processing speeds of about five frames per second and only frontal face recognition. Still, it will likely be powerful enough, and more affordable, for smaller use cases such as door access.
GPU chips are powerful hardware designs capable of supporting facial recognition with outstanding performance. Their substantial on-board memory, high memory bandwidth, and considerable floating-point computation capabilities make GPUs the best option for the complex, computation-hungry AI algorithms behind facial recognition. GPUs are also suitable for enabling facial recognition in surveillance systems, which requires simultaneously applying facial recognition across multiple video channels. A separate CPU chip is needed to build a system using these GPUs.
This year we recommend two GPUs from NVIDIA. They are extremely competent and provide a good balance between computational power and price.
GPU acceleration cards are most appropriate for workstations or on-premises servers. Our article How to Build a Proper Workstation for Facial Recognition provides more details.
These chips are complete yet affordable solutions for enabling facial recognition in small, mass-market IoT devices. They all integrate CPUs, resulting in simpler, cheaper hardware options. SoCs may seem less capable than discrete GPU options, with lower power consumption and a smaller form factor, but several FaceMe facial recognition models (detailed in section 3.4) can run on these AIoT devices, while still providing highly accurate facial recognition results.
More details on building ARM-based AIoT devices can be found in this article: How to Build a Proper Workstation for Facial Recognition.
Chipsets are often designed to run on specific operating systems (OS). An excellent facial recognition engine should support as many processor + OS combinations as possible. FaceMe supports one of the market’s most comprehensive lists of chipsets and more than 10 OSes:
CyberLink designed FaceMe to be highly versatile, allowing flexible customization of multiple combinations of hardware chips and OSes that match end users’ unique needs. FaceMe’s multi-OS support is ideal for cross-platform solutions. Developers have access to several GPU acceleration options, harnessing OpenVINO, NVIDIA CUDA/TensorRT, Intel Movidius, NVIDIA Jetson, Qualcomm SNPE, MediaTek NeuroPilot, and more, to speed up deep learning algorithms and further optimize performance.
Designing an excellent facial recognition system for a high-performance workstation or PC with GPU (or VPU) needs careful consideration, as dozens of concurrent video streams may run between the CPU, GPU, and memory over the system bus. Even an excellent facial recognition algorithm will be slow if implemented improperly on a system architecture level. The system architecture design should minimize the data flow between CPU, GPU, and memory.
FaceMe has an optimized system architecture that allows it to deliver the best performance. For example, on a single workstation, FaceMe with an NVIDIA RTX A5000 can handle 489 to 727 frames per second (the exact number may vary depending on which FaceMe facial recognition model is used). This is equivalent to handling 48 to 72 concurrent video channels (each with 10 frames per second) per GPU — an outstanding cost-performance offering.
Several cost-constrained use cases don’t require more than basic frontal face recognition (e.g., smart door locks). These devices are driving market demand for lightweight AI models that can achieve reasonable facial recognition at a lower cost. FaceMe provides three models for this purpose:
One important attribute of a leading facial recognition solution like FaceMe is its flexibility regarding the hardware it can run on. FaceMe can be deployed across workstations, computers, mobile, and IoT devices. Let’s examine a few examples.
Organizations looking to deploy a facial recognition SDK over tens, or even hundreds, of video channels across large facilities will benefit from a workstation powered by a high-end GPU capable of simultaneously handling multiple IP camera video feeds. Locations such as department stores, airports, industrial plants, and hospitals, may employ dozens or even hundreds of cameras for varied purposes. Typically, they will be used for security and access control to visitor behavior analysis, crowd management, and VIP customer identification. These are all . Connecting all the cameras to one or a few central workstations running facial recognition is the easiest, most robust, and likely most economical solution.
To learn more about the benefits of FaceMe for workstations, check out our partnership with VIVOTEK and the successful integration in their facial recognition solution.
Facial recognition is most commonly powered by PCs for smaller scale deployments. Obvious examples are a store or restaurant which wants to identify VIPs, automatically clock in employees, or get alerts for block-listed people. During a pandemic, a clear advantage of facial recognition is that it enables the monitoring of mask wearing and body temperature for both employees and customers. Store or restaurant management can install IP or USB cameras at the front and back doors and simply connect them to a PC that runs robust integrated facial recognition software. A value-priced, ready-to-deploy software solution that includes all these features is FaceMe Security. Learn more about crucial considerations for building top-of-the-line industrial PCs.
The potential of facial recognition technology on mobile devices extends far beyond unlocking a cell phone. One compelling fintech use case is the integration of eKYC (electronic know your customer) facial recognition technologies on mobile phones to strengthen identity verification for online banking, loan applications, insurance, and more.
The current fast-paced innovation in edge computing, driving better performance while cutting costs, also opens the door to endless IoT device use cases powered by facial recognition. Smart kiosks offer a compelling example. Frequent travelers are familiar with Global Entry and Clear kiosks, which use facial recognition. Now fast-food restaurants, hospitals, and hotels are deploying smart kiosks and integrating facial recognition. Major hotel chains have eagerly introduced self-check-in kiosks to cut wait times.
Adding a facial recognition engine like FaceMe provides exciting, personalized experiences for guests by using their opted-in faces as the only ID they need through their stay. Take a look at our extended article on smart IoT devices.
As reviewed earlier, edge-based facial recognition is more secure than cloud-based options that require individuals’ pictures and videos to be sent through the internet to a cloud server, a process inherently vulnerable to attacks and leakage. Edge-based computing avoids most risks as the only data captured and stored takes the form of encrypted face templates, and the entire process can run without any cloud connection.
FaceMe uses AES-256 bit encryption to secure all data before it is stored in any database. AES is one of the best symmetric encryption algorithms, and 256-bit is the highest security option supported. The face template is protected by storing the encrypted file (with a secret key) outside the source or platform server, keeping it secure even if physical devices are compromised or stolen.
Individuals must first opt-in to any facial recognition program requiring face enrollment. In edge-based solutions, the captured information will consist of template data (a mathematical number in a very high dimension) for future matching and identification purposes. The template doesn’t contain an actual face image, it can’t be used to recompose someone’s face, and it is kept separate from any personally identifiable information (PII) that could identify a person.
The encrypted data that is captured when performing facial recognition is only used to establish a match with a pre-enrolled template stored in a secure database. Many data privacy laws and regulations (such as GDPR, CCPA, BIPA, and LGPD) count biometric data as personal information. Therefore, any business looking to employ facial recognition must always obtain the user’s consent.
When evaluating various facial recognition providers, it is important to locate their headquarters and essential facilities. For instance, the U.S. government has expressed fair concerns around surveillance technologies from companies linked to China and Russia – as they might not have adequate opt-in or data protection requirements. Most facial recognition solutions are safe and rigorously apply strict data and privacy protection standards. But as an end-user, you should be able to fully trust your service provider, especially when it comes to security, privacy, and protection of human rights.
While debates on the technology continue, there are many success stories where facial recognition deployment has increased safety, resulting in positive user experiences. A recent survey conducted by the Security Industry Association (SIA) found that most Americans (68%) believe facial recognition can make society safer, with particular support for the technology in airports (75% for airlines; 69% for TSA), office buildings (70%), and banks (68%).
Key relevant use cases fall under five major categories:
Example: FaceMe Security offers a comprehensive facial recognition application in surveillance and security.
AI biometrics can power many use cases in specific vertical segments. For more on these use cases, read our article highlighting the Top 7 Use Cases for Facial Recognition in 2022.
Facial recognition technology is poised to make our world a better place. But, to do so, individuals need broader levels of education on ethical implementation to feel more comfortable with, and accepting of, businesses that have openly adopted AI biometric technology as a new, safe standard.
Facial recognition, and the potential it holds, is clearly a more positive advancement than detractors would like to admit. It is businesses keeping their employees safe by automating secure access control in the office. It is retailers enhancing customer experiences in their stores. It is manufacturers simplifying access to their tiers of restricted areas. It is banks and fintech companies introducing much stronger authentication and cutting-edge security controls. And that’s just the tip of the iceberg.
Facial recognition is the future of AI biometric technology. The industry needs to better educate consumers and debunk the many falsehoods circulating about this technology, while explaining its positive value and potential for good. Facial recognition also needs to be appropriately regulated to not hinder innovation but bring forth its many benefits.
Join thousands of successful professionals and stay updated with the latest trends in AI facial recognition. Subscribe to our newsletter today!