In this article, we will detail key considerations for building an ARM-based AIoT (Artificial Intelligence of Things) device to best equip it for facial recognition.
First, let’s start by defining an ARM-based AIoT device. AIoT devices are connected to the internet and run AI algorithms to carry out tasks. Inside the actual device are ARM-based SOC (systems-on-a-chip) or SOM (systems-on-a-module), with a CPU for general purpose processing and a GPU for accelerations. Many new AIoT vendors also incorporate APU (AI Processing Unit) or NPU (Neural Processing Unit) into the SOC or SOM so that AI applications can run more efficiently on the edge (instead of cloud servers).
ARM-based devices are energy efficient, using only a fraction of the power of x64 computers. ARM processors are often favored for thin edge-based AIoT devices, kiosks, industrial tablets, digital signs, wearables and mobile devices.
The use cases for AIoT devices go far and wide – bringing value to the smart home, smart office and more. We have outlined a few of the most prevalent examples by industry below.
There are many ways facial recognition benefits the manufacturing industry. First, it can streamline staff clocking in and out by instantly recognizing their face when they enter and leave the facility. With facial recognition-enabled AIoT devices stationed at doorways, it can be used to alert management if an unsuspected visitor attempts to enter the facility. It can also be used to ensure safety protocols are being met. For example, AIoT devices can connect into equipment to require face ID and perform verification before granting access to use the machine.
As they explore ways to differentiate from competitors, especially online stores, retailers can transform their customers’ in-store experience by deploying a range of AIoT devices using facial recognition. Imagine a shopper who, as they are browsing a store, walk up to a self-service kiosk that uses facial recognition to recognize them as a VIP customer and loyalty program member. It can bring up their profile or account, suggest items for the shopper and offer promotions. AIoT devices with facial recognition can also be used for streamlined checkout and, for retail employees, unlocking registers through face authentication.
With facial recognition, many office buildings’ access control and monitoring tasks can be seamlessly automated and made more secure. AIoT devices equipped with facial recognition can be placed at a building’s entryway and instead of requiring employees to use a key card, access will be granted based on face detection and recognition. The same solution can be applied to control access to restricted areas, like for IT personal access to the server room.
Another prominent use case for facial recognition and AIoT is for digital signage. For example, a traveler at an airport could walk up to a smart and interactive sign, use their pre-enrolled face ID to prompt the screen to display information about their flight and any changes that might have occurred, such as a gate change or delay.
AIoT is a key driver for healthcare innovation, finding its way in a number of areas, from medical equipment to patient identification to access monitoring. One compelling AIoT use case is medicine cabinets using facial recognition to grant access. Smart cabinets equipped with facial recognition offer greater security and control over protected substances, ensure only authorized people have access and automatically keeps a precise log of who took which drug and when.
The smart doorbell is an increasingly popular and consumer-friendly use case for facial recognition and AIoT. Swapping conventional locks for this type of device is extremely helpful for any family that is looking to improve home security. With facial recognition, smart doorbells can instantly identify and recognize neighbors, family and friends, then follow the rules set by the homeowner, such as automatically unlocking for family members or sending alerts when an unidentified individual approaches the door.
Similar to the smart doorbell, AIoT smart locks with facial recognition can improve the security for many businesses and across industries. Hotels and cruise ships can use them for keyless room access. Healthcare facilities can use them to restrict access to private areas. Commercial buildings can use them for employee access to the office. The use cases are endless and, unlike keys and access cards that can be borrowed or stolen, it only grants access to the authorized individuals.
A newer and emerging use case for AIoT and facial recognition is for cars, in particular its integration into infotainment systems. When one person sits in the driver seat, facial recognition identifies the specific driver and automatically applies their preferred settings, such as favorite radio stations, temperature, seat and mirrors’ position, etc.
Now that we have reviewed some use cases, it is important to discuss the top considerations for building an AIoT device with facial recognition. The specs of an AIoT device will vary depending on the use case and environment it will be used in. Performance requirements, cost constraints, power consumption and more should all be taken into account.
To determine the computing performance requirements, you will first need to clarify exactly what the device will be used for. Think about what you want the facial recognition to do and how complex the tasks will be. You will also want to consider how many faces will need to be detected and recognized per second. Do you only need the device to recognize one face at a time or do you need it to perform recognition of several faces concurrently? Take smart locks and door access control. If you only need it to recognize one face at a time and individuals approach the door at a controlled distance, a smaller AI model with less computing performance will work perfectly fine. The FaceMe® H (high precision) model is very sufficient for this exact use case. However, if you want to identify block listed people by performing facial recognition through a camera placed at the entrance of a busy shopping mall, you will need more computing power.
You also need to determine if the use case requires people identification or simply collecting demographic information. For example, with smart signs, if you only need it to identify demographic information (such as age and gender), you can get away with a lower computing performance. If you need the smart sign to also perform facial recognition and identification, you will need higher performance.
The more computing performance your AIoT device requires, the higher the cost will be. So if budget is a key consideration for your AI and facial recognition project, you will need to take performance requirements into consideration.
Each chip vendor (such as NVIDIA, Intel, Qualcomm, etc.) offers their own pricing model, with tiered product lines and tiered costs based on the interface (such as Wi-Fi, HDMI or USB). For example, the NVIDIA Jetson solution is a commonly used chipset because it delivers good performance. However, its performance comes at a higher price.
The best way to think about form factor is size. If you require the AIoT device to match certain dimensions, such as a smart lock that fits on a unique door, size will likely be a significant constraint. Not so much if the device powers a giant digital screen.
Computing performance and power go hand-in-hand. The higher the device’s performance, the more power is required. However, when vision technologies and AI algorithms are accelerated through dedicated chips, they become more power efficient.
The MediaTek i350 chipset, with the addition of an APU (AI Processing Unit), is a good illustrative example. The APU and NeuroPilot platform work together and enable the AI algorithm to perform faster, while saving power. When compared to a general purpose CPU instruction sets, an APU enhances efficiency for large parallel data computations like multiply-accumulate operations for multiple data points at once, which is commonly used in image processing for computer vision AI algorithms.
NVIDIA’s Jetson Family.
MediaTek’s Pumkin Developer Board.
Here are the four most common AIoT device configurations for facial recognition, in order of highest performance and cost to lowest.
The Jetson was first introduced by NVIDIA in 2014 as the Jetson TX. Since then, NVIDIA has rapidly revamped the product through several generations. The latest iteration is the Jetson Xavier NX. It provides a good balance between AI inference performance, power consumption and cost. Jetson platforms can handle more than one video input for facial recognition at a time, along with other applications. If you require a device to perform multiple tasks at once, Jetson is a great option.
NVIDIA provides tools and SDKs, including CUDA, TensorRT, DeepStream and more, which are some of the most mature options available on the market. All of these tools will enable the AI algorithm to run smoothly, which makes it easy to deploy facial recognition using Jetson. NVIDIA provides JetPack as the full development package, which includes a modified version of Ubuntu and all required toolchains and libraries.
Based on our testing, Jetson Xavier NX can support FaceMe VH model for 72 fps, each input image contains 1 face, and image size in 1080p resolution.
Jetson devices are also quite small, with a form factor as small as a mini PC less than 5” x 5” and 10 watts. However, these are some of the costlier devices.
Qualcomm provides a wide range of SOC, from those that go into mobile phones and tablets, like the Snapdragon line, to AIoT Edge products like the QCS410/610. The Qualcomm Neutral Processing SDK (SNPE), which enables algorithms to run on GPU or DSP chips as part of the SOC, is one of the best platforms to enhance runtime speed for AI algorithms and facial recognition.
Through our testing of FaceMe® on an Advantech industrial tablet with Snapdragon 660 using Android OS, we found the end-to-end facial recognition processing time to be reduced by 40% and CPU usage down by 64% by enabling AI to run with SNPE on GPU. The Snapdragon 660 ran the VH model for 16fps and H for 26fps, both at 720p.
The MediaTek i350 provides very reasonable performance at a competitive price for AIoT devices. it works best for devices with a touchscreen, such as POS terminals, smart locks, industrial tablets or display panels embedded into smart home appliances or fitness equipment. MediaTek is known for providing great turnkey solutions to unbranded mobile phones. They provide SDKs, sample codes and tools, which make it easy for anyone to build their own applications. i350 supports both Android and Ubuntu.
When we tested it with FaceMe®, MediaTek i350 ran the VH model for 8fps and the H model for 18fps, both at 720p.
NXP Semiconductors provide a wide variety of mixed signal and standard products for security, identification, automotive, networking, radio, analog and power management expertise. Currently, the NXP i.MX8M Plus is in pre-production phase. It is expected to be available in early 2021. This is the first time NXP includes a NPU into their SOC. This will effectively speed up AI algorithms and reduce CPU workloads when running AI applications.
In order to support the APU on NXP, TensorFlow Lite framework and inference engines are used. TensorFlow Lite is an open source project, mostly developed by Google, which uses it in many of its AI applications. It is a feature-rich, stable and mature platform. The i.MX8M Plus is great for porting a number of AI applications into the platform. If you have extra AI applications that you want to support on this platform, it is also very easy to port your applications on to it.
From our testing with FaceMe®, the iMX8M Plus ran the VH model for 5fps and the H model for 6.3fps, at 720p.
Here’s a short summary showing the performance of each platform
* For above testing, each image contains one face
The two most popular operating systems used in AIoT devices for facial recognition are Linux (Ubuntu and Debian) and Android. When selecting which OS is right for you, it will be important to consider the needs of your specific use case. FaceMe® is one of the most versatile facial recognition engines, which supports more than ten OS, including Linux, Ubuntu, Redhat, CentOS, Android, iOS or Windows. Of the top two, there are a few key differences to note.
Linux is very customizable and flexible. You can completely personalize components of the OS for your specific use case, making it as slim as possible. You can easily remove services you won’t need, enabling the software to run quicker and save power. Ubuntu is one of the most popular Linux variations and is widely used, from edge devices to cloud servers. Ubuntu Linux releases long-term support versions (LTS) about every two years to help keep the underlying technology up-to-date, with support provided for up to five years. Given that it is open source, there is a rich set of code libraries at your disposal, which can greatly speed up the development of your project.
Android is developed by Google, and its kernel is based on a modified version of Linux. It is most commonly used for smart phones and tablets. Android is a developer-friendly solution and is often the preferred OS for AIoT devices. Android toolchains, IDE (Android Studio) and SDKs are mature and easy to develop on from the start. The professional tools can be used at no cost and do not carry any royalties. Android applications are often developed in the programming language Java which, compared to C++, is much easier to code and manage resources or memory. It also supports the integration of components that are written in C++ into the Java program, enabling the benefits from both languages into the platform.
There are several popular AIoT configurations available to engineers and developers when building systems for facial recognition. Knowing what is best for you should not be a difficult feat. When evaluating options, the most important thing is understanding the use case. Then think about the requirements for performance, form factor, extensibility and budget.
Once you think you have the right build and design for your use case, we recommend conducting a proof of concept (POC) to identify any necessary fixes and improvements before officially deploying the application.
For a full overview of facial recognition, how it works and how it can be deployed, read Edge-based Facial Recognition - The Ultimate Guide.
For how is facial recognition used in 2021, read Facial Recognition – How is It Used in 2021?
Our team of experts will be happy to answer your questions and schedule a demo. Free evaluation versions of the FaceMe® are available to qualified contacts.