What is facial re­cog­ni­tion?

Facial re­cog­ni­tion is an al­gorithmic process that iden­ti­fies in­di­vidu­als or verifies their identity using unique biometric facial features. These systems provide a more efficient and accurate veri­fic­a­tion process compared to tra­di­tion­al methods; however, they also present sig­ni­fic­ant chal­lenges, es­pe­cially con­cern­ing data pro­tec­tion.

What is a facial re­cog­ni­tion system?

Facial re­cog­ni­tion is a tech­no­logy used to identify and verify in­di­vidu­als by analysing their unique facial features. These systems work by capturing distinct biometric markers – such as the shape of the eyes and nose – and con­vert­ing them into math­em­at­ic­al patterns, which are then matched against a database.

Modern facial re­cog­ni­tion systems can identify people in photos, videos, and even in real time. This enables, for example, two images to be compared to determine if they depict the same person. Ad­di­tion­ally, these systems can search extensive image or video archives to locate a specific face.

How does facial re­cog­ni­tion work?

Facial re­cog­ni­tion is a multi-stage process that draws on tech­no­lo­gies from the fields of computer vision and ar­ti­fi­cial in­tel­li­gence. While facial re­cog­ni­tion systems may vary in structure and operation, face iden­ti­fic­a­tion generally follows this basic process:

1. Face detection: The first step is locating a face in an image or video, typically using computer vision. This tech­no­logy captures facial data not only from the front but also in profile.
2. Face analysis: Next, the system analyses the face’s biometric features. Key variables include the depth of eye sockets, the distance between the eyes, the shape of cheekbones, and the contours of the lips, ears, and chin. Most systems use 2D images for this analysis, as these are easier to match with publicly available photos and databases.
3. Creation of a faceprint: The algorithm converts the captured facial features into a digital signature called a ‘faceprint’, which is a math­em­at­ic­al rep­res­ent­a­tion of the face. Since every person has their own facial features, this is unique, just like a fin­ger­print.
4. Com­par­is­on with database: The facial re­cog­ni­tion system compares the created faceprint with a database of known faces and evaluates the prob­ab­il­ity of a facial match. The highly developed com­par­is­on al­gorithms achieve high accuracy despite vari­ations in lighting, facial ex­pres­sions, and camera angles.

What are the most important areas of ap­plic­a­tion for facial re­cog­ni­tion systems?

Facial re­cog­ni­tion tech­no­lo­gies are now used for a wide range of ap­plic­a­tions. The most important areas of ap­plic­a­tion include:

• Smart­phones: Many smart­phones now offer facial re­cog­ni­tion as an option for unlocking the device. According to Apple’s statement on ‘Face ID’, the prob­ab­il­ity of a random face unlocking an iPhone is less than one in a million.
• Law en­force­ment: In the US and other countries, facial re­cog­ni­tion is in­creas­ingly used to locate in­di­vidu­als wanted by the police. Officers can even use mobile devices on-site to take a photo and compare it with databases in real time.
• Airports and border controls: A growing number of trav­el­lers carry biometric passports, allowing them to bypass long queues with ePassport control. Facial re­cog­ni­tion is also deployed at major events, like the Olympic Games, to enhance security.
• Banking: The banking apps of many financial in­sti­tu­tions allow users to au­then­tic­ate trans­ac­tions using facial re­cog­ni­tion. As no password or PIN needs to be entered, cyber criminals have no op­por­tun­ity to capture relevant data. This increases the security of online banking.
• Health­care: A facial re­cog­ni­tion system can be used to stream­line patient re­gis­tra­tion in hospitals. Facial re­cog­ni­tion also makes it possible to recognise emotions and pain in the people being treated.

Five practical ap­plic­a­tion examples for facial re­cog­ni­tion

• The e-commerce giant Amazon has developed a cloud-based facial re­cog­ni­tion system called Rekog­ni­tion. Beyond face-based user veri­fic­a­tion, it supports mood analysis and can scan videos to flag po­ten­tially offensive content.
• The tech company Apple allows its customers to unlock their smart­phone using facial re­cog­ni­tion. It’s also possible to use facial re­cog­ni­tion to log into apps and confirm purchases.
• British Airways enables trav­el­lers (depending on the airport) to verify their identity via facial re­cog­ni­tion. This elim­in­ates the need to show your passport or boarding pass.
• Coca Cola uses facial re­cog­ni­tion, among other things, to reward customers in China for recycling bottles and cans. In Australia, the company displays per­son­al­ised ad­vert­ising on its vending machines and in Israel, facial re­cog­ni­tion is used in con­nec­tion with event marketing.
• The social media platform Facebook has been using a facial re­cog­ni­tion tool in the USA since 2010 to auto­mat­ic­ally tag people in photos (only on a voluntary basis since 2019).

What’s the role of ar­ti­fi­cial in­tel­li­gence in facial re­cog­ni­tion?

Ar­ti­fi­cial in­tel­li­gence is essential to the de­vel­op­ment and operation of modern facial re­cog­ni­tion systems. AI tools enable the con­tinu­ous im­prove­ment of tech­no­logy through machine learning. Cor­res­pond­ing systems use the data provided to adapt their al­gorithms and thus become in­creas­ingly efficient over time.

Neural networks form the basis of modern facial re­cog­ni­tion systems. So-called con­vo­lu­tion­al neural networks (CNNs) are used to process facial images in stages, creating highly accurate fa­ce­prints even in sub­op­tim­al con­di­tions. CNNs can perform this pro­cessing in real time, making them par­tic­u­larly valuable for security-critical ap­plic­a­tions such as access control and sur­veil­lance systems.

What op­por­tun­it­ies and risks does the use of facial re­cog­ni­tion entail?

Facial re­cog­ni­tion offers con­sid­er­able potential, par­tic­u­larly in the areas of security and ef­fi­ciency. Today’s gen­er­a­tion of facial re­cog­ni­tion systems enable both fast and reliable iden­ti­fic­a­tion of people, which is useful for access control as well as for fighting crime and solving criminal offenses. Facial re­cog­ni­tion also improves the user ex­per­i­ence – for example as an unlocking option for smart­phones. Moreover, facial re­cog­ni­tion allows companies to provide per­son­al­ised services and stream­line processes.

The primary risks as­so­ci­ated with facial re­cog­ni­tion revolve around data pro­tec­tion and privacy. These systems enable in­di­vidu­als to be iden­ti­fied and monitored without their knowledge, raising the risk of misuse by gov­ern­ments, companies, and cy­ber­crim­in­als. Ad­di­tion­ally, experts have voiced concerns about the accuracy of facial re­cog­ni­tion, par­tic­u­larly regarding ethnic minor­it­ies, where misid­en­ti­fic­a­tion occurs more fre­quently.