Face Recognition Uses and Applications Across Sectors

Overview

Face recognition systems, originally developed for managing criminal photos and forensic investigations, are now used widely in security and commercial systems. Compared with other identity verification methods, face recognition is direct, user-friendly, convenient, and robust. Its use has expanded into many aspects of daily life, improving operational efficiency while providing high levels of security and reliability.

Primary Uses of Face Recognition

There are two main uses of face recognition in everyday scenarios. The first is 1:1 verification, which answers the question "Are you a specific person?" For example, the system is told "I am Zhang San" and then verifies whether the person standing in front of the device is indeed Zhang San. The second use is 1:N identification, where the system identifies "who am I" by matching a captured photo against a database of tens of thousands or millions of people.

These are two distinct applications. Today, 1:1 verification in controlled environments has become practical for many uses. By contrast, dynamic 1:N identification still faces significant challenges and has not yet met all practical application requirements.

Applications in Finance

Face recognition is increasingly applied in financial services for card issuance, remote loans, account opening, and payments. Both new internet financial institutions and traditional banks, securities firms, and insurers have adopted or tested face recognition technologies.

There are three main financial application areas: self-service terminals, counter systems, and mobile finance and marketing.

Self-service Terminals

Face recognition is integrated into self-service devices to compare a live-captured photo with stored photos or ID photos and to provide a similarity score. Staff can then decide whether to accept the transaction or perform manual review. Current self-service terminals can support functions such as self-service card issuance, account changes, and password resets.

Mobile Finance and Marketing

Remote identity verification via face recognition typically involves two approaches. Users can perform face verification using mobile phones and other personal devices. Financial institutions can also embed face recognition in portable terminals to provide on-site services to customers.

Counter Systems

Counter systems focus on networked face verification by comparing live photos with stored ID photos from national databases to achieve "identity-document consistency." This reduces subjective errors from visual inspection and is already used in counter account opening and other services at banks, insurance companies, and securities firms.

Applications in Public Security

Face recognition photo-matching systems enable rapid identity checks across databases containing thousands to millions of images, such as missing persons, detained individuals, permanent residents, temporary residents, wanted persons, and other priority lists. These systems accelerate identification for criminal investigations and other public security tasks.

Typical deployment involves establishing comparison centers at provincial and municipal levels and comparison terminals at local police stations, criminal investigation centers, or using GPRS/CDMA connections from mobile phones and PDAs to send photo comparison requests. By leveraging large collections of face images, these systems speed up suspect identification and support more intelligent, scalable public security operations.

Key public security applications include:

1. Tracking fugitives: Photo-matching systems speed up suspect identification and reduce reliance on low-efficiency manual searches.
2. Missing persons investigations: Submitted photos are fed into the system for matching, retrieval, and subsequent human confirmation.
3. Processing detained suspects at police stations: Photos of detained persons, including those whose identity is difficult to verify, are matched against various photo libraries to rule out suspects involved in major cases or to check criminal records.
4. Identification of unidentified deceased: Frontal photos are captured and submitted for matching. If images are damaged or distorted, composite images or artist renderings can be produced for comparison.
5. Witness-based screening: Witness descriptions can be used with face composite systems to generate images for database comparison.
6. Extracting frames from video surveillance: Surveillance images often suffer from blur, rotation, and poor lighting. Composite images or manual adjustments may be used to create a standard photo for comparison.
7. Crowd incidents in public venues: Long-focus cameras can capture facial close-ups for later matching, with composite image enhancement when necessary.
8. ID card photo matching: Comparing ID card photos with system databases can extract records similar to the ID photo, making existing ID photo resources useful for investigations.
9. Assistance in missing person searches, sheltering, compulsory treatment, and border control: Photo-matching increases efficiency and reduces manual workload.
10. Other queries: Searches for permanent residents, temporary residents, key individuals, and wanted persons across various databases.

Face recognition and analysis techniques must support accurate computation, retrieval, and filtering across databases containing tens of millions of photos, ensuring results are suitable for manual secondary review. The typical system architecture includes image acquisition, preprocessing, feature extraction, database search, and human review stages.

Applications in Social Security

To protect pension funds and prevent fraudulent claims, face recognition has been introduced into identity verification systems for the social security sector. Using unique facial features, these systems verify beneficiaries' identities accurately and conveniently, reducing payment errors and fraud.

Self-service verification terminals automatically read ID information, capture live facial images, extract the ID photo, and compare the live image with the ID image using high-resolution face recognition analyzers. This process helps prevent identity fraud involving photos, videos, or 3D models.

Applying face recognition in social security can reduce pension leakage, lower workload, improve service quality, reduce friction in interactions, and make processes more convenient for retirees.

Applications in Airport Border Control

Air travel has become a major transportation mode, increasing the need for efficient and robust airport security. Many airports now use high-resolution face-to-ID comparison systems to assist manual inspection.

These systems compare ID photos with live facial images to quickly determine whether the ID belongs to the person presenting it, with reported recognition rates above 98% under suitable conditions. When discrepancies are detected, the system prompts security personnel for additional checks.

Cameras at security checkpoints capture facial images as passengers enter departure areas and compare them with ID photos. The system can tolerate changes such as hairstyle or makeup. Captured facial images can also be stored as surveillance data for later retrieval or linked to law enforcement databases for evidence and verification.

In airport border control, face recognition helps manage passenger flows and verify traveler identities, reducing opportunities for suspicious persons to board or escape scrutiny.

Technical Challenges

Despite decades of research, face recognition still faces several unresolved technical challenges:

1. Lighting: Illumination is a longstanding issue in machine vision. Because faces are three-dimensional, lighting-induced shadows can amplify or obscure facial features.
2. Pose and expression: Pose variations involve rotations of the head around three axes, which can cause partial loss of facial information, especially for out-of-plane rotations. Most algorithms are optimized for frontal or near-frontal faces; extreme pitch or yaw and large expression changes such as crying, laughing, or anger can significantly reduce accuracy.
3. Occlusion: In uncontrolled capture scenarios, occlusions from glasses, hats, or other accessories can produce incomplete face images, impairing feature extraction and potentially causing detection failures, which is especially problematic in surveillance environments.
4. Age variation: Facial appearance changes with age, especially for younger people. A person’s appearance may change substantially from youth to old age, lowering recognition rates across different age groups.
5. Inter-subject similarity: Human faces share similar structures and features, which makes distinguishing between individuals more challenging even though facial structure aids detection.
6. Image quality: Sources vary in resolution and noise. Low-resolution or noisy images from mobile cameras or remote surveillance pose challenges for effective recognition, while the impact of very high-resolution images also requires research.
7. Limited samples: Statistical learning methods, which dominate the field, require large training sets. Face images occupy a complex manifold in high-dimensional space, and available samples typically cover only a small portion of that space, making learning from small samples an open problem.
8. Massive data scale: Classical methods such as PCA and LDA train easily on small datasets but struggle or fail when scaled to massive datasets.
9. Large-scale recognition: As database sizes grow, algorithm performance tends to degrade, posing challenges for accuracy and efficiency at scale.