What facial recognition does

Facial recognition is software that identifies or verifies who you are by analyzing the unique patterns in your face. It captures an image of your face, converts that image into mathematical data, and compares it against a database of other faces to find a match or confirm your identity.

The technology works in two main ways. Identification means the system searches a database to figure out who you are — like when an airport camera scans your face against a watchlist. Verification means the system confirms you are who you claim to be — like when your phone unlocks because it recognizes your face.

Facial recognition is not new, but it has become much faster and more accurate in the last five years. Modern systems can work in poor lighting, at angles, and even with partial face coverage, though accuracy still varies by lighting conditions, camera quality, and the diversity of faces in the training data.

Key Takeaways

  • Facial recognition converts your face into mathematical patterns and compares them to a database to identify or verify who you are.
  • You encounter it in phones, airports, law enforcement databases, retail stores, and social media platforms, though you may not always know it is running.
  • The technology can be inaccurate, especially for people with darker skin tones, and accuracy depends on lighting, camera quality, and the faces used to train the system.
  • Your face is collected and stored by private companies and government agencies, and the rules about how they use it vary widely by location and organization.
  • You have limited control over facial recognition in public spaces, but you can disable it on your own devices and understand where it is being used.

Where facial recognition is used right now

Your phone likely has facial recognition built in. iPhones use Face ID, and many Android phones use similar systems to unlock the device or authorize payments. These systems store your face data on your device itself, not on a company server, though the exact storage method varies by manufacturer.

Airports and border control use facial recognition to verify your identity against your passport or visa. The U.S. Department of Homeland Security has used it at major airports for years. Some countries, including China and the United Kingdom, use it to identify people in public spaces and match them against criminal databases.

Law enforcement agencies in the United States maintain facial recognition databases. The FBI's Next Generation Identification system contains millions of photos, many from driver's licenses and mugshots. Local police departments also use facial recognition to search for suspects, though policies about accuracy standards and human review vary by department.

Retailers and shopping centers use facial recognition to track foot traffic, identify repeat customers, and in some cases flag people suspected of shoplifting. Social media platforms like Facebook and Google Photos use it to organize your photos and suggest tags. Some employers use it for time tracking or building access.

How accurate facial recognition actually is

Accuracy depends on the specific system, the quality of the image, and the faces being compared. Under ideal conditions — good lighting, straight-on angle, high-resolution camera — modern facial recognition can be very accurate. The National Institute of Standards and Technology tested leading systems in 2023 and found error rates below 1 percent for some algorithms.

But real-world conditions are rarely ideal. Poor lighting, angles, masks, glasses, and aging all reduce accuracy. More importantly, accuracy is not equal across all groups. Multiple studies have found that facial recognition systems are significantly less accurate for people with darker skin tones, particularly dark-skinned women. This gap exists because many systems were trained on datasets that contained far more light-skinned faces.

When law enforcement uses facial recognition to identify suspects, the system typically returns a list of possible matches, not a definitive identification. An officer must then manually review those matches and decide whether to investigate further. The problem is that officers sometimes treat a facial recognition match as stronger evidence than it actually is, especially if the system has not been tested for accuracy on the specific population being searched.

What data is collected and stored

When facial recognition is used on you, the system captures or accesses an image of your face and converts it into a mathematical representation called a faceprint. This faceprint is what gets stored and compared, not the original photo, though the original photo is often kept as well.

Who stores this data depends on where the recognition happens. If you unlock your phone with Face ID, Apple or Samsung stores the faceprint on your device. If an airport scans your face, the Department of Homeland Security stores it. If a retailer uses facial recognition in their store, that company stores the data. If a police department uses facial recognition to search for you, they store your faceprint in their database.

Retention policies vary widely. Some organizations delete facial data after a transaction is complete. Others keep it indefinitely. There is no single federal law in the United States that governs how long facial recognition data must be kept or when it must be deleted. Some states, including Illinois and Texas, have passed laws requiring consent before facial recognition is used, but enforcement is inconsistent.

The accuracy problem and who it affects most

The gap in accuracy across racial groups is not a minor technical issue — it has real consequences. When a facial recognition system is less accurate for a particular group, that group is more likely to be misidentified, and misidentification can lead to wrongful investigation or arrest.

Several documented cases show this risk. In 2020, a Black man in Detroit was arrested based partly on a facial recognition match that later proved incorrect. In 2021, another Black man in New Jersey was arrested after facial recognition misidentified him. Both cases involved systems that had not been validated for accuracy on Black faces.

The problem is compounded because facial recognition is often used as a starting point for investigation, and once an officer has a suspect name, confirmation bias can set in. An officer may be more likely to find evidence that matches the facial recognition result, even if the match was wrong.

What you can control and what you cannot

On your own devices, you have choices. You can disable Face ID on your iPhone in Settings > Face ID & Passcode. You can turn off facial recognition in Google Photos by going to Settings > Search and turn off "Face grouping." You can adjust privacy settings on Facebook and other social media to limit how your face is used.

In public spaces, your control is much more limited. You cannot prevent a retailer from using facial recognition in their store, though some states require them to post a notice. You cannot prevent law enforcement from searching facial recognition databases for your image. You cannot prevent an airport from scanning your face at security.

What you can do is understand where facial recognition is being used. Many organizations are required to disclose it, though the disclosure is often buried in a privacy policy. If you are concerned about facial recognition in a specific location — a store, workplace, or government building — you can ask directly whether it is being used.

The difference between identification and verification

These two uses of facial recognition raise different concerns. Verification — confirming you are who you claim to be — is generally lower-risk because you are consenting to the scan and you know it is happening. Unlocking your phone with Face ID is verification. Scanning your passport at an airport is verification.

Identification — searching a database to figure out who you are — is higher-risk because you may not know it is happening and you have not consented. A police officer using facial recognition to search for a suspect is identification. A retailer using facial recognition to identify a shoplifter is identification. In these cases, accuracy matters much more because a wrong match can have serious consequences.

Some jurisdictions have started restricting identification use. San Francisco banned government use of facial recognition in 2019, though the ban has exceptions for airports and law enforcement investigations. Other cities have passed similar restrictions. These bans typically allow verification uses while restricting identification uses in public spaces.

Frequently Asked Questions

Can facial recognition work if I am wearing a mask or glasses?

Modern systems can work with masks and glasses, but accuracy drops. During the COVID-19 pandemic, mask-wearing reduced accuracy significantly for many systems. Glasses can cause glare or reflections that interfere with the scan. The best results still come from an unobstructed, well-lit face.

Is facial recognition the same as a fingerprint scan?

No. Fingerprint scanning reads the unique ridges on your fingertips. Facial recognition reads the proportions and patterns of your face. Fingerprints do not change over time, but faces do — aging, weight changes, and facial hair all affect facial recognition accuracy. Fingerprints are also harder to capture without your knowledge, while facial recognition can work from a distance.

What should I do if I think I was misidentified by facial recognition?

If you were arrested or investigated based on facial recognition, you have the right to know that facial recognition was used. You can request the facial recognition report and the original match score through discovery in a criminal case, or by filing a public records request if it involves a government agency. Having an attorney review the match and the system's accuracy is important.

Can I opt out of facial recognition?

For devices you own, yes — you can disable it in settings. For public spaces and government databases, it depends on your location. Some states require opt-in consent before facial recognition can be used on you. Most do not. If you are concerned about a specific use, contact the organization directly and ask about their facial recognition policy.

Does facial recognition work better on some faces than others?

Yes. Systems are consistently less accurate on darker skin tones, particularly for women. This is because most training datasets contained more light-skinned faces. Some newer systems have been trained on more diverse datasets and show better accuracy across groups, but the gap has not been eliminated. When facial recognition is used in law enforcement or other high-stakes situations, this accuracy gap is a serious concern.