Building Your Own Facial Recognition System

In a world more aware than ever about the data it produces and the surrounding privacy issues, facial recognition is at the top of the list of concerns, and rightfully so. There are already cases of the terrible applications of facial recognition in China where the government’s racial profiling of Uighur Muslims. But it’s not just a Chinese problem, there are issues all over, for example, in Russia street cameras comb the streets for “people of interest”, in Israel there have been reports of companies tracking Palestinians in the West Bank. Even in America, where there have been large issues for the tech giant Amazon in selling facial recognition software to law enforcement departments. It goes further than just the nefarious applications, these companies store your information when you leverage their services, meaning your digital fingerprint isn’t owned by you — it’s owned by FAANG. In order to help return that ownership back to you, let’s build our own facial recognition app, that way we can own our data. For this we’ll use a free dashboard from devias on github and build a login page that uses our face to log us in.

Prerequisites

So, before we get started, let’s go over some prerequisites for this project. Here are the things you’ll need in order to successfully build your own facial recognition system.

• Understanding how facial recognition works
• A T3 API Key
• Python3.7+
• Postgresql version 12.6
• PostgREST version 7.0.1
• React version 17.0.2
• Node version 10.19.0

Those who just want to look at the code, here’s a link to the repo. Now, let’s develop a better understand of how facial recognition works and how we will use it.

How Does Facial Recognition Work?

For this article we won’t be diving into Convolution Neural Networks, Harris Corners or other technical means of the mechanical inter-workings of these systems. Instead, we want to focus on how these systems determine who is who.

In facial recognition software, the machine learning models are first trained on hundreds of thousands or millions of images that have been painstakingly annotated by hand of key facial features within a picture of someone’s face, like the image below.

source: quora.com

The points on the face above calculated by a machine learning model that “looks” at an image and does it’s best to figure out where certain facial features are located. Using this information, facial recognition systems can calculate the geometry of your face, or your facial fingerprint, if you will. Then, this fingerprint is compared against a database of fingerprints for similarity in facial features. What sets facial recognition systems apart is the number of features they use to measure your face as more features, typically mean higher accuracy.

For our system, what we’ll do is take a few pictures first to build a more accurate digital fingerprint, that way we don’t have to keep our head in the exact same spot every time we want to use it.

The Setup

As I mentioned earlier, we’ll need a T3 API Key to create our facial recognition app. T3 is an API service that I’ve created that offers free machine learning APIs to developers. For this, we’ll use the /facekp endpoint to get the key points from a face.

Home Screen for T3 APIs

If you don’t have an API key and are subscribed to the /facekp endpoint, check out this quick tutorial on how to get started with T3 APIs.

Creating Your Fingerprint

In this section we will be building the data models that will store our digital fingerprints and finally making our own!

In the repository I linked earlier there is a python file named fingerprintr.py that we will use to build our digital fingerprint. In order to run this program, open a terminal in the root directory of the repository and run the following command.

$ python3 tools/fingerprintr.py

This program will use our webcam to take a 16 pictures of us. With those 16 pictures we will use the /facekp endpoint to get the key points for every image and then average them together. The reason why we want to average the key points is to help our generalize our finger print — this way we do not have to line our face up perfectly every time, instead it should just work.