ESP32 Cam Face Recognition

ESP32 Cam Face Recognition cover

If you search on Google "esp32 cam face recognition arduino", you won't find a single result that shows you how to create a standalone sketch that does it (in ~191.000 reported from Google).


Not. A. Single. Result.


The few relevant ones that appear (even the one ranked #1) only tells you "Load the default WebServer example and it does face recognition".

But what if you don't want the default WebServer? How do you create your very own project that does face recognition without the web GUI and all the other stuff?

I've spent hours to bring this feature into my EloquentEsp32Cam Arduino library and now I'll show you how easy it can be.

Requirements

This project only works with ESP32S3 boards. If you have the cheap AiThinker ESP32 cam, it won't work. Non-S3 chips cannot handle the heavy computations required to perform face recognition.

You will also need to install EloquentEsp32Cam >= 2.3.8 from the Arduino Library Manager.

Face recognition quickstart

The following sketch is a standalone project that allows you to

  1. enroll new faces for later recognition
  2. recognize the current detected face

It has ~150 lines of code, but there are many blank lines and comments, so don't be scared. I'll first detail the most important parts of the code, then show the complete sketch at the end of the post.

Includes

To perform face recognition (tell which person a given face belongs to), you first need to do face detection (tell if and where in the image there's a face). So you need to import the files for these 2 tasks.

#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/face/detection.h>
#include <eloquent_esp32cam/face/recognition.h>

// create short names for namespaced global variables
using eloq::camera;
using eloq::face::detection;
using eloq::face::recognition;
1 2 3 4 5 6 7 8

Setup

Now you have to configure the camera, the face detector and the face recognizer.

If you read other posts from this blog, the following code will look familiar. If this is your first time here, I suggest you read the ESP32 Camera Quickstart tutorial to get familiar with the style of the EloquentEsp32Cam library.

// !!!!REPLACE WITH YOUR OWN MODEL!!!!
camera.pinout.freenove_s3(); // e.g. xiao(), lilygo_tcamera_s3(), ...
camera.brownout.disable();
// face recognition only works at 240x240
camera.resolution.face();
camera.quality.high();

// face recognition only works with accurate detection
detection.accurate();
detection.confidence(0.7);

// face recognition confidence
recognition.confidence(0.85);

// init camera
while (!camera.begin().isOk())
    Serial.println(camera.exception.toString());

// init recognizer
while (!recognition.begin().isOk())
    Serial.println(recognition.exception.toString());
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Enroll new face

To enroll a new face you need one line of code.

// to enroll a face, there must be a face in the frame!
if (!recognition.detect().isOk())
    return;

if (recognition.enroll(name).isOk())
    Serial.println("Success!");
else
    Serial.println(recognition.exception.toString());
1 2 3 4 5 6 7 8

Recognize face

Guess how many lines of code you need to recognize a face...

// to recognize a face, there must be a face in the frame!
if (!recognition.detect().isOk())
    return;

if (!recognition.recognize().isOk()) {
    Serial.println(recognition.exception.toString());
    return;
}

// you have access to the recognition result
// via recognition.match
Serial.print("Recognized face as ");
Serial.print(recognition.match.name.c_str());
Serial.print(" with confidence ");
Serial.print(recognition.match.similarity);
Serial.print(" (");
Serial.print(recognition.benchmark.millis());
Serial.println("ms)");
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Perform action on face recognition

Let's say you're making a smart door lock project and you want to unlock the door when you are recognized. How easy is it?

void loop() {
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    if (!recognition.detect().isOk())
        return;

    if (!recognition.recognize().isOk()) {
        Serial.println(recognition.<em>exception</em>.toString());
        return;
     }
<br>    if (recognition.match.name != "simone") {
        Serial.println("You are not authorized");
        return;
    }

    // open lock
    digitalWrite(RELAY_PIN, HIGH);
}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Complete code

See source

Filename: Face_Recognition.ino

/**
 * ESP32S3 Face Recognition
 * (not detection!)
 *
 * Enroll a couple of faces 2-3 times each,
 * then watch the ESP32 camera recognize between the two!
 *
 * Only works on ESP32 S3 chip.
 */
#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/face/detection.h>
#include <eloquent_esp32cam/face/recognition.h>


using eloq::camera;
using eloq::face::detection;
using eloq::face::recognition;


String prompt(String message);


/**
 *
 */
void setup() {
    delay(4000);
    Serial.begin(115200);
    Serial.println("Begin");

    // !!!!REPLACE WITH YOUR OWN MODEL!!!!
    camera.pinout.freenove_s3(); // e.g. xiao(), lilygo_tcamera_s3(), ...
    camera.brownout.disable();
    // face recognition only works at 240x240
    camera.resolution.face();
    camera.quality.high();

    // face recognition only works with accurate detection
    detection.accurate();
    detection.confidence(0.7);

    // face recognition confidence
    recognition.confidence(0.85);

    // init camera
    while (!camera.begin().isOk())
        Serial.println(camera.exception.toString());

    // init recognizer
    while (!recognition.begin().isOk())
        Serial.println(recognition.exception.toString());

    Serial.println("Camera OK");
    Serial.println("Face recognizer OK");

    // delete stored data, if user confirms
    if (prompt("Do you want to delete all existing faces? [yes|no]").startsWith("y")) {
        Serial.println("Deleting all existing faces...");
        recognition.deleteAll();
    }

    // dump stored faces, if user confirms
    if (prompt("Do you want to dump existing faces? [yes|no]").startsWith("y")) {
        recognition.dump();
    }

    Serial.println("Awaiting for face...");
}


/**
 *
 */
void loop() {
    // capture picture
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    // run face detection (not recognition!)
    if (!recognition.detect().isOk())
        return;

    // if face is found, ask user to enroll or recognize
    String answer = prompt("Do you want to enroll or recognize? [e|r]");

    if (answer.startsWith("e"))
        enroll();
    else if (answer.startsWith("r"))
        recognize();

    Serial.println("Awaiting for face...");
}


/**
 * Ask user for input
 */
String prompt(String message) {
    String answer;

    do {
        Serial.print(message);

        while (!Serial.available())
            delay(1);

        answer = Serial.readStringUntil('\n');
        answer.trim();
    } while (!answer);

    Serial.print(" ");
    Serial.println(answer);
    return answer;
}


/**
 * Enroll new person
 */
void enroll() {
    String name = prompt("Enter person name:");

    if (recognition.enroll(name).isOk())
        Serial.println("Success!");
    else
        Serial.println(recognition.exception.toString());
}


/**
 * Recognize current face
 */
void recognize() {
    if (!recognition.recognize().isOk()) {
        Serial.println(recognition.exception.toString());
        return;
    }

    Serial.print("Recognized face as ");
    Serial.print(recognition.match.name.c_str());
    Serial.print(" with confidence ");
    Serial.print(recognition.match.similarity);
    Serial.print(" (");
    Serial.print(recognition.benchmark.millis());
    Serial.println("ms)");
}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148

Want more contents?

I put hours of work into the code and posts I write. As far as I love writing content to help people upgrade their skills with Arduino programming, I need to pay my bills.

If you want to make sure I can continue creating new awesome content like this, consider helping me by buying my "Mastering the ESP32 Camera" ebook.

Become an ESP32-CAM EXPERT

Subscribe to my newsletter

Join 1007 businesses and hobbysts skyrocketing their Arduino + ESP32 skills twice a month