ESP32 cam Person Detection

ESP32 cam Person Detection cover

Person detection running on our cheap ESP32 cam has become a common task nowadays.

Only a few years ago this would have been simply impossible because of lack of both hardware (Arduino boards used to feature a mediocre 16 kb RAM) and software (neural networks support for embedded systems was simply non-existent).

As of today, lots of things has changed and person detection has even become one the get started projects for TinyML. Nonetheless, many tutorials on the subject are a bit convoluted and pretty hard to follow, let alone modify and integrate into your very own project.

But it doesn't have to be that way.

I'm going to show you how you can add person detection on your very own ESP32 camera project in 5 minutes!

Software requirements

You will need to install 3 libraries:

  1. tlfm_esp32: the TensorFlow runtime for the ESP32
  2. EloquentTinyML: a wrapper around TensorFlow
  3. EloquentEsp32Cam: to interact with camera easily

You can install them all from the Arduino Library Manager.

Be sure you have the latest version of each!

Arduino sketch

This part is easy, you don't have to write much code nor configure many options.

See source

Filename: PersonDetectionExample.ino

/**
 * Run person detection on ESP32 camera
 *  - Requires tflm_esp32 library
 *  - Requires EloquentEsp32Cam library
 *
 * Detections takes about 4-5 seconds per frame
 */
#include <Arduino.h>
#include <tflm_esp32.h>
#include <eloquent_tinyml.h>
#include <eloquent_tinyml/zoo/person_detection.h>
#include <eloquent_esp32cam.h>

using eloq::camera;
using eloq::tinyml::zoo::personDetection;


void setup() {
    delay(3000);
    Serial.begin(115200);
    Serial.println("__PERSON DETECTION__");

    // camera settings
    // replace with your own model!
    camera.pinout.freenove_s3();
    camera.brownout.disable();
    // only works on 96x96 (yolo) grayscale images
    camera.resolution.yolo();
    camera.pixformat.gray();

    // init camera
    while (!camera.begin().isOk())
        Serial.println(camera.exception.toString());

    // init tf model
    while (!personDetection.begin().isOk())
        Serial.println(personDetection.exception.toString());

    Serial.println("Camera OK");
    Serial.println("Point the camera to yourself");
}

void loop() {
    Serial.println("Loop");

    // capture picture
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    // run person detection
    if (!personDetection.run(camera).isOk()) {
        Serial.println(personDetection.exception.toString());
        return;
    }

    // a person has been detected!
    if (personDetection) {
        Serial.print("Person detected in ");
        Serial.print(personDetection.tf.benchmark.millis());
        Serial.println("ms");
    }
}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

The sketch can be broken into the following parts.

Configure

// configure camera
camera.pinout.freenove_s3();
camera.brownout.disable();
// only works on 96x96 (yolo) grayscale images
camera.resolution.yolo();
camera.pixformat.gray();

// init camera
while (!camera.begin().isOk())
    Serial.println(camera.exception.toString());

// init tf model
while (!personDetection.begin().isOk())
    Serial.println(personDetection.exception.toString());
1 2 3 4 5 6 7 8 9 10 11 12 13 14

Run

// capture picture
if (!camera.capture().isOk()) {
    Serial.println(camera.exception.toString());
    return;
}

// run person detection
if (!personDetection.run(camera).isOk()) {
    Serial.println(personDetection.exception.toString());
    return;
}
1 2 3 4 5 6 7 8 9 10 11

Test

// test if a person is detected
if (personDetection) {
    // do whatever you want here...
}
1 2 3 4

Customize

Customizing the sketch is easy. You can add your own logic to be performed when a person is detected inside the if block above.

A few use cases you can implement:

Feel free to let me know what you implemented with the form below.

If you're stuck, don't hesitate to ask for help.

Appendix. Camera config

To run the above code, set these configurations in your Tools menu (left is for ESP32, right form ESP32S3).

image.jpg 155.7 KB

Become an ESP32-CAM EXPERT

Subscribe to my newsletter

Join 1007 businesses and hobbysts skyrocketing their Arduino + ESP32 skills twice a month