Motion detection is the task of detecting when the scene in the ESP32 camera field of view changes all of a sudden.

This change may be caused by a lot of factors (an object moving, the camera itself moving, a light change...) and you may be interested in get notified when it happens.

For example, you may point your ESP32 camera to the door of your room and take a picture when the door opens.

In a scenario like this, you are not interested in localizing the motion (knowing where it happened in the frame), only that it happened.

Motion detection with PIR (infrared sensor)

Most tutorials on the web focus on human motion detection, so they equip the ESP32 with an external infrared sensor (a.k.a PIR, the one you find in home alarm systems) and take a photo when the PIR detects something.

If this setup works fine for you, go with it. It's easy, fast, low power and pretty accurate.

But the PIR approach has a few drawbacks:

1. you can only detect living beings: since it is based on infrared sensing, it can only detect when something "hot" is in its field of view (humans and animals, basically). If you want to detect a car passing, it won't work
2. it has a limited range: PIR sensors reach at most 10-15 meters. If you need to detect people walking on the street in front of your house at 30 meters, it won't work
3. it needs a clear line-of-sight: to detect infrared light, the PIR sensor needs no obstacles in-between itself and the moving object. If you put it behind a window to detect people outside your home, it won't work
4. it triggers even when no motion happened: the PIR tecnique is actually a proxy for motion detection. The PIR sensor doesn't actually detects motion: it detects the presence of warm objects. For example, if a person comes into a room and lies down on the sofa, the PIR sensor will trigger for as long as the person doesn't leave the room

Motion detection without PIR (image based)

On the other hand, image motion detection can fulfill all the above cases because it performs motion detection on the camera frames, comparing each one with the previous looking for differences.

If a large portion of the image changed, it triggers.

Video motion detection has its drawbacks, nonetheless:

1. power-hungry: comparing each frame with the previous frame means the camera must be always on. While with the PIR sensor you can put the camera to sleep, now you have to continuously check each frame
2. insensitive to slow changes: to avoid false triggers, you will set a lower threshold on the image portion that need to change to detect motion (e.g. 10% of the frame). If something is moving slowly in your field of view such that it changes less than 10% of the frame, the algorithm will not pick it up.

Take some time to review the pros and cons of video motion detection now that you have a little more details.

Hardware requirements

This project works with ESP32S3 and non-S3 boards.

Software requirements

EloquentEsp32Cam >= 2.2

Motion detection configuration

In the sketch below, you can configure a few parameters for the detector algorithm:

1. skip(n): don't run detection on the first few frames. Many times, the camera needs a little time to settle and this prevents false positives at startup
2. stride(n): the image is divided into a grid where each cell size equals the stride. The motion is analyzed only at the corners of each cell. The larger the stride, the faster the execution time, but the coarser the detection. Keep in mind that motion detection already happens on a 1/8th version of the original image! So, if you start with a VGA image (640x480), motion detection happens on a 80x60 grid. If you set stride(2), motion detection happens instead on a 40x30 grid. You can leave this value to 1 if you're not sure, it is still pretty fast.
3. threshold(n): if the value of the pixels at each corner changed from the previous frame by more than this value, that point is marked as a moving point. The higher the value, the less sensitive to noise and small changes the detection is.
4. ratio(percent):  this is the ratio of moving points / total points above which motion is triggered. The higher the value (from 0 to 1), the more pixels need to be marked as moving to return true at detection time
5. rate limiter(n): when an object is moving in front of the camera, it will trigger the detection for all the time it is passing by. Most often, you only want to get a single trigger for each event (for example to turn on an actuator or send a notification) and not get spammed with consecutive activations. You can configure how many seconds should elapse from one trigger to the next.

Motion detection quickstart

See source

/**
 * Motion detection
 * Detect when the frame changes by a reasonable amount
 *
 * BE SURE TO SET "TOOLS > CORE DEBUG LEVEL = DEBUG"
 * to turn on debug messages
 */
#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/motion/detection.h>

using eloq::camera;
using eloq::motion::detection;


/**
 *
 */
void setup() {
    delay(3000);
    Serial.begin(115200);
    Serial.println("___MOTION DETECTION___");

    // camera settings
    // replace with your own model!
    camera.pinout.freenove_s3();
    camera.brownout.disable();
    camera.resolution.vga();
    camera.quality.high();

    // configure motion detection
    // the higher the stride, the faster the detection
    // the higher the stride, the lesser granularity
    detection.stride(1);
    // the higher the threshold, the lesser sensitivity
    // (at pixel level)
    detection.threshold(5);
    // the higher the threshold, the lesser sensitivity
    // (at image level, from 0 to 1)
    detection.ratio(0.2);
    // optionally, you can enable rate limiting (aka debounce)
    // motion won't trigger more often than the specified frequency
    detection.rate.atMostOnceEvery(5).seconds();

    // init camera
    while (!camera.begin().isOk())
        Serial.println(camera.exception.toString());

    Serial.println("Camera OK");
    Serial.println("Awaiting for motion...");
}

/**
 *
 */
void loop() {
    // capture picture
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    // run motion detection
    if (!detection.run().isOk()) {
        Serial.println(detection.exception.toString());
        return;
    }

    // on motion, perform action
    if (detection.triggered())
        Serial.println("Motion detected!");
}

Pretty easy, no?

This basic sketch will cover most of your use cases for smart projects, like:

• intruder alert
• vision-based triggers (very inefficient - not recommended)
• starting object detection

Switch resolution on the fly

To get a fast detection, you want to run the motion algorithm on a frame that is as small as possible (VGA or even QVGA).

But what if you want to capture a picture of what caused the motion and save it on your SD card for later inspection? In this case, you want a frame resolution as large as possible, instead.

This would be a dilemma... if the EloquentEsp32cam library didn't addressed this specific (but common) use case!

The resolution object has a function construct that allows you to change frame size on the fly, do your things (e.g. take a picture), and switch back to the original resolution automatically.

See source

/**
 * Run motion detection at low resolution.
 * On motion, capture frame at higher resolution
 * for SD storage.
 *
 * BE SURE TO SET "TOOLS > CORE DEBUG LEVEL = INFO"
 * to turn on debug messages
 */
#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/motion/detection.h>

using eloq::camera;
using eloq::motion::detection;


/**
 *
 */
void setup() {
    delay(3000);
    Serial.begin(115200);
    Serial.println("___MOTION DETECTION + SWITCH RESOLUTION___");

    // camera settings
    // replace with your own model!
    camera.pinout.freenove_s3();
    camera.brownout.disable();
    camera.resolution.vga();
    camera.quality.high();

    // see example of motion detection for config values
    detection.skip(5);
    detection.stride(1);
    detection.threshold(5);
    detection.ratio(0.2);
    detection.rate.atMostOnceEvery(5).seconds();

    // init camera
    while (!camera.begin().isOk())
        Serial.println(camera.exception.toString());

    Serial.println("Camera OK");
    Serial.println("Awaiting for motion...");
}

/**
 *
 */
void loop() {
    // capture picture
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    // run motion detection
    if (!detection.run().isOk()) {
        Serial.println(detection.exception.toString());
        return;
    }

    // on motion, perform action
    if (detection.triggered()) {
        Serial.printf(
          "Motion detected on frame of size %dx%d (%d bytes)\n",
          camera.resolution.getWidth(),
          camera.resolution.getHeight(),
          camera.getSizeInBytes()
        );

        Serial.println("Taking photo of motion at higher resolution");

        camera.resolution.at(FRAMESIZE_UXGA, []() {
          Serial.printf(
            "Switched to higher resolution: %dx%d. It took %d ms to switch\n",
            camera.resolution.getWidth(),
            camera.resolution.getHeight(),
            camera.resolution.benchmark.millis()
          );

          camera.capture();
          
          Serial.printf(
            "Frame size is now %d bytes\n", 
            camera.getSizeInBytes()
          );

          // save to SD...
        });

        Serial.println("Resolution switched back to VGA");
    }
}

On my Freenove S3 board, it takes 44 milliseconds. You have to test this time for your own board and see if it fits your project. If you think 44 ms will make you miss the source that triggered motion (fast moving object), then your only chance is to increase the resolution by default and keep the detection running "slower".

CAUTION: ESP32-cam boards from AiThinker (non-S3) may not have enough memory to capture pictures at maximum resolution (XGA, 1600x1200). In that case you can still switch to higher resolutions (e.g. SVGA), though not full resolution.

Event-Driven motion detection

In the Quickstart sketch, we saw how easy and linear it is to run motion detection; it only requires a few lines of code. Nevertheless, the loop() function is pretty lengthy now because it has to continuously check if one or more faces are present in the frame.

In this section, I'm going to show you how to move all the logic into the setup() function instead. We will exploit a style of programming called event driven (or reactive).  Event driven programming consists in registering a listener function that will run when an event of interest happens. In our case, the event of interest is motion being detected.

Why is this useful?

As I said, because it allows for leaner loop() code, where you can focus on running other tasks that need to occur in parallel to motion detection. Often, event listeners also help to isolate specific functionalities (motion detection handling) into their own routines, visually de-cluttering other tasks' code.

Here's the updated sketch.

The configuration part is exactly the same as before. The new entry is the daemon object which does 2 things:

1. accepts event listeners to be run on events of interest
2. runs the motion detection code in background

To register the callback to run when a single face is detected, we use

detection.daemon.onMotion(listener_function);

Now you could even leave the loop() empty and you still will see the Motion detected message printed on the Serial Monitor when you move your camera around.

Motion detection to MQTT

In your specific project, detecting motion may only be the first step in a larger system.

Maybe you want to log the time motion events were detected in a database, or get a notification on your phone, or an email... whatever. There are many ways to accomplish this goal. One of the most popular in the maker community is using the MQTT protocol as a mean of systems communication.

The EloquentEsp32Cam has a first party integration for MQTT.

In the following sketch, you will have to replace the test.mosquitto.org broker with your own and (if required) add proper authentication. Beside that, the sketch will work out of the box.

Software requirements

EloquentEsp32Cam >= 2.2
PubSubClient >= 2.8 

What is the payload that will be uploaded? It is a JSON object with the following structure.

{"motion": true}

Motion detection to Telegram

In the tutorial about sending text and pictures to Telegram we saw how easy this can be.

Now we will see how easy it is to integrate Telegram into motion detection by leveraging the event-driven approach.

The motion detection configuration setup is the same as in previous examples. Telegram configuration is also the same. The only addition is the definition of an actual event listener for the motion event.

detection.daemon.onMotion([]() {
  Serial.println("Motion detected");
 
  if (!telegram.to(TELEGRAM_CHAT).send(camera.frame).isOk())
    Serial.println(telegram.exception.toString());
});