ESP32 cam autonomous car

ESP32 cam autonomous car cover

In my article ESP32 cam object detection, I showed you how to train and deploy an Edge Impulse object detection model to your ESP32 camera.

In this tutorial, I'll take a step further: we'll use the object detection results to drive an autonomous car!

Let's start with the end result.

Pretty awesome, right?

Do you want to replicate it right now?

If you already have a toy car, it will take ~20 minutes to train the model and program the board.

Software requirements

EloquentEsp32Cam >= 2.6

Arduino IDE Tools configuration for ESP32S3

!!! Install ESP32 core version 2.x, version 3.x won't work !!!

Board ESP32S3 Dev Module
Upload Speed 921600
USB Mode Hardware CDC and JTAG
USB CDC On Boot Disabled
USB Firmware MSC On Boot Disabled
USB DFU On Boot Disabled
Upload Mode UART0 / Hardware CDC
CPU Frequency 240MHz (WiFi)
Flash Mode QIO 80MHz
Flash Size 4MB (32Mb)
Partition Scheme Huge APP (3MB No OTA/1MB SPIFFS)
Core Debug Level Info
PSRAM OPI PSRAM
Arduino Runs On Core 1
Events Run On Core 1
Erase All Flash Before Sketch Upload Disabled
JTAG Adapter Disabled

Get a car

This tutorial is not an instructables. I will not teach you how to build an Arduino-controlled car because there are a lot of tutorials on the web dedicated to this topic. I'll link some of the ones I followed to build mine:

*no affiliate links (sadly)

For the building process, search on YouTube if you need help. You only have to connect the motors and power supply to the L298N module, then the L298N module to the ESP32. Nothing else.

Test the car

Before moving on, be sure your car is working.

Load the following sketch, set the correct motor pin numbers and test that the car behaves like expected when you send commands over Serial.

See source

Filename: Car_Test.ino

/**
 * Test car wiring
 */
#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/car/two_wheels_car.h>

using eloq::camera;
using eloq::car::Motor;
using eloq::car::TwoWheelsCar;

// replace with your motor pins
Motor left(39, 40);
Motor right(42, 41);
TwoWheelsCar testCar(left, right);


/**
 *
 */
void setup() {
    delay(3000);
    Serial.begin(115200);
    Serial.println("___CAR TEST___");

    // how many millis motors will run
    testCar.defaultDuration(200);
    testCar.stop();

    Serial.println("Enter one of f (forward), b (backward), l (left), r (right)");
}


/**
 *
 */
void loop() {
    if (!Serial.available())
        return;

    String cmd = Serial.readStringUntil('\n');

    if (cmd.startsWith("f")) testCar.forward();
    else if (cmd.startsWith("b")) testCar.backward();
    else if (cmd.startsWith("l")) testCar.left();
    else if (cmd.startsWith("r")) testCar.right();
}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46

Train an object detection model

Please refer to the aforementioned ESP32 cam object detection for detailed instructions.

Program the car

Now that you have all the pieces in place, it's time to program the ESP32 camera to drive the car based on the object detection results.

The following sketch assumes that:

  1. the camera is placed vertically
  2. left and right motor pins are set correctly

See source

Filename: Autonomous_Car.ino

/**
 * Autonomous car driven by Edge Impulse FOMO
 * From: https://eloquentarduino.com/esp32-cam-autonomous-car
 *
 * Tested on ESP32S3 camera
 */
#include <your_edge_impulse_inferencing.h>
#include <eloquent_esp32cam.h>
#include <eloquent_esp32cam/car.h>

using eloq::camera;
using eloq::car::Motor;
using eloq::car::Car;

/**
 * Replace with your motor pins
 */
Motor left(39, 40);
Motor right(42, 41);
Car fomoCar(left, right);



/**
 *
 */
void setup() {
    delay(3000);
    Serial.begin(115200);
    Serial.println("___AUTONOMOUS CAR___");

    // replace with your board
    camera.pinout.freenove_s3();
    camera.brownout.disable();
    camera.resolution.yolo();
    camera.pixformat.rgb565();

    // how many millis motors will run
    // to follow given object
    fomoCar.defaultDuration(100);
    fomoCar.stop();

    // if you mounted the camera "backward"
    // (see video), you have to reverse the motors
    // left.reverse();
    // right.reverse();

    // init camera
    while (!camera.begin().isOk())
        Serial.println(camera.exception.toString());

    Serial.println("Camera OK");
    Serial.println("Put object in front of camera");
}

/**
 *
 */
void loop() {
    // capture picture
    if (!camera.capture().isOk()) {
        Serial.println(camera.exception.toString());
        return;
    }

    // run FOMO
    if (!fomo.run().isOk()) {
        Serial.println(fomo.exception.toString());
        return;
    }

    // let the car follow the object
    fomoCar.follow(fomo);
}
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74

Wait...

Is it that simple? How???

Well, that's the power of the EloquentEsp32cam library.

Become an ESP32-CAM EXPERT

Subscribe to my newsletter

Join 1177 businesses and hobbysts skyrocketing their Arduino + ESP32 skills twice a month