Level up your TinyML skills

How to get started with MicroPython Machine Learning

Convert scikit-learn Machine Learning models to space-efficient, low complexity MicroPython code

MicroPython Machine Learning

Python is my personal favourite programming language.

It is clean, concise and powerful. Being able to run Python (either MicroPython or CircuitPython) on microcontrollers is such a joyful experience.

Starting from today, we have a new feature to joy for: Machine Learning.

Most TinyML work and projects so far have been developed in C/C++, which is a natual choice since it is math-heavy and C/C++ is the fastest language in terms of execution time.

Nonetheless, not all TinyML projects require microseconds execution time and many tasks can easily handle a bit of slowdown.

Thanks to the everywhereml Python package we can easily port a growing list of classifiers to MicroPython with a single line of code.

Let's see it in action.

MicroPython Machine Learning project: MNIST digits classification

This project is an exact clone of the Attiny Machine Learning project: MNIST digits classification: we train a Random Forest to classify 8x8 images of handwritten digits.

Refer to the Attiny article for more information on the task.

Start by installing the everywhereml library.

pip install everywhereml>=0.2.2

This is the Python code to train the model.

from everywhereml.sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_digits
from sklearn.model_selection import train_test_split

X, y = load_digits(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

clf = RandomForestClassifier(n_estimators=7, max_leaf_nodes=20)
clf.fit(X_train, y_train)

print('Score: %.2f' % clf.score(X_test, y_test))
>>> Score: 0.89

Now we can port the classifier to MicroPython with a single line of code.

print(clf.to_micropython_file('random_forest.py'))

>>>

try:
    from time import ticks_us, ticks_diff
except ImportError:
    from time import time_ns

    def ticks_us(): return int(time_ns() * 1000)
    def ticks_diff(a, b): return a - b

class RandomForestClassifier:
    """
    # RandomForestClassifier(base_estimator=DecisionTreeClassifier(), bootstrap=True, ccp_alpha=0.0, class_name=RandomForestClassifier, class_weight=None, criterion=gini, estimator_params=('criterion', 'max_depth', 'min_samples_split', 'min_samples_leaf', 'min_weight_fraction_leaf', 'max_features', 'max_leaf_nodes', 'min_impurity_decrease', 'random_state', 'ccp_alpha'), max_depth=None, max_features=auto, max_leaf_nodes=20, max_samples=None, min_impurity_decrease=0.0, min_samples_leaf=1, min_samples_split=2, min_weight_fraction_leaf=0.0, n_estimators=7, n_jobs=None, num_outputs=10, oob_score=False, package_name=everywhereml.sklearn.ensemble, random_state=None, template_folder=everywhereml/sklearn/ensemble, verbose=0, warm_start=False)
    """

    def __init__(self):
        """
        Constructor
        """
        self.latency = 0
        self.predicted_value = -1

        self.votes = [0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000]

    def predict(self, x):
        """
        Predict output from input vector
        """
        self.predicted_value = -1
        started_at = ticks_us()

        self.votes = [0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000, 0.00000000000]

        idx, score = self.tree0(x)
        self.votes[idx] += score

        idx, score = self.tree1(x)
        self.votes[idx] += score

        idx, score = self.tree2(x)
        self.votes[idx] += score

        idx, score = self.tree3(x)
        self.votes[idx] += score

        idx, score = self.tree4(x)
        self.votes[idx] += score

        idx, score = self.tree5(x)
        self.votes[idx] += score

        idx, score = self.tree6(x)
        self.votes[idx] += score

        # get argmax of votes
        max_vote = max(self.votes)
        self.predicted_value = next(i for i, v in enumerate(self.votes) if v == max_vote)

        self.latency = ticks_diff(ticks_us(), started_at)
        return self.predicted_value

    def latencyInMicros(self):
        """
        Get latency in micros
        """
        return self.latency

    def latencyInMillis(self):
        """
        Get latency in millis
        """
        return self.latency // 1000

    def tree0(self, x):
        """
        Random forest's tree #0
        """
        if x[28] <= 0.5:
            if x[52] <= 2.5:
                return 6, 101.0
            else:
                if x[35] <= 3.0:
                    return 0, 131.0
                else:
                    return 6, 101.0
        else:
            if x[60] <= 2.5:
                return 7, 135.0
            else:
                if x[21] <= 5.5:
                    if x[34] <= 12.5:

... redacted for brevity ...

Add the random_forest.py file to your own MicroPython project.

To test the model accuracy, we'll use some random samples from the MNIST dataset.

FILENAME: mnist.py

mnist = [ [0, 0, 5, 13, 9, 1, 0, 0, 0, 0, 13, 15, 10, 15, 5, 0, 0, 3, 15, 2, 0, 11, 8, 0, 0, 4, 12, 0, 0, 8, 8, 0, 0, 5, 8, 0, 0, 9, 8, 0, 0, 4, 11, 0, 1, 12, 7, 0, 0, 2, 14, 5, 10, 12, 0, 0, 0, 0, 6, 13, 10, 0, 0, 0], [0, 0, 0, 12, 13, 5, 0, 0, 0, 0, 0, 11, 16, 9, 0, 0, 0, 0, 3, 15, 16, 6, 0, 0, 0, 7, 15, 16, 16, 2, 0, 0, 0, 0, 1, 16, 16, 3, 0, 0, 0, 0, 1, 16, 16, 6, 0, 0, 0, 0, 1, 16, 16, 6, 0, 0, 0, 0, 0, 11, 16, 10, 0, 0], [0, 0, 0, 4, 15, 12, 0, 0, 0, 0, 3, 16, 15, 14, 0, 0, 0, 0, 8, 13, 8, 16, 0, 0, 0, 0, 1, 6, 15, 11, 0, 0, 0, 1, 8, 13, 15, 1, 0, 0, 0, 9, 16, 16, 5, 0, 0, 0, 0, 3, 13, 16, 16, 11, 5, 0, 0, 0, 0, 3, 11, 16, 9, 0], [0, 0, 7, 15, 13, 1, 0, 0, 0, 8, 13, 6, 15, 4, 0, 0, 0, 2, 1, 13, 13, 0, 0, 0, 0, 0, 2, 15, 11, 1, 0, 0, 0, 0, 0, 1, 12, 12, 1, 0, 0, 0, 0, 0, 1, 10, 8, 0, 0, 0, 8, 4, 5, 14, 9, 0, 0, 0, 7, 13, 13, 9, 0, 0], [0, 0, 0, 1, 11, 0, 0, 0, 0, 0, 0, 7, 8, 0, 0, 0, 0, 0, 1, 13, 6, 2, 2, 0, 0, 0, 7, 15, 0, 9, 8, 0, 0, 5, 16, 10, 0, 16, 6, 0, 0, 4, 15, 16, 13, 16, 1, 0, 0, 0, 0, 3, 15, 10, 0, 0, 0, 0, 0, 2, 16, 4, 0, 0], [0, 0, 12, 10, 0, 0, 0, 0, 0, 0, 14, 16, 16, 14, 0, 0, 0, 0, 13, 16, 15, 10, 1, 0, 0, 0, 11, 16, 16, 7, 0, 0, 0, 0, 0, 4, 7, 16, 7, 0, 0, 0, 0, 0, 4, 16, 9, 0, 0, 0, 5, 4, 12, 16, 4, 0, 0, 0, 9, 16, 16, 10, 0, 0], [0, 0, 0, 12, 13, 0, 0, 0, 0, 0, 5, 16, 8, 0, 0, 0, 0, 0, 13, 16, 3, 0, 0, 0, 0, 0, 14, 13, 0, 0, 0, 0, 0, 0, 15, 12, 7, 2, 0, 0, 0, 0, 13, 16, 13, 16, 3, 0, 0, 0, 7, 16, 11, 15, 8, 0, 0, 0, 1, 9, 15, 11, 3, 0], [0, 0, 7, 8, 13, 16, 15, 1, 0, 0, 7, 7, 4, 11, 12, 0, 0, 0, 0, 0, 8, 13, 1, 0, 0, 4, 8, 8, 15, 15, 6, 0, 0, 2, 11, 15, 15, 4, 0, 0, 0, 0, 0, 16, 5, 0, 0, 0, 0, 0, 9, 15, 1, 0, 0, 0, 0, 0, 13, 5, 0, 0, 0, 0], [0, 0, 9, 14, 8, 1, 0, 0, 0, 0, 12, 14, 14, 12, 0, 0, 0, 0, 9, 10, 0, 15, 4, 0, 0, 0, 3, 16, 12, 14, 2, 0, 0, 0, 4, 16, 16, 2, 0, 0, 0, 3, 16, 8, 10, 13, 2, 0, 0, 1, 15, 1, 3, 16, 8, 0, 0, 0, 11, 16, 15, 11, 1, 0], [0, 0, 11, 12, 0, 0, 0, 0, 0, 2, 16, 16, 16, 13, 0, 0, 0, 3, 16, 12, 10, 14, 0, 0, 0, 1, 16, 1, 12, 15, 0, 0, 0, 0, 13, 16, 9, 15, 2, 0, 0, 0, 0, 3, 0, 9, 11, 0, 0, 0, 0, 0, 9, 15, 4, 0, 0, 0, 9, 12, 13, 3, 0, 0] ]

And this is the main.py file.

FILENAME: main.py

from time import sleep from random_forest import RandomForestClassifier from mnist import mnist clf = RandomForestClassifier() while True: for idx, x in enumerate(mnist): print('exptected %d, got %d' % (idx, clf.predict(x))) sleep(1)

Pretty easy, right?

You only have to call clf.predict(x) to get the predicted value back.

Random Forest is only the first of a few classifiers that will be implemented in the next weeks, but it's one of the most performant and you can go a long way with it already!

Get monthly updates

Do not miss the next posts on TinyML and Esp32 camera. No spam, I promise

We use Mailchimp as our marketing platform. By submitting this form, you acknowledge that the information you provided will be transferred to Mailchimp for processing in accordance with their terms of use. We will use your email to send you updates relevant to this website.

Having troubles? Ask a question

Need a one to one call? Book now!

© Copyright 2022 Eloquent Arduino. All Rights Reserved.