Gesture-controlled small car

I bought a remote-controlled car for my son last Christmas and modified it to control by gesture using Raspberry Pi. This is the note for it.

Photo by Franco Antonio Giovanella on Unsplash

Background

I bought the following remote-controlled car for my son’s Christmas gift which I can build from scratch by myself.

FREENOVE Three-Wheeled Smart Car Kit for Raspberry Pi 5 4 B 3 B+ B A+, Robot Project, App Control…

It comes with the software shared on GitHub and includes the remote controller displayed on a monitor as GUI.

GitHub - Freenove/Freenove_Three-wheeled_Smart_Car_Kit_for_Raspberry_Pi: Apply to FNK0021

I would say it’s quite easy to build and reasonable price. Moreover, it was very fun, which is most important. However, it seems too early for my son to drive it through the controller.

So I started to modify it for him to play with the car in his way, which is a gesture.

What I did

I explored some well-known computer vision models first, which are OpenCV, Mediapip, and Yolo.

Home

MediaPipe | Google for Developers

YOLOv8: A New State-of-the-Art Computer Vision Model

Then I made sample codes for one of each. I used pre-trained models.

GitHub - atsss/opencv_on_RPi

GitHub - atsss/mediapipe_on_RPi

GitHub - atsss/yolo_on_RPi

Finally, I decided to go with Mediapipe because the gesture detection model is already there and the performance was quite smooth.

Then, before jumping into controlling the car by gestures, I made simple and small code snippets first, which are for controlling an LED by gestures.

The following gif is a demo of it. When I open my hand, the LED turns on. When I close my hand, the LED turns off. Otherwise, nothing happens. I put my prototype codes below as well.

Three_wheeled_Car/prototypes at a8d4cc7b6c8c97a634ffb9048eedd11ef1af1dde · atsss/Three_wheeled_Car

At last, I merge the prototype codes and the shared codes.

Three_wheeled_Car/Client/gesture_control.py at a8d4cc7b6c8c97a634ffb9048eedd11ef1af1dde ·…

Three_wheeled_Car/Client/wheel_control.py at a8d4cc7b6c8c97a634ffb9048eedd11ef1af1dde ·…

The following gif is a demo of it. Currently, I have only Stop , Move forward , and Move backward . A thumbs-up means to move forward. A thumbs-down means to move backward. An open hand means to stop. But it became more interactive and my son can also control it by himself.

That’s it!

Appendix

CV models

If I had to use OpenCV for object detection, I felt the Rasberry Pi processing was a bit slow and thought I would need to tune the settings. However, the Yolo v8 seemed quite smooth. I need to investigate the exact pros and cons more but Yolo v8 would be my first chice

I2C

python-smbus , which is Linux I2C modules for Python, is no longer. Instead, python3-smbus is available.

Python-smbus has no installation canidate. Package is not availabe even after updates - Raspberry…

smbus2 is compatible with python3-smbus but the shared code needs to be modified

piwheels

Check I2C status with i2c-tool

I2C Toolsの簡単な使い方について - バージョン管理された人