How to set up OpenCV env for Raspberry Pi

It took more time to run OpenCV on Raspberry Pi than I expected. I share my experience here.

Photo by Paul Skorupskas on Unsplash

First of all

The following specification is my environment.

• Raspberry Pi CM4
• Raspberry Pi OS — Bullseye/Bookworm
• OpenCV 4.5.1
• Picamera2
• Camera sensor: IMX219
• boot/config.txt

Background

I made a mini car controlled for my son by Raspberry Pi last Christmas. I can drive the car using my laptop terminal and send commands through TCP. It worked fine but I wanted it to be more interactive with my son. So I decided to use a camera and drive it by the camera input.
I started the project at the beginning of this year and finally made it work this week. I thought it was a piece of cake and it would take a few days because OpenCV is very popular library and I had used it in my job before. However, it took almost 10 days to work it on Raspberry Pi. There is a lot of information about it but I couldn’t tell which is correct. So I’ve decided to share my findings.

What I did

Initially, I tried to install OpenCV through piwheel, which is the place where libraries compiled specially for Raspberry Pi are. In most cases, you don’t have any problems to install packages from here.

piwheels

This time, I followed the opencv-python instruction about installing dependencies and opencv-python itself.

opencv-python

However, this time, the sample code didn’t work.

import cv2
import numpy as np


def camera():
    cap = cv2.VideoCapture(0)
    isOpened = cap.isOpened()
    if not isOpened:
        return
    while True:
        result, frame = cap.read()
        if not result:
            return
        
        # To display video
        cv2.imshow('camera', frame)
        
        # To finish video
        key = cv2.waitKey(1)
        if (key == 13) or (key == 27):
            break

    cap.release()
    cv2.destroyAllWindows()

camera()

What happened to me was that isOpened was True but I got False as the result from cap.read() . I couldn’t tell what was the root cause because there weren’t logs for it. Then I decided to install it following the instructions provided by OpenCV.

OpenCV: Install OpenCV-Python in Ubuntu

In the docs, they recommend to install it through apt command. Through apt , We don’t have to consider the dependencies by ourselves unlike piweel because the apt manages it. I quickly researched and found the opencv-python provided by piweel tries to follow the latest version on their environment and, otherwise, python3-opencv provided by apt doesn't care about the latest version but cares about making it work in Ubuntu. So the apt makes more sense in my case.

Then I ran the same sample code as before at that moment but it didn’t work either. However, I got some warning logs this time, which was very helpful in understanding the problem. The logs are below.

[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1824) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Failed to allocate required memory.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (914) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (501) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

I googled the warnings and ran into a GitHub comment discussing them.

Use camera (Waveshare) under Bullseye (64) with OpenCV (failed camera access with OpenCV) · Issue…

Long story short, the interface of libcamera, which is provided by Raspberry Pi OS to operate a camera, is not compatible with OpenCV and Picamera can bridge both. Then I installed picamera2 following their document.

https://datasheets.raspberrypi.com/camera/picamera2-manual.pdf

Then I changed the sample code for Picamera2 like below.

import cv2
from picamera2 import Picamera2

picam2 = Picamera2()
picam2.preview_configuration.main.size = (800,800)
picam2.preview_configuration.main.format = "RGB888"
picam2.preview_configuration.align()
picam2.configure("preview")
picam2.start()

while True:
    im= picam2.capture_array()
    cv2.imshow("Camera", im)
    if cv2.waitKey(1)==ord('q'):
        break

cv2.destroyAllWindows()

Finally, It worked well. It is quite slow so I need to improve the performance. That’s my next step for now. I also created the installer and test code in my repository.

GitHub - atsss/opencv_on_RPi

I did a lot of things to reach the result like changing boot/config.txt and adding dependencies and most of them failed. I hope this article helps someone somehow.

That’s it!