Using MediaPipe on Raspberry Pi with Built-in Camera: Fixing CV2 Video Capture Issue

Introduction:

Raspberry Pi is a versatile single-board computer popular among hobbyists, educators, and professionals alike. Its compact size and affordability make it an excellent platform for various projects, including computer vision applications. However, when working with the built-in camera module on Raspberry Pi and trying to utilize OpenCV's cv2.VideoCapture for video capture, users often encounter issues. In this blog post, we'll explore how to overcome this challenge by leveraging MediaPipe, a robust library for building machine learning pipelines.

Problem Statement:

The built-in camera module on Raspberry Pi offers a convenient way to capture images and videos. However, using cv2.VideoCapture to access the camera feed directly can be problematic, especially with certain configurations or versions of OpenCV. Users may encounter errors or find that the camera feed does not display as expected.

Solution Overview:

To overcome the limitations of cv2.VideoCapture on Raspberry Pi's built-in camera, we can integrate MediaPipe into our computer vision pipeline. MediaPipe is an open-source framework developed by Google Research for building cross-platform AI pipelines. It provides ready-to-use solutions for various tasks, including face detection, pose estimation, hand tracking, and more.

Implementation:

We'll demonstrate how to use MediaPipe for face detection with the built-in camera module on Raspberry Pi. Below is a Python code snippet illustrating the implementation:

code

import cv2
import cvzone
from cvzone.FaceDetectionModule import FaceDetector
from picamera2 import Picamera2


picam2 = Picamera2()
picam2.preview_configuration.main.size = (640,480)
picam2.preview_configuration.main.format = "RGB888"
picam2.preview_configuration.align()
picam2.configure("preview")
picam2.start()
detector = FaceDetector(minDetectionCon=0.5)
list=[]
while True:
    im= picam2.capture_array()
    im, bboxs= detector.findFaces(im,draw=True)
    if bboxs:
        bbox = bboxs[0]["bbox"]
        # bboxInfo - "id","bbox","score","center"center = bboxs[0]["center"]cv2.circle(im, center, 5, (255, 0, 255), cv2.FILLED)
        x, y, w, h = bbox
        cv2.rectangle(im, (x, y), (x+w, y+h), (0, 255, 0), 2)  # Draw a rectangle around the face
        center = bboxs[0]["center"] 
        # print(x,y,w,h)
    
       
    cv2.imshow("image",im)
    key = cv2.waitKey(1)
    if key == 27:  # esc
         break
cv2.destroyAllWindows() 

video implementation:-

https://youtube.com/shorts/CiIQEQLD9Dk?si=mMg0HzSbia9FeK-Q

By leveraging MediaPipe for face detection on Raspberry Pi's built-in camera, we can overcome the limitations of cv2.VideoCapture and ensure smooth operation of computer vision applications. MediaPipe offers robust and efficient solutions for various tasks, making it an excellent choice for Raspberry Pi projects requiring real-time processing of visual data. Experiment with different MediaPipe pipelines to explore additional capabilities and enhance your Raspberry Pi projects further.