Detect objects with Raspberry Pi using Google Coral Edge TPU
I finally managed to find time for testing the Coral USB Accelerator from Google. This USB device provides Edge TPU as a coprocessor enabling high-speed ML inferencing in local devices.
I wanted to track objects using a standard IP camera and Raspberry Pi. Raspberry Pi will record the RTSP stream from the IP camera and will pass the image to Coral USB Accelerator to do all the heavy lifting.
I’ll describe next how this was implemented. The steps are:
- Setting up Coral for Raspberry Pi (using Docker)
- Packaging the Coral’s object detection library as a Docker image
- Grabbing images from RSTP stream
- Detecting objects using Coral USB Accelerator
Check my earlier post for how to run applications on Raspberry Pi using Docker:
Setting up Coral for Raspberry Pi
Coral’s website has nice instructions on how to install everything: https://coral.ai/docs/accelerator/get-started Basically, you need to install Edge TPU runtime for accessing the Edge TPU on the USB device and Tensorflow Lite for using the object detection model.
As I wanted to run everything on a Docker container, I took a Debian image and installed everything on it.
Packaging the Coral’s object detection library as a Docker image
I have a Java app that holds the business logic so I wanted to package the whole object detection logic into a Docker image with a REST API. This way both applications can be run independently on Raspberry Pi and I don’t have to execute any Python scripts from Java code.
For this purpose, I created a simple Python script that loads the object detection model with Tensorflow Lite and builds a REST API with FastAPI. The detection code can be found from Github https://github.com/google-coral/tflite/tree/master/python/examples/detection and the different object detection models from https://coral.ai/models/
The necessary dependencies are installed in the Docker image and the app is run with Uvicorn:
When running it on Raspberry Pi, you need to add the location of the USB devices as a volume so Coral USB Accelerator can be accessed from the container.
docker run --name objdet \
-p 8000:8000 \
-v /dev/bus/usb:/dev/bus/usb \
<DOCKER IMAGE NAME>Grabbing images from RSTP stream
The main two libraries I am using are ONVIF-Java and JavaCv, for communicating with Onvif-compatible cameras and for grabbing images from the camera’s RTSP stream, respectively. I’m not going into details on how to do it here. There are plenty of tutorials for both of these libraries.
But when you have set up your camera, you can use JavaCV to load the image from your camera and then convert it to a byte array to be send it to the REST API.
The returned JSON array contains the predicted labels with a probability and a bounding box containing the location of the object in the image.
[{"label": "car", "score": 0.83984375, "bbox": [40, 10, 409, 282]}]The time spent to perform a single inference seems to be only some milliseconds but the total time is 40–60 ms per image. I’m running this on an older version of Raspberry Pi and the app is recording video from the IP camera at the same time, which probably explains the slowness.
