Computer vision with Raspberry Pi and alternatives in 2022

Last updated: October 11, 2022

Introduction.

Introduced on February 29, 2012, the Raspberry Pi has become a huge success. Now, after 4 generations, over more than 23 million have been sold. With such a market, other manufacturers inevitably try to get their piece of the action. Nowadays, many Raspberry alternatives with equal or superior specifications are available. This page investigates how to employ successfully this board or an alternative for your computer vision project. The list at the end is certainly not complete, it is more an impression.

First, what is a Raspberry exactly? It is a so-called System On a Chip. In the old days of computing, every single chip had its own functionality, like RAM, ROM, UART, PIO, CPU, etc. A large printed circuit board was needed for all the wiring. See the image above of the first 8080 microcomputers in 1978. As technology progresses, chips contain more transistors, more functionality. Today, the entire 8080 PCB, as shown above, can easily fit into a tiny chip. In other words, all systems go in a single chip. Smartphones would not exist without these SOCs.

Six releases later, the Raspberry is no longer a simple DIY project but has grown mature. In many cases, it has sufficient performance to replace a PC with computer vision tasks. And with its I/O on board, interfacing with the real world become also effortless. A Raspberry or an alternative can be a small, cheap, robust solution.

Preparations.

Before you build your computer vision project on a Raspberry Pi or an alternative, there are some things that need to be sorted out.

The first and most important question is of course: can the program be executed on a single computer board? Let's be realistic, a Raspberry Pi or for that matter a fast alternative is unable to match the modern PCs. Not in terms of computing power, but also in terms of memory size.

If you have already an application running on a PC, it is not that difficult to estimate the required memory usage, nor the execution time. Apply a factor eight on the later, and if the performance is still acceptable, your application could run on a Raspberry. In addition, keep in mind that the Raspberry does not have CUDA GPU acceleration. A fraction of this GPU computing power can be found in the VideoCore. Read more about this topic here.

If the application must be written from scratch, we always recommend starting in a PC environment. Writing vision programs always require a lot of debugging. No better place than a normal computer with all the appropriate tools installed. At the end of the day, you can always make the decision to port the app to a Raspberry or an alternative. If not possible, no harm is done. You can still run the project on a PC. One exception; when an almost identical application is available in the public domain, we tend to adopt such a version to see if it brings us what we want.

Python vs C++.

This is a never-ending discussion. Some people love to work in C++. Others adore the comfortable Python language. And never the twain shall meet.

It doesn't matter which language you choose. As long as you have a good feeling about it and can program what you want. Python uses a little more memory and is a bit slower. C++, on the other hand, is faster, but it is much hard to debug. One word of advice. Once started, stick to your choice.

Do not transfer your project from a PC to a Raspberry and simultaneously port the software from one language to another. Without exception, converting software takes a lot of time and is always notorious error prone.

Operating system.

A Raspberry and most alternatives like NanoPi have ARM processors. A Linus-like operating system such as Raspbian, a Debian-based distribution, as well as third-party Ubuntu, is well suited for these processors. Once Windows has recognized the huge market potential of ARM cores in smartphones, it has also launched an operating system suitable for the Raspberry and its variants: Windows 10 IoT.

Again, it doesn't matter which operating system you prefer. They can all do there job. Note, the last two boards in the list below are made with Atom Intel core. These two certainly prefer a Windows OS. However, there are two points to keep in mind: drives and community.

Drivers.

Of course, when using a Raspberry and the prescribed hardware and software, an operating system such as Raspbian has all drivers pre-installed.

But this is not always the case. Suppose you want to use your own camera, or certain GPU software is not supported by the operating system, you have to look for the proper driver. Programmable hardware is meaningless without the software drivers. It is therefore imperative that you can rely on the delivery of the software drivers for all functionality on the card. Also their documentation. We have seen beautiful boards with very impressive features but without the drivers. Consequently, we could not use them. Searching for drivers can be very tedious, as we experienced using the Banana Pi M2 Zero. In the end, we have to rebuild the entire Linux OS after developing the OV5640 camera driver ourselves.

Community.

The community is just as important as the available drivers. With thousands of enthusiastic members, forums are a rich source of information. Why reinvent the wheel? There is always someone out there who has had the same problems as you and has found the solution. It saves you so much time. If you have made a choice for a board, first check all related forums. See what kind of problems they are dealing with. And whether there are good solutions. You will probably have to deal with the same problems. We cannot emphasize enough the importance of an internet community.

Cooling.

The CPU on every board must be cooled when running at high clock rates. Some boards like the Odroid UX4 comes with a fan. Other like NanoPi or the Raspberry has no cooling at all. Without cooling, the CPU certainly gets too hot and the clock automatically lowers for a while, the so-called throttle. Even worse, every 10 °C (18 °F) increase in heat halves the life of a CPU, as explained here. Tiny heatsinks as shown below will absolutely not do, their capacity is too low. Use large ones or fans.

Periphery.

Every board has its usual periphery. The Raspberry Pi 3 B+ comes with 4x USB 2.0, GPIO, Ethernet, WiFi, Bluetooth. The Odroid UX4 has 1x USB 2.0, 3x USB 3.0, GPIO and Gigabit Ethernet. The NanoPi M4 has 2x USB 2.0, 2x USB 3.0, GPIO, Giga Ethernet, WiFi, and Bluetooth. As can see, there are just minor differences between the boards.

More important for vision applications is, of course, the interfacing of the camera.

NanoPi M4 has two 4-Lane MIPI-CSI camera interfaces, Raspberry has one 4-Lane MIPI-CSI camera interface and the Odroid UX4 has not a single camera interface.

In the case of the Odroid UX4, the camera must be connected to USB or Ethernet.

Also if one wants to use another camera than the standard 5 megapixel OV5647 from Raspberry, for example a camera with better optics or with a global shutter, like the Arducam Global Shutter OV7251 or OV928, then finding the right drivers becomes now even more important. The supplier of the camera may be able to offer support. OpenCV has also a number of universal USB camera routines.

GPU.

The diagram below shows the Raspberry SoC BCM2835. Here you can see what role the GPU (VideoCore IV) plays in the data stream when capturing live video. It receives the captured frames through the CSI-2 interface (Unicam) and writes the frames into RAM. Next, some image signal processing (ISP) is done. After this step, the image is ready to be displayed as an unencoded output (YUV or RGB). The data is copied over to the CPU via DMA. If encoding like H264 or MPEG is desired, the frame goes to the next H264 block where these operations will take place. Once finished, the output is copied to the CPU. The VPU coordinates all interaction with the CPU through the VCHI.

Note also the RAM below in the diagram. It is a shared memory between the CPU and GPU. It is one physical component. This is in contrast to normal GPUs on video cards, which has there own memory. The entire chain applies to Raspberry cameras. All drivers and other software are included within the operating system. It may be clear that the use of another camera (for example an Ethernet cam) has serious consequences for the video data flow and how the new drivers must somehow restore at least parts of the chain.

Practice.

Always when you decide to do your computer vision with a Raspberry Pi or an alternative, start with the desired camera. See if you can get live images on your board of choice. Once this is achieved, the most difficult part is done. There are still many problems to solve, but at least your hardware and software do their basic functions.

Next, you have to think about some kind of environment where the application will be built. The worst case is building images on a PC, burning them to the microSD and loading them into your board. This kind of trial and error takes an enormous amount of time.

It is better to take a series of photos or video streams with the board, save them and build the application on a PC using them. Once satisfied with the result, you can port the application to the board. You still have to make an image of your program and burn it to microSD. But now you can be pretty sure, everything works as it should be.

One tip: there are programs such as VisualGDB that enable live debugging of a Linux system from a second PC running Visual Studio. Perhaps a possibility.

Raspberry Pi and recent alternatives.

Below a selection is made between Raspberry Pi and some recent alternatives. Most have extensive GPU hardware on the chip. The last two boards are not ARM, but Intel-based. Pixy2 is just for fun. Please note that the price quoted is from January 2022, before the global severe chip shortage.

	Raspberry Pi 3 B+ 4x Cortex-A53 CPU VideoCore IV GPU 1.2 GHz - 1 GB ★☆☆☆☆ (16) € 40 Parent of all boards. Still one of the most sold. Lots of code and support available.
	Raspberry Pi 4 B 4x Cortex-A72 CPU VideoCore VI GPU 1.5 GHz - 1/2/4/8 GB ★★☆☆☆ (30) € 40/€ 40/€ 60/€ 85 The successor to the Raspberry Pi 3 with a faster processor, USB 3.0 and GigaEhternet.
	Rock Pi 4 B 2x Cortex-A72 + 4x Cortex-A53 CPU 4x Mali-860 GPU 1.8/1.4 GHz - 1/2/4 GB ★★★☆☆ (53) € 68/€ 77/€ 100 Pin to pin replacement of the Raspberry Pi 4 with a faster processor, USB 3.0 and GigaEhternet.
	Pine A64-LTS 4x Cortex-A53 CPU 2x Mali-400 GPU 1.2 GHz - 2 GB ★★☆☆☆ (22) € 40 Replacement for the Raspberry with more GPU performance and RAM.
	Le Potato 4x Cortex-A53 CPU 2x Mali-450 GPU 2.0 GHz - 2 GB ★★☆☆☆ (36) € 50 Good replacement for the Raspberry. Same board layout and pinning. Higher clock, hence the 36 ranking.
	Asus Tinker board 4x Cortex-A17 CPU 1x Mali-764 GPU 1.8 GHz - 2 GB ★★★☆☆ (43) € 65 Pin for pin replacement for a Raspberry. Despite the 32 bits, a lot faster. And more sophisticated interfacing.
	NanoPi M4 2x Cortex-A72 + 4x Cortex-A53 CPU 1x Mali-864 GPU 2.0 + 1.5 GHz - 2 GB ★★★☆☆ (57) € 75 (Discontinued) A fast hexa-core alternative. Due to the affordable Rockchip an attractive choice.
	RockPro64 6x Cortex-A53 CPU 4x Mali-860 GPU 1.5 GHz - 4 GB ★★★☆☆ (60) € 100 Another fast hexa-core alternative. With a quad-core GPU coprocessor.
	Odroid XU4 4x Cortex-A15 + 4x Cortex-A7 CPU 6x Mali-628 GPU 2+1.4 GHz - 2 GB ★★★☆☆ (64) € 75 Octa-core 32 bits alternative. With a hexa-core GPU coprocessor. A lot of power for twice the price of a Raspberry. Note, no camera slot
	HiKey960 4x Cortex-A73 + 4x Cortex-A53 CPU 8x Mali-G71 GPU 2.3+1.4 GHz - 6 GB ★★★★☆ (134) € 250 An ultra high-performance alternative. With the superior specs comes also a matching price.
	Pixy2 2x Cortex-M4 CPU - 204 MHz - 264 KB ☆☆☆☆☆ (0,4) € 60 A cute little camera with a small 32-bit processor. With almost no memory or clock speed. But still able to do great tricks according to the promo video.
	LattePanda Intel Atom Z8350 quad-core Intel HD Graphics 1.8 GHz - 2 GB ★★★☆☆ (85) € 140 A Windows 10 pre-installed credit card computer. Price without Windows license.
	Minnowboard Turbot Intel Atom E3845 quad-core Quad Intel Gen 7 1.9 GHz - 2 GB ★★★☆☆ (87) € 200 Another Intel alternative. Windows, Linux or Android suitable. High-performance graphics.