Install ncnn deep learning framework on a Raspberry Pi 4.

Last updated: May 16, 2023

Introduction.

This page will guide you through the installation of Tencent's ncnn framework on a Raspberry Pi 4. The given C ++ code examples are written in the Code::Blocks IDE for the Raspberry Pi 4. We only guide you through the basics, so in the end, you can build your application. For more information about the ncnn library, see: https://github.com/Tencent/ncnn. Perhaps unnecessarily, but the installation is the C ++ version. It is not suitable for Python.

Tip.

Regularly, we get the question if we have an SD image of a Raspberry Pi 4 with pre-installed frameworks and deep-learning examples.

We are happy to comply with this request. Please, find a complete working Raspberry Pi 4 dedicated to deep learning on our GitHub page. Download the zip file from our GDrive site, unzip and flash the image on a 16 GB SD-card, and enjoy!

Dependencies.

The ncnn framework has almost no dependencies. It requires protobuf to load ONNX models. And OpenCV would be useful, but not necessary.

Version check.

Please check your operating system before installing ncnn on your Raspberry Pi 4. Run the command uname -a and verify your version with the screen dump below.

In case of a 64-bit operating system, please check also your C++ compiler with the command gcc -v. It must also be an aarch64-linux-gnu version. In case of a different gcc version, reinstall the whole operating system with the latest version. The guide is found here: Install 64 bit OS on Raspberry Pi 4. You must have a 64-bit C ++ compiler as we are going to build the ncnn libraries.

Also note the zram swap size of more than 3 Gbyte after installation according to our instructions.

RTTI.

RTTI stands for Run-Time Type Identification. It is a C ++ mechanism used at runtime to get the type and memory size of an object, yet not been defined. Normally, a programmer knows the type of variable and can allocate memory that holds the object in advance. Obtaining memory from an operating system with all its processes and threads can be a relatively time-consuming operation. Modern C compilers know how much memory is required and one call to memory management is sufficient. It's one of the main advantages over Python, which is oblivious to memory requirements until it hits the line of code with a variable.

It is best not to use RTTI if you want to write the fastest possible code. It is also the case in the ncnn framework.

By default, it is compiled with a -fno-rtti flag that prevents the use of RTTI. Compiled with this flag, custom-defined layers like those found in YOLOV5 are only usable if the remaining program is still not using RTTI.

Sometimes it is not possible to avoid RTTI. Especially in mature code that requires a new function without rewriting all the code. OpenCV uses the RTTI mechanism in some places.

There is a problem here. When you compile ncnn with OpenCV, the compiler returns an error on the -fno-rtti flag. Removing the flag sometimes works with the ncnn code, depending on the type of DNN used.

At this point, the used build flag -D NCNN_DISABLE_RTTI=OFF becomes clear. It tells the compiler ncnn will allow RTTI. It means that you can now run ncnn in its full functionality without getting into problems with OpenCV. Or, for that matter, any other piece of software using RTTI.

Performance-wise, you will not notice any difference on your Raspberry Pi.

Installation.

Install OpenCV first if it is not already installed. The installation guide is here and takes about an hour.

The entire installation of ncnn on a Raspberry is as follows.

Raspberry 64-bit (aarch64)

# check for updates
$ sudo apt-get update
$ sudo apt-get upgrade
# install dependencies
$ sudo apt-get install cmake wget
$ sudo apt-get install build-essential gcc g++
$ sudo apt-get install libprotobuf-dev protobuf-compiler
# download ncnn
$ git clone --depth=1 https://github.com/Tencent/ncnn.git
# install ncnn
$ cd ncnn
$ mkdir build
$ cd build
# build 64-bit ncnn
$ cmake -D NCNN_DISABLE_RTTI=OFF -D NCNN_BUILD_TOOLS=ON \
-D CMAKE_TOOLCHAIN_FILE=../toolchains/aarch64-linux-gnu.toolchain.cmake ..
$ make -j4
$ make install
# copy output to dirs
$ sudo mkdir /usr/local/lib/ncnn
$ sudo cp -r install/include/ncnn /usr/local/include/ncnn
$ sudo cp -r install/lib/libncnn.a /usr/local/lib/ncnn/libncnn.a

Raspberry 32-bit (armv7l)

# check for updates
$ sudo apt-get update
$ sudo apt-get upgrade
# install dependencies
$ sudo apt-get install cmake wget
$ sudo apt-get install build-essential gcc g++
$ sudo apt-get install libprotobuf-dev protobuf-compiler
# download ncnn
$ git clone --depth=1 https://github.com/Tencent/ncnn.git
# install ncnn
$ cd ncnn
$ mkdir build
$ cd build
# build 32-bit ncnn
$ cmake -D NCNN_DISABLE_RTTI=OFF \
-D NCNN_BUILD_TOOLS=ON \
-D CMAKE_EXE_LINKER_FLAGS=-ldl \
-D CMAKE_TOOLCHAIN_FILE=../toolchains/pi3.toolchain.cmake ..
$ make -j4
$ make install
# copy output to dirs
$ sudo mkdir /usr/local/lib/ncnn
$ sudo cp -r install/include/ncnn /usr/local/include/ncnn
$ sudo cp -r install/lib/libncnn.a /usr/local/lib/ncnn/libncnn.a

If everything went well, you will get two folders. One with all header files and one with the library as shown in the screen dumps.

Please note also the folder with the examples. Many different types of deep learning are covered here. The references to the actual deep learning models can sometimes cause errors due to version changes in the ncnn library. We recently received the following repository from nihui with the latest models: https://github.com/nihui/ncnn-assets/tree/master/models.

Tools.

The ncnn framework comes with some useful tools, located in the ~/ncnn/build/install/bin folder.

If you don't need the tools, or the examples, you can delete the entire ncnn folder. It saves you about 164 MB of disk space.

$ cd ~

$ sudo rm -rf ncnn

Deep learning software for Raspberry Pi

Deep learning examples for Raspberry Pi

Install ncnn on Raspberry Pi 4 - Q-engineering

Q-engineering