Install TensorFlow Lite 2 on Raspberry Pi 4
64-OS TensorFlow Lite
Last updated: November 7, 2022
Advise.
Use a 64-bit OS. Deep learning applications use a lot of resources. Some deep learning models will even not run on a Raspberry Pi given the limited amount of memory and CPU power. That's why we strongly advise to use a 64-bit operating system on your Raspbeery Pi. Nowadays it's evoled to a fullgrown and robust OS. With the 64-bit OS, your deep learning application will simply be a lot faster.
Introduction.
TensorFlow Lite will be installed on your Raspberry Pi 4 with a 32-bit operating system, along with some examples. The manual describes the version of TensorFlow Lite up to version 2.6. You can always install an older version by changing the version number in the download command. For example, $ wget -O tensorflow.zip https://github.com/tensorflow/tensorflow/archive/v2.2.1.zip, will download version 2.2.1 on your Raspberry Pi.
We also guide you through installing TensorFlow Lite on an 'old' Raspberry Pi zero, with its armv6 processor.
Roadmap.
TensorFlow continues to grow. Each new release requires more resources, supporting software and libraries. It increasingly puts a heavy burden on your Raspberry Pi. It explains why the latest versions don't work well on 'old' Raspberry Pis with an 'outdated' operating system.
On the other hand, it is not advisable to install a very old TensorFlow version on the latest Bullseye. You will be forced to downgrade some system libraries, which will prevent other software from working. The motto here: "Go with the Flow".
Below is an overview. A green checkmark indicates a C++ tar ball or Python3 wheel is available on our GitHub repo. An empty green box means there is no wheel, but installation is still possible. A grey box specifies hardware or software limitations that do not allow a 'normal' installation.
| System | TF 2.0 | TF 2.1 | TF 2.2 | TF 2.3 | TF 2.4 | TF 2.5 | TF 2.6 | TF 2.7 | TF 2.8 | TF 2.9 | TF 2.10 |
| Raspberry Pi 32-bit Buster | |||||||||||
Raspberry Pi 32-bit Bullseye | |||||||||||
Raspberry Pi 64-bit Buster | √ | √ | √ | √ | |||||||
Raspberry Pi 64-bit Bullseye | √ | √ | √ | √ | √ | ||||||
| Raspberry Pi Ubuntu 18.04 | √ | √ | √ | √ | |||||||
| Raspberry Pi Ubuntu 20.04 | √ | √ | √ | √ | |||||||
Raspberry Pi Ubuntu 22.04 | √ | √ | √ | √ | |||||||
| Jetson Nano JetPack 4.6 | √ |
Preparations.
To compile Tensorflow Lite, you need enough memory onboard. Make sure you have at least a 1.5 GByte of memory available. In the case of the Raspberry Pi Zero (2), with its 500 MByte RAM, you need to expand the swap space to minimal 1024 MByte. Please follow the instructions given on this page on how to do so.
If you want to run the TensorFlow Lite examples we provide, please make sure you have OpenCV 4.5 installed on your Raspberry Pi according to our guide.
Besides OpenCV, you also needed Code::Blocks installed. It would be comforting if you have already run our first C++ example, James.mp4, on your Pi.
Install TensorFlow Lite.
To build a very fast deep learning application, we need to install the C++ API libraries. The procedure is simple. Just copy the GitHub repository and run two scripts. The commands are listed below. The whole procedure works well for all versions up to 2.6. Above version 2.6 TensorFlow Lite has dropped the compile scripts. Now, you have to build the C++ libraries with CMake. Please see our 64-bit guide on this subject.
When everything is done, you end up with a screen like this.
The TensorFlow Lite flat buffers are also needed. If you have the latest Bullseye OS running on your Raspberry Pi, you will get errors during the compilation of the flatbuffers. It has to do with the gcc version 10.2.1 giving some warnings now the 'error' status. The flatbuffers team has already fixed these warnings. However, TensorFlow still uses the 'old' version of the flatbuffers.
Please use the following commands.
If everything went well, you should have the two libraries and two folders with header files as shown in the slide show.
As of version 2.3.0, Tensorflow Lite uses dynamic linking. At runtime libraries are copied to RAM and pointers are relocated before TF Lite can run. This strategy gives greater flexibility. It all means that TensorFlow Lite now requires glibc 2.28 or higher to run. From now on, link the libdl library when building your application, otherwise, you get undefined reference to symbol dlsym@@GLIBC_2.17 linker errors. The symbolic link can be found at /lib/aarch64-linux-gnu/libdl.so.2 on a 64-bit Linux OS, or /lib/arm-linux-gnueabihf/libdl.so.2 on a Raspberry Pi 32-bit OS. Please see our examples on GitHub.
Install TensorFlow Lite on an 'old' Raspberry Pi Zero.
The installation above includes all Raspberry Pi with an ARMv7l chip (RPi 2, RPi 3) or an ARMv8-a (RPi 4). However, the Raspberry Pi Zero ships with an ARMv6. It is possible to install TensorFlow on the Raspberry Pi Zero. Just replace the definition TARGET:=armv7l with TAGRET:=armv6 in the file rpi_makefile.inc. Then follow the same steps as above as if you were dealing with a Raspberry Pi 4.
The compilation will take approximately 4½ hours. Remember that with only one core, clocked at 1GHz and no NEON registers, you are entering the zen zone. On our GitHub page you will find an example of TF-Lite running on a Raspberry Pi Zero, which classifies objects.
TensorFlow Lite models.
You cannot run normal TensorFlow models on the Lite software, they must be converted before use. These TensorFlow pages explain how to do this. Google has some ready-made models available on the net here.
Object detection.
Another application is detecting objects in a scene. TensorFlow Lite host one model for now. COCO SSD MobileNet v1 recognize 80 different objects. It can detect up to ten objects in a scene. On GitHub we have a C++ example of the famous Skyfall intro running on a bare Raspberry Pi 4 for 32-bit.
Classification.
The most well known is, of course, the classifications of objects. Google hosts a wide range of TensorFlow Lite models, the so-called quantized models in their zoo. The models are capable of detecting 1000 different objects. All models are trained with square images. Therefore, the best results are given when your input image is also square-like. All models are supported on GitHub by our C ++ software samples for both the 32-bit and 64-bit Raspberry and Ubuntu 18.04 or 20.04 operating system.
Pose estimation.
This neural network tries to estimate a person pose in a scene. It recognizes certain key features like elbows, knees, ankles in an image. TensorFlow Lite supports two models, a single person and a multi-person version. We have only used the single person model because it gives reasonable good results when the person is centred and in full view in a square-like image. Please, find the 32-bit Raspbain C++ example at our GitHub page.
Segmentation.
With semantic image segmentation, a neural network attempts to associate every pixel in the scene with a particular object. You could say, it tries to detect the outline of objects. Tensorflow Lite has one segmentation model capable of classifying 20 different objects. Keep in mind that only reasonable sized objects can be recognized, not a scene of a highway with lots of tiny cars. The C++ examples can be found here for 32-bit.
Frame rate.
Here, some frame rates are given of the several TensorFlow Lite models tested on a bare Raspberry Pi 4. The overclock frequencies are indications. Ubuntu always crashes above 1950 MHz when running deep learning models with the 4 cores simultaneous. Some models could run at 1950 MHz, others not higher than 1825 MHz. The 32-bit Raspbian is capable of a clock rate of 1950 MHz for all the examples. Frame rates are only based on model run time (interpreter->Invoke()). Grabbing and preprocessing of a image are not taken into account. Also noteworthy is the higher frame rate on a Raspbian for the MobileNet models compared to Ubuntu. The guide to installing Ubuntu along with OpenCV and TensorFlow Lite can be found here. Overclocking is covered here. This is the guide to install the Raspberry Pi 64-bit operating system. Installing OpenCV 4.3.0 on the 64-bit Raspberry OS is explained here.
| Model | Raspberry Pi 4 64 bit OS 1950 MHz | Raspberry Pi 4 64 bit OS 1500 MHz | Raspberry Pi 4 64 bit Ubuntu 1850 MHz | Raspberry Pi 4 64 bit Ubuntu 1500 MHz | Raspberry Pi 4 32 bit OS 1950 MHz | Raspberry Pi 4 32 bit OS 1500 MHz |
| MobileNet-V1 SSD (300x300) | 26.8 FPS | 21.5 FPS | 24.0 FPS | 17.2 FPS | 17.0 FPS | 13.8 FPS |
| MobileNet-V1 (224x224) | 38.5 FPS | 32.2 FPS | 22.9 FPS | 22.5 FPS | 33.0 FPS | 22.2 FPS |
| MobileNet-V2 (224x224) | 45.5 FPS | 37 FPS | 19,7 FPS | 19,5 FPS | 36.2 FPS | 28.0 FPS |
| Inception-V2 (224x224) | 11.6 FPS | 9.5 FPS | 10.0 FPS | 8.7 FPS | 8.9 FPS | 6.9 FPS |
| Inception-V4 (299x299) | 2.1 FPS | 1.7 FPS | 2.0 FPS | 1.8 FPS | 1.6 FPS | 1.3 FPS |
| Unet (257x257) | 7.5 FPS | 6.8 FPS | 7.2 FPS | 6.6 FPS | 4.0 FPS | 3.6 FPS |
| PoseNet (257x257) | 10.3 FPS | 9.2 FPS | 9.4 FPS | 8.7 FPS | 5.0 FPS | 4.3 FPS |
Python installation of TensorFlow Lite.
For completeness, the Pythons installation of TensorFlow Lite 2.1.0 is given here. It is only one command. There are no wheels for TensorFlow Lite version 2.2.0, 2.3.0 and 2.3.1. Python examples can be found everywhere on the net. Google has also made an example here.
$ pip3 install https://dl.google.com/coral/python/tflite_runtime-2.1.0.post1-cp37-cp37m-linux_armv7l.whl
Deep learning examples for Raspberry Pi