Install Ubuntu 18.04 + OpenCV + TensorFlow Lite on Raspberry Pi 4
This long article guides you through the installation of Ubuntu 18.04 on a Raspberry Pi 4. The Ubuntu version used has been specially developed for a Raspberry Pi 4. Now, for the first time, your Pi is running on a 64-bit operating system, while the well-known Raspbian operating system is still 32 bits.
At the same time, we will install OpenCV 4.2.0 and TensorFlow Lite 2.1.0 to see if a 64-bit operating system gives the promised performance boost.
A word of caution. The installed Ubuntu (aarch64-linux-gnu) is based on an ARM core. Not all Linux software may be running on this version because it may be developed for an Intel processor (x86_64-linux-gnu). Another point is the hardware interface on the Raspberry Pi. It can be a bit of a headache to make everything work as we know it. We have chosen for the 18.04 (Bionic Beaver) version instead of the 19.10 (Eoan Ermine) because the support of the later will be dropped half 2020. The first step installing Ubuntu 18.04 is flashing it on a SD card.
64 Gbyte SD-micro.
We use Ubuntu on a 64 Gbyte SD card, so it will not become exhausted immediately. Flashing an image on an SD-micro card up to 32 Gbyte this is nothing special. However, above 32 Gbyte, Windows will always format the card with the exFAT file system, a conversion unsupported by your Raspberry Pi. The Raspberry Pi only works with a FAT32 system. Special software is needed to format your 64 or 128 Gbyte SD-micro card. We use MiniTool Partition wizard.
The slide show gives you an impression of how to format an SD card that has already been used for a Linux distribution. If you have a new card, you have probably one large section instead of the two we have. Before using the tool, be sure your working on the SD card, not on a Windows disk in your machine.
With the SD-card formatted, the Ubuntu image can now be flashed. Download the ubuntu-18.04.4-preinstalled-server-arm64+raspi3.img.xz image from the Ubuntu site. We use the 64-bit version for the Raspberry Pi 4. The image is best be flashed by Etcher.
If your Raspberry Pi has proper cooling, you might consider overclocking. Ubuntu will run somewhat faster. However, don't push your luck too far. During intensive calculations, for instance, running deep learning models, your system may crash. We use 1925 MHz as an absolute maximum. Above this number, your system will still start up, but there is a chance that it will crash at a certain point. For example, at 1950 MHz, we could not properly execute our TensorFlow Lite example. More information on our overclocking page.
To overclock, you have to add a few lines in the usercfg.txt file in the root folder of your SD-card. Please, use only Notepad++ as it supports Linux line endings of text files (\n) instead of Windows (\r\n).
Add the following lines in your usercfg.txt. If you are not comfortable with the 1925 MHz, or if you have doubts about your cooling, you can lower this number. Your Raspberry Pi 4 runs default at 1500 MHz, so these extra lines are not necessary if you run at that speed. By the way, you can always modify the figures at any time. Some issues with the Raspberry Pi 4 Gbyte RAM version have been reported. We assume that the Ubuntu Foundation will solve this problem as quickly as possible. In the meantime, limit the RAM usage to 3 Gbytes with the line total_mem=3072.
The SD card can now be inserted in the Raspberry Pi. Very important; you must have a working internet connection from the very start. Power up and see many lines scroll across your screen during the installation of Ubuntu for the first time. After about 30 seconds you end up with this screen. If you are missing the prompt, or the login line, give a <enter>.
Log in with ubuntu. Password is also ubuntu. You will now be asked to provide a new password, so let's do that.
After a few lines you see the ubuntu prompt and your Ubuntu 18.04 is installed properly.
Before the desktop can be installed, there are a few administrative tasks to do. One very important thing is the swapping memory. You must give Ubuntu more breathing space than a Raspbian OS. We use zram-config to take care of the swapping. Please follow the next commands. The last line installs your Ubuntu desktop. This will take about 20 minutes to complete.
# sure there are upgrades, so get them
ubuntu@ubuntu:$ sudo apt-get update
ubuntu@ubuntu:$ sudo apt-get upgrade
# install zram
ubuntu@ubuntu:$ sudo apt-get install zram-config
# install your desktop
ubuntu@ubuntu:$ sudo apt-get install ubuntu-desktop
The desktop must be start-up once with the command startx. Next time it is loaded right away. With the desktop now installed, you can customize it the way you like. Enable WiFi etc.
Before we can fully enjoy Ubuntu, we must increase the standard swap space. Enter the following command in a terminal window.
$ sudo nano /usr/bin/init-zram-swapping
# after saving the modifications, restart
Add a multiplier (* 5) at the end of the mem.... line. This gives you about 4.5 Gbyte of swap memory.
The second modification is the swap priority near the end of the file. Here, change the number 5 to 3.
Save and close with <Ctrl+X>, <Y>, <Enter>. Before the changes can take effect, reboot.
Next time, you can check your system with the command htop. You get the following screen, where at the top the actual and maximum swap is given.
The final step is, of course, the installation of the beautiful Raspberry desktop screen we have created for you. You can download it here.
Now you have a fully functional and pretty fast 64-bits Ubuntu machine on your Raspberry Pi 4. You can do what you like, LibreOffice, browsing, coding, building apps, you name it. And, you will not easily limited with 64 Gbyte memory onboard. By the way, your clock speed can be checked with the command: $ lscpu | grep MHz.
Install OpenCV 4.2.1.
The next step in our journey to the fastest deep learning app on a bare Raspberry Pi 4 is the installation of OpenCV. We will use the newest 4.2.1 version. For those who have done this before, there are not many surprises. Be sure your swap space, set in init-zram-swapping, is more then 3 Gbyte, otherwise the installation will fail. Please note that the Raspberry Pi 4 ARMv8 core no longer supports VFPV3, but instead has FP16, smaller 16-bits floating point registers on board. To keep in mind when providing the options on the CMake command line. Let start, as usual, with the installation of the dependencies.
# to get your administation in order
$ sudo apt-get update
$ sudo apt-get upgrade
# now we can get the whole lot$ sudo apt-get install build-essential cmake git unzip pkg-config
$ sudo apt-get install libjpeg-dev libpng-dev libtiff-dev
$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev
$ sudo apt-get install libgtk2.0-dev libcanberra-gtk*
$ sudo apt-get install libxvidcore-dev libx264-dev
$ sudo apt-get install python3-dev python3-numpy python3-pip
$ sudo apt-get install python-dev python-numpy
$ sudo apt-get install libtbb2 libtbb-dev libdc1394-22-dev
$ sudo apt-get install libv4l-dev v4l-utils
$ sudo apt-get install libopenblas-dev libatlas-base-dev libblas-dev
$ sudo apt-get install liblapack-dev gfortran libhdf5-dev
$ sudo apt-get install libprotobuf-dev libgoogle-glog-dev libgflags-dev
$ sudo apt-get install gcc-arm*
$ sudo apt-get install protobuf-compiler
With the dependencies in place, it is time to download OpenCV and unzip the packages, followed by some administartion.
# download OpenCV 4.2.0
$ cd ~
$ wget -O opencv.zip https://github.com/opencv/opencv/archive/4.2.0.zip
$ wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.2.0.zip
# upzip both packages
$ unzip opencv.zip
$ unzip opencv_contrib.zip
# rename the folders
$ mv opencv-4.2.0 opencv
$ mv opencv_contrib-4.2.0 opencv_contrib
# finally, make the build directory
$ cd opencv
$ mkdir build
$ cd build
This is a important step. Here you tell CMake what, where and how to make OpenCV on your Ubuntu system.
Note, there are only bare spaces before the -D flags, not tabs. Keep also a single space between the -D and the argument. Otherwise, CMake will misinterpreter the command. The two last dots are not a typo, it tells CMake where the CMakeLists.txt file is located; one directory up. The OPENCV_GENERATE_PKGCONFIG flag must be set because we are going to use the package config when compiling TensorFlow Lite C++ API.
$ cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
-D ENABLE_NEON=ON \
-D BUILD_TIFF=ON \
-D WITH_FFMPEG=ON \
-D WITH_GSTREAMER=ON \
-D WITH_TBB=ON \
-D BUILD_TBB=ON \
-D BUILD_TESTS=OFF \
-D WITH_EIGEN=OFF \
-D WITH_V4L=ON \
-D WITH_LIBV4L=ON \
-D WITH_VTK=OFF \
-D OPENCV_ENABLE_NONFREE=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D INSTALL_PYTHON_EXAMPLES=OFF \
-D BUILD_NEW_PYTHON_SUPPORT=ON \
-D BUILD_opencv_python3=TRUE \
-D OPENCV_GENERATE_PKGCONFIG=ON \
-D BUILD_EXAMPLES=OFF ..
Please note the absence of the ENABLE_VFPV3=ON flag used by the building of OpenCV for the Raspbian 32-bits ARMv7 version.
If you had used this flag, cmake would generate erros. See the screendum below.
The error is quite misleading; Regex: "command line option. * Is valid for. * But not for C ++ '. This suggests an incompatibility of compilers instead of an unavailable option. 'Option VFPV3 not applicable' would be a better warning. OpenCV notorious gives these types of warnings, good to keep in mind. You can simply remove all your flags from the cmake command line and see if the build now succeeds. If so, add your flags one by one. This way, your problem will be located soon.
Hopefully, everything went well and CMake comes with a report that looks something like the screenshot below.
With all compilation directives in place, you can start the build with the following command. This will take a while (± 55 min). After the compilation, the libraries can be installed and added to the database.
$ make -j4
# install the libraries
$ sudo make install
$ sudo ldconfig
$ sudo apt-get update
You can now check your installation in Python 3. It all speaks for itself.
Install TensorFlow Lite.
With OpenCV on place, it is now time to install TensorFlow Lite. We only install the C++ API libraries. We need them later to build the app. The procedure is very simple. Just clone the GitHub repository and run two scripts. The commands are listed below.
$ git clone https://github.com/tensorflow/tensorflow
$ cd tensorflow
For those who have previously installed TensorFlow Lite on a Raspberry Pi, note the subtle difference in the last line. Here we use build_aarch64_lib because Ubuntu is a 64-bit operating system compared to the 32-bit Raspbian versions, build_rpi_lib.
The last step is building the TensorFlow Lite flat buffers. Please use the following commands.
$ cd ~/tensorflow/tensorflow/lite/tools/make/downloads/flatbuffers
$ mkdir build
$ cd build
$ cmake ..
$ make -j4
$ sudo make install
$ sudo ldconfig
If everything went well, you should have the two libraries and two folders with header files as shown in the slide show.
The last installation is the C++ IDE. You can use many programs. We like Code::Blocks. It is relatively simple and has good debug facilities. You can install the program with the next command.
sudo apt-get install codeblocks
First, download the software from our GitHub page. Please be sure it's the 64-bits version for Ubuntu, not the 32-bits Raspbian version.
Once all the files are placed in a folder (we used ~/software/tensorflow_lite) on your Raspberry Pi, start Code::Blocks and load the TestTensorFlow_Lite.cbp project file. Or simply double click on the TestTensorFlow_Lite.cbp in your File Manager.
- Once the project is loaded, Code::Blocks looks like this. First, select the release mode. We are not going to debug the code now.
- Open the Build option dialog with menu option Project → Build options... Notice the search directories in this dialog. For both Debug and Release there are four directories given. The well known OpenCV folder (/usr/local/include/opencv4) and the tensor library (/home/ubuntu/tensorflow). Indeed, the one we just created. The two other directories are pointing at the location where some flat buffer software is located.
- In the linker settings, there are references to two libraries who will be statical loaded into the code; libtensorflow-lite.a and libflatbuffers.a. There are two additional options given in the Other linker options list. -pthread enables the POSIX thread engine for TensorFlow. The second flag is the OpenCV package load.
- Check the Release mode optimalisation on the Compiler Flags tab; -O3 must be selected.
- Close the dialog, build and run the program.
- The output is a single file located in the Release folder. Before you can run the application here, you need to copy the associated files to this location.
As stated in the introduction, Ubuntu does not support all the Raspberry Pi specific hardware, like the Raspicam and GPIO's. There are some guides on the internet explaining who to install the Raspicam software. We didn't test them. Most procedures re-install old 32-bits libraries to get the Raspicam working.
We simply used an ordinary webcam. You can download several utilities to get a webcam working under Ubuntu. We used Camorama Webcam Viewer, which can be installed directly from the Ubuntu software collection. Once installed, OpenCV is also capable of using the webcam in the TensorFlow-Lite application. Alter line 115 in MobileNetV1.cpp and you have a live object detector.