Install TensorFlow 2.3.1 on Jetson Nano.
Introduction.
TensorFlow is a large software library specially developed for deep learning. It consumes a vast amount of resources. You can execute TensorFlow on a Jetson Nano, but don't expect miracles. It can run your models, if not too complex, but it will not be able to train new models. Nor can it perform the so-called transfer learning. In addition to running your pre-built deep learning models, you may use the library to convert so-called frozen TensorFlow models to TensorFlow Lite flat buffer models.
If you only want to get some impression of deep learning, please consider installing TensorFlow Lite. It is much faster and uses far fewer recourses, as being designed for mobile phones and small computers like a Jetson Nano. There are many ready build models you can use. See our installation guide here.
We discuss two installations, one for Python 3 and one C++ API library.
Numpy.
At the time of writing (December 2020), we were experiencing some issues with Numpy. When building TensorFlow from scratch we encounter '//tensorflow/python:bfloat16_lib' failed errors. It is related to using the latest Numpy version in conjunction with CUDA, as shown in GitHub ticket #40688.
We strongly recommend checking your Numpy version with pip3 list | grep numpy. If you have installed a version higher than 1.18.5, downgrade using pip3 uninstall numpy && pip3 install numpy==1.18.5.
Incidentally, the problems only occur if you use CUDA support. If you build TensorFlow without the CUDA backend, the latest version of Numpy (1.19.4) will work just fine.
Keep this also in mind when installing other software packages on your Jetson Nano. Unintended, they might upgrade your Numpy to later versions.
The shortcut.
TensorFlow is installed by a Google software installer called Bazel. In the end, Bazel generates a wheel to install the TensorFlow Python version or a tarball when it comes to installing the C++ version. We have posted the Bazel outcomes on our GitHub page. Feel free to use these shortcuts. The whole TensorFlow installation procedure from start to end takes many hours (±50 for Python, ±10 for the C++ library, overclocked to 1900 MHz). With all the tedious work already done, it takes still one hour and twenty minutes to install TensorFlow 2.3.1 on your Nano. For the diehards, the complete procedure is covered later in this manual.
TensorFlow 2.3.1 for Python 3.
The whole shortcut procedure is found below. The wheel was too large to store at GitHub, so Google drive is used. Please make sure you have latest pip3 and python3 version installed, otherwise, pip may come with the message ".whl is not a supported wheel on this platform".The installation will require about 2.1 GByte of disk space.
JetPack 4 comes with Python 3.6.9. Undoubtedly, the Python version will upgrade over time and you will need a different wheel. See out GitHub page for all the wheels.
# get a fresh start
$ sudo apt-get update
$ sudo apt-get upgrade
# install pip and pip3
$ sudo apt-get install python-pip python3-pip
# remove old versions, if not placed in a virtual environment (let pip search for them)
$ sudo pip uninstall tensorflow
$ sudo pip3 uninstall tensorflow
# install the dependencies (if not already onboard)
$ sudo apt-get install gfortran
$ sudo apt-get install libhdf5-dev libc-ares-dev libeigen3-dev
$ sudo apt-get install libatlas-base-dev libopenblas-dev libblas-dev
$ sudo apt-get install liblapack-dev
$ sudo -H pip3 install -U testresources numpy# upgrade setuptools 39.0.1 -> 50.3.2
$ sudo -H pip3 install --upgrade setuptools
$ sudo -H pip3 install pybind11 protobuf google-pasta
$ sudo -H pip3 install -U six mock wheel requests gast
$ sudo -H pip3 install Cython==0.29.21
# install h5py with Cython version 0.29.21 (± 6 min @1950 MHz)
$ sudo -H pip3 install h5py==2.10.0
$ sudo -H pip3 install keras_applications --no-deps
$ sudo -H pip3 install keras_preprocessing --no-deps
# install gdown to download from Google drive
$ pip3 install gdown
# download the wheel
$ gdown https://drive.google.com/uc?id=1oeSnkgJpwyudtTx-f5CE84B7e-Vkv3yK
# install TensorFlow (± 12 min @1500 MHz)
$ sudo -H pip3 install tensorflow-2.3.1-cp36-cp36m-linux_aarch64.whl
When the installation is succesful, you should get the following screendump.
TensorFlow 2.3.0 C++ API.
If you are planning to program in C++, you will need the C++ API build of TensorFlow instead of the Python version. Installing the C ++ library using the pre-build tarball from our GitHub page saves you a lot of time. Please follow the procedure below.
# get a fresh start
$ sudo apt-get update
$ sudo apt-get upgrade
# remove old versions (if found)
$ sudo rm -r /usr/local/lib/libtensorflow*
$ sudo rm -r /usr/local/include/tensorflow
# the dependencies
$ sudo apt-get install wget curl libhdf5-dev libc-ares-dev libeigen3-dev
$ sudo apt-get install libatlas-base-dev zip unzip
# install gdown to download from Google drive (if not already done)
$ sudo -H pip install gdown
# download the tarball
$ gdown https://drive.google.com/uc?id=1rlviefeu4w2amLsybZIOpcZP140WioDS
# unpack the ball
$ sudo tar -C /usr/local -xzf libtensorflow-2_3_1-JetsonNano.tar.gz
You should end up having your TensorFlow library installed at the /usr/local/lib location and the header files in the folder usr/local/include/tensorflow/c.
Installation from scratch.
Warning.
The complete installation of TensorFlow 2.3.1 from scratch takes more than two days on an overclocked Jetson Nano. To be precise, 50 hours on 2 GHz. Another disturbing point is the amount of disk space you need. The full build takes approximately 14 GByte. After that, you can free up an 11.5 GByte. In other words, don't start if you have a 32 GByte SD card. Or you can't miss your Nano for two days.
Memory swap size.
Building the full TensorFlow 2.3.1 package requires more than 6 Gbyte of RAM. It's best to temporarily reinstall dphys-swapfile to get the extra space of your SD card. Once the installation is complete, we will delete dphys-swapfile. Follow the following commands.
# install dphys-swapfile
$ sudo apt-get install dphys-swapfile
# give the required memory size
$ sudo nano /etc/dphys-swapfile
# reboot afterwards
$ sudo reboot.
If all went well, you should have something like this.
For the record, the figure shown is total amount of swap space allocated by dphys-swapfile and zram. Don't forget to remove dphys-swapfile when your done.
Bazel.
Bazel is a free software tool from Google used for automatically building and testing software packages. You could compare it to CMake, used by OpenCV, but the latter only builds software and has no test facility. Bazel is written in Java, a platform-independent language, largely based on C ++ in terms of syntax. To compile Bazel, we must first install Java and some other dependencies with the following commands.
# get a fresh start
$ sudo apt-get update
$ sudo apt-get upgrade
# install pip and pip3
$ sudo apt-get install python-pip python3-pip
# install some tools
$ sudo apt-get install build-essential zip unzip curl
# install Java
$ sudo apt-get install openjdk-11-jdk
Next, we can download and unzip the Bazel software. We need Bazel release 3.1.0 for TensorFlow 2.3.1, so be sure you install the right version.
$ wget https://github.com/bazelbuild/bazel/releases/download/3.1.0/bazel-3.1.0-dist.zip
$ unzip -d bazel bazel-3.1.0-dist.zip
$ cd bazel
During installation, Bazel uses a predefined ratio of the available working memory. This ratio is too small due to the limited size of the RAM of the Jetson Nano. To prevent crashes, we must define the size of this memory to a maximum 1600 Mbyte during the procedure. This is done by adding some extra information to the script file compile.sh. You can add the text -J-Xmx1600M to the line that begins with run..(around line 154). See the screen below. Use the well-known <Ctrl + X>, <Y>, <Enter> to save the change.
$ nano scripts/bootstrap/compile.sh -c
Once the Java environment for Bazel has been maximized to 1600 Mb, you can start building the Bazel software with the next commands. When finished, copy the binary file to the /usr/local/bin location so that bash can find the executable anywhere. The final action is to delete the zip file. The total build takes about 33 minutes.
# start the build
$ env EXTRA_BAZEL_ARGS="--host_javabase=@local_jdk//:jdk" bash ./compile.sh
# copy the binary
$ sudo cp output/bazel /usr/local/bin/bazel
# clean up
$ cd ~
$ rm bazel-3.1.0-dist.zip
# if you have a copied bazel to /usr/local/bin you may also
# delete the whole bazel directory, freeing another 500 MByte
$ sudo rm -rf bazel
Build TensorFlow 2.3.1 for Python 3.
With Bazel up and running we can start building TensorFlow 2.3.1 on our Jetson Nano for Python 3. It is almost becoming standard practice. First, install some dependencies, then download the zip from GitHub and finally unpack the software.
# get a fresh start
$ sudo apt-get update
$ sudo apt-get upgrade
# install pip and pip3
$ sudo apt-get install python3-pip
# the dependencies
$ sudo apt-get install build-essential make cmake wget zip unzip
$ sudo apt-get install libhdf5-dev libc-ares-dev libeigen3-dev
$ sudo apt-get install libatlas-base-dev libopenblas-dev libblas-dev
$ sudo apt-get install gfortran liblapack-dev
# upgrade setuptools 39.0.1 -> 50.3.2
$ sudo -H pip3 install --upgrade setuptools
$ sudo -H pip3 install keras_applications --no-deps
$ sudo -H pip3 install keras_preprocessing --no-deps
$ sudo -H pip3 install pybind11
$ sudo -H pip3 install h5py==2.10.0
of
$ sudo apt-get install python3-h5py
# download TensorFlow 2.3.1
$ wget -O tensorflow.zip https://github.com/tensorflow/tensorflow/archive/v2.3.1.zip
# unpack and give the folder a convenient name
$ unzip tensorflow.zip
$ mv tensorflow-2.3.1 tensorflow
The final step before building the Python 3 installation wheel is to configure Bazel. This is done by a script file and the command-line options. Let's start with the script file. With the following command, Bazel asks you a few questions. Define Python 3 as the default Python version.
$ cd tensorflow
$ ./configure
jetson@nano:~/tensorflow$ ./configure
You have bazel 3.1.0- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python3]: <enter>
Found possible Python library paths:
/usr/local/lib/python3.6/dist-packages
/usr/lib/python3.6/dist-packages
/usr/lib/python3/dist-packages
Please input the desired Python library path to use. Default is [/usr/local/lib/python3.6/dist-packages] <enter>
/usr/lib/python3/dist-packages
Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: n
No OpenCL SYCL support will be enabled for TensorFlow.
Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.
Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.
Do you wish to build TensorFlow with TensorRT support? [y/N]: y
TensorRT support will be enabled for TensorFlow.
Found CUDA 10.2 in:
/usr/local/cuda-10.2/targets/aarch64-linux/lib
/usr/local/cuda-10.2/targets/aarch64-linux/include
Found cuDNN 8 in:
/usr/lib/aarch64-linux-gnu
/usr/include
Found TensorRT 7 in:
/usr/lib/aarch64-linux-gnu
/usr/include/aarch64-linux-gnu
Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Each capability can be specified as "x.y" or "compute_xy" to include both virtual and binary GPU code, or as "sm_xy" to only include the binary code.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]: 5.3
Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.
Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: <enter>
Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native -Wno-sign-compare]: <enter>
Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: n
Not configuring the WORKSPACE for Android builds.
Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
--config=mkl # Build with MKL support.
--config=monolithic # Config for mostly static monolithic build.
--config=ngraph # Build with Intel nGraph support.
--config=numa # Build with NUMA support.
--config=dynamic_kernels # (Experimental) Build kernels into separate shared objects.
--config=v2 # Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
--config=noaws # Disable AWS S3 filesystem support.
--config=nogcp # Disable GCP support.
--config=nohdfs # Disable HDFS support.
--config=nonccl # Disable NVIDIA NCCL support.
Configuration finished
With the script file now all set and done, the massive build can start with the command below. The -Xmx3278m sets the memory size of the Java environment where Bazel runs. Give it as much space as possible. We advise 80% of your Jetson Nano memory size. In our case we have 4 GByte onboard, so 3278 MByte leaves just enough space for other threads apart from Bazel.
Another point is the number of cores that Basel uses. Always use just one core. Even with two, the build crashed due to memory shortage. Better a little slower than a crash and start all over again.
Close all other applications during the build, the less other processes are running, the faster your compilation will be.
Before the build can begin, give the location of the CUDA to the NIVIDIA configuration and its library database.
# reveal the CUDA location
$ sudo sh -c "echo '/usr/local/cuda/lib64' >> /etc/ld.so.conf.d/nvidia-tegra.conf"
$ sudo ldconfig
# start the great build
$ sudo bazel --host_jvm_args=-Xmx3278m build \
--config=opt \
--config=noaws \
--config=nogcp \
--config=nohdfs \
--config=nonccl \
--config=monolithic \
--config=cuda \
--config=v2 \
--local_cpu_resources=1 \
--define=tflite_pip_with_flex=true \
--copt=-ftree-vectorize \
--copt=-funsafe-math-optimizations \
--copt=-ftree-loop-vectorize \
--copt=-fomit-frame-pointer \
//tensorflow/tools/pip_package:build_pip_package
After 50 hours compilation, hopefully, you will see with the following screen.
Now we have to generate the wheel and install it. This is done by the commands below.
A few words about the installation of scipy. TensorFlow 2.3.1 uses scipy version 1.14.1. It is known to be problematic to install this version on a Raspberry Pi, due to the lack of a proper wheel. Some may install scipy from source. However, if you install the dependencies of scipy first, your newly created TensorFlow wheel will install scipy 1.14.1 correctly in the end. These dependencies are highlighted in light blue. By the way, this installation takes about an hour and a half.
# synthesize the wheel
$ sudo bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
# install some dependencies (if not already done)
$ sudo apt-get install libatlas3-base libopenblas-dev libopenblas-base libblas-dev
$ sudo apt-get install libgfortran5 liblapack-dev
# upgrade setuptools 47.1.1 -> 50.3.0
$ sudo pip3 install --upgrade setuptools
$ sudo pip3 install pybind11
$ sudo pip3 install Cython==0.29.21
# install h5py with Cython version 0.29.21 (± 6 min @1950 MHz)
$ sudo pip3 install h5py==2.10.0
$ sudo pip3 install -U --user six wheel mock
# get to the folder where the wheel is located and install tensorflow
$ cd /tmp/tensorflow_pkg
$ sudo pip3 install tensorflow-2.3.1-cp37-cp37m-linux_aarch64.whl
Install the TensorFlow 2.3.1 C++ API.
As mentioned earlier, you can install the TensorFlow 2.3.1 C++ API very quickly by using the tarball on our GitHub page; no need to go through lengthy installation procedures. For those who want to build the API themselves, the installation guide now follows.
First, install Bazel if not already done. The procedure is also descript above. Once Bazel is working, you can install the dependencies and download TensorFlow 2.3.1, if not already done for the Python 3 installation earlier.
# the dependencies
$ sudo apt-get install build-essential make cmake wget zip unzip
$ sudo apt-get install libhdf5-dev libc-ares-dev libeigen3-dev
$ sudo apt-get install libatlas-base-dev
# download TensorFlow 2.3.1
$ wget -O tensorflow.zip https://github.com/tensorflow/tensorflow/archive/v2.3.1.zip
# unpack and give the folder a convenient name
$ unzip tensorflow.zip
$ mv tensorflow-2.3.1 tensorflow
Again, we need to configure Bazel before the actual build can start. Despite we are going to build a C++ API, the Bazel script file needs to be set to Python 3 as default Python version and 'no' to all other questions.
$ cd tensorflow
$ ./configure
Once all the questions of the script file are answered, you end up with the same output screen as shown above.
The last step is the command line with its options. Again, almost identical to the one earlier used. Most important here is the --config=monolithic flag. Without this direction, TensorFlow will not work with OpenCV or visa versa. Just like the pip compilation, the memory for Java environment (-Xmx1624m) has to be maximized to 80% of the memory the Raspberry Pi has onboard. The last line indicates a tarball library build instead of a pip wheel.
$ sudo bazel --host_jvm_args=-Xmx1624m build \
--config=opt \
--config=noaws \
--config=nogcp \
--config=nohdfs \
--config=nonccl \
--config=monolithic \
--config=v2 \
--local_cpu_resources=1 \
--copt=-ftree-vectorize \
--copt=-funsafe-math-optimizations \
--copt=-ftree-loop-vectorize \
--copt=-fomit-frame-pointer \
//tensorflow/tools/lib_package:libtensorflow
Hopefully, you will see with the following screen.
The last step is installing the tarball. This is done by the command below.
$ sudo tar -C /usr/local -xzf bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz
Once installed you must have the same folders (/usr/local/lib and usr/local/include/tensorflow/c) as shown at the beginning of this page.
Cleaning.
After a successful installation, there are many files no longer needed. The intermediate object files generated by bazel occupy some 11.5 GByte (!) of your disk. Removing doesn't harm anything.
# assuming your still in ~/tensorflow
$ sudo bazel clean
If you had to reinstall dphys-swapfile, it's time to uninstall it again. This way you will extend the life of your SD card.
# remove the dphys-swapfile (if installed)
$ sudo /etc/init.d/dphys-swapfile stop
$ sudo apt-get remove --purge dphys-swapfile