The current state of Jetson devices through NVidia's JetPack is based on Python 3.6 ( including all the pre-built wheels and docker containers ). Python 3.6 is increasingly breaking a lot of dependencies solely due to the fast pace evolution of Python every version. This post serves as a guide to anyone setting up their Jetson for ML experiments. The primary motivation is it took me about 20+ hours ( yes, I didn't get a good night/morning sleep ) to setup the device without libraries breaking. To prevent that from happening to you or anyone in the future, here's a small guide:

Hardware Setup

1. Ensure a stable power source
2. SD Card or external SSD a minimum 128 GB (recommended: anything above 256GB)
3.  Wired mouse + keyboard + monitor for the initial setup
4. Flash the Sd card / SSD with the image from the official website and set it up as per their instructions (not covered here).

Installation(important ones)

The estimated time for this section takes around 6 hours to complete.

1. Install Python 3.8+ - install your required/favourite version of Python(as of today, I'd recommend 3.8) - sudo apt-get install python3.8-dev [Estimated time: -5 mins]
2. Create a virtual environment from this newly installed python version: python3.8 -m venv venv (the second venv being the name of your virtual environment) Note: If venv is not installed, python3.8 -m pip install venv [Estimated time: 1-5 mins]
3. Install cmake, ninja, and some other important tools and libraries. sudo apt-get install cmake ninja libopenblas-dev libatlas-base-dev gfortran [Estimated time: 2-10 mins]
4. Install the required python libraries: pip install numpy pybind11 scipy scikit-learn scikit-image [Estimated time: 1-5 hours - Scipy is built from source]

Installation(optional)

Estimated time for this section takes around 7 hours to complete.

PyTorch: Building from source [Estimated time: 5 hours]

TorchVision: Building from source [Estimated time: 30 mins - 1 hour]

git clone https://github.com/pytorch/vision && cd vision && pip install -e .

MMCV [ Estimated time: 30 mins]

Follow the instructions on their website

MMDet [Estimated time: 10 - 30 mins]

Follow the instructions on their website

Make sure you've tested your required methods through and through(a successful import doesn't necessarily mean a successful installation). Now, to the essential part: Memory management. My tasks involved heavy memory utilization, multiprocessing, etc. all of which requires some more intervention.

Memory

Shared memory

Increase shared memory capacity: sudo mount -o remount,size=8G /dev/shm  This allows slightly larger amount of data to be shared between processes when doing multiprocessing related tasks. Remember, this must be executed once every boot.

Swap memory

Disclaimer: the boot sequence initiates a swap memory test on reboot which always fails on my Jetson, but the boot from power off works fine. Proceed with caution

1. sudo fallocate -l 8.0G /swapfile
2. sudo chmod 600 /swapfile
3. sudo mkswap /swapfile
4. sudo swapon /swapfile
5. sudo nano /etc/fstab
6. sudo <your_fav_editor> /etc/fstab and add /swapfile none swap 0 0 at the end
7. Shutdown your Jetson and boot it up.

This enables handling large datasets and models with ease when compared to the default configuration. An additional caution when using the dGPU, the memory is shared with the CPU, so the memory optimizations there must purely be code-based(sharding, lazy loading, etc.)

General tips

1. Use Vim/Emacs. Avoid VSCode as much as possible
2. Use SSH. Avoid using a monitor as much as possible
3. Docker comes pre-installed, but use it only when the load is not a lot.
4. Keep backing up files/results/etc.