The current state of Jetson devices through NVidia's JetPack is based on Python 3.6 ( including all the pre-built wheels and docker containers ). Python 3.6 is increasingly breaking a lot of dependencies solely due to the fast pace evolution of Python every version. This post serves as a guide to anyone setting up their Jetson for ML experiments. The primary motivation is it took me about 20+ hours ( yes, I didn't get a good night/morning sleep ) to setup the device without libraries breaking. To prevent that from happening to you or anyone in the future, here's a small guide:

Hardware Setup

Ensure a stable power source
SD Card or external SSD a minimum 128 GB (recommended: anything above 256GB)
Wired mouse + keyboard + monitor for the initial setup
Flash the Sd card / SSD with the image from the official website and set it up as per their instructions (not covered here).

Installation(important ones)

The estimated time for this section takes around 6 hours to complete.

Install Python 3.8+ - install your required/favourite version of Python(as of today, I'd recommend 3.8) - sudo apt-get install python3.8-dev [Estimated time: -5 mins]
Create a virtual environment from this newly installed python version: python3.8 -m venv venv (the second venv being the name of your virtual environment) Note: If venv is not installed, python3.8 -m pip install venv [Estimated time: 1-5 mins]
Install cmake, ninja, and some other important tools and libraries. sudo apt-get install cmake ninja libopenblas-dev libatlas-base-dev gfortran [Estimated time: 2-10 mins]
Install the required python libraries: pip install numpy pybind11 scipy scikit-learn scikit-image [Estimated time: 1-5 hours - Scipy is built from source]

Installation(optional)

Estimated time for this section takes around 7 hours to complete.

PyTorch: Building from source [Estimated time: 5 hours]

Instructions! (Click to expand) export USE_NCCL=0
export USE_DISTRIBUTED=0
export USE_QNNPACK=0
export USE_PYTORCH_QNNPACK=0
export TORCH_CUDA_ARCH_LIST="5.3;6.2;7.2"
export PYTORCH_BUILD_VERSION=1.8.0
export PYTORCH_BUILD_NUMBER=1
git clone --recursive --branch <your_version> https://github.com/pytorch/pytorch && cd pytorch
python setup.py bdist_wheel
pip install dist/torch......whl # the only file in the dist folder

TorchVision: Building from source [Estimated time: 30 mins - 1 hour]

git clone https://github.com/pytorch/vision && cd vision && pip install -e .

MMCV [ Estimated time: 30 mins]

Follow the instructions on their website

MMDet [Estimated time: 10 - 30 mins]

Follow the instructions on their website

Make sure you've tested your required methods through and through(a successful import doesn't necessarily mean a successful installation). Now, to the essential part: Memory management. My tasks involved heavy memory utilization, multiprocessing, etc. all of which requires some more intervention.

Memory

Shared memory

Increase shared memory capacity: sudo mount -o remount,size=8G /dev/shm This allows slightly larger amount of data to be shared between processes when doing multiprocessing related tasks. Remember, this must be executed once every boot.

Swap memory

Disclaimer: the boot sequence initiates a swap memory test on reboot which always fails on my Jetson, but the boot from power off works fine. Proceed with caution

sudo fallocate -l 8.0G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
sudo nano /etc/fstab
sudo <your_fav_editor> /etc/fstab and add /swapfile none swap 0 0 at the end
Shutdown your Jetson and boot it up.

This enables handling large datasets and models with ease when compared to the default configuration. An additional caution when using the dGPU, the memory is shared with the CPU, so the memory optimizations there must purely be code-based(sharding, lazy loading, etc.)

General tips

Use Vim/Emacs. Avoid VSCode as much as possible
Use SSH. Avoid using a monitor as much as possible
Docker comes pre-installed, but use it only when the load is not a lot.
Keep backing up files/results/etc.