How to Use t-SNE Effectively

How to Use t-SNE Effectively

https://distill.pub/2016/misread-tsne/

1. Hyperparameter

- perplexity values in the range (5 - 50) suggested

- iterate until reaching a stable configuration.

2. Cluster sizes in a t-SNE plot mean nothing

- expands dense clusters, and contracts sparse ones, evening out cluster sizes

3. Distances between clusters might not mean anything

- may not be one perplexity value that captures distances across all clusters

- perplexity is a global parameter.

4. Random noise doesn’t always look random.

- need to do in various perplexity values

5. You can see some shapes, sometimes 

- need to do in various perplexity values

6. For topology, you may need more than one plot

- need to do in various perplexity values

PlantSeg : tool for cell instance aware segmentation in densely packed 3D volumetric images.

 https://github.com/hci-unihd/plant-seg

 

Install PlantSeg

conda create -n plant-seg -c pytorch -c conda-forge cudatoolkit=10.1 -c lcerrone -c abailoni -c cpape -c awolny pytorch nifty=1.0.9 plantseg

To install pytorch for a certain cudatoolkit version

conda install pytorch  cudatoolkit=10.1 -c pytorch
plantseg --gui

To designate a certain cuda GPU device when run plant-seq with

CUDA_VISIBLE_DEVICES=0 plantseg --gui

To check cuda is available in pytorch

import torch
torch.cuda.is_available()

To check GPU usage

nvidia-smi -l 1 

To check CUDA version

nvcc -V

 

How To Use GPU with PyTorch

https://wandb.ai/wandb/common-ml-errors/reports/How-To-Use-GPU-with-PyTorch---VmlldzozMzAxMDk

 

How to run python code from Terminal in multiple sessions with multiple GPUs

1. set CUDA device

$ CUDA_VISIBLE_DEVICES=0 python test1.py  # Uses GPU 0.

$ CUDA_VISIBLE_DEVICES=1 python test2.py  # Uses GPU 1.

$ CUDA_VISIBLE_DEVICES=2,3 python test3.py  # Uses GPUs 2 and 3.

 or

2. add in python code

import os

os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"   

os.environ["CUDA_VISIBLE_DEVICES"]="0"

 

 ref:

https://stackoverflow.com/questions/34775522/tensorflow-multiple-sessions-with-multiple-gpus

https://stackoverflow.com/questions/37893755/tensorflow-set-cuda-visible-devices-within-jupyter