Best practices

• Maximum memory/cpu: Each node has 1024G memories and 96 CPUs with 8 GPUs, so one should allocate no more than 128G of memory and no more than 12 CPUs per GPU
• Interactive jobs: Please do not use more than 2 concurrent interactive jobs and limit them to 2 hours
• Avoiding slow connections: To avoid having your GPUs scatter across different nodes (and not using the fast connection between GPUs on the same node), add the -N 1 flag to the SLURM submission.
• Monitoring efficiencies: You can use reportseff (link) to get a nice table of the GPU and GPU memory efficiencies of your jobs like so: pip install pipx && pipx run reportseff -g -u $USER.
  • Alternatively, you can set --mail-type=end and --mail-user=<YourNetID>@princeton.edu and you will receive a summary of your resource usage at the end of your job.
  • Jobs that do not make efficient use of the allocated GPUs might get killed (check your emails if you suspect this was the case).

Getting started

To request compute, fill out this form.

If you are approved, make sure you submit jobs with the correct partition and account.

To find your account information:

sacctmgr show assoc where User=<netid> format=Account,QoS,User,Partition

Your partition can be found by looking at the corresponding partition of your qos in the cluster setup section.

When submitting a job on slurm, specify your partition with -Pand account with -A.

Introduction to the cluster

• Della tutorial ← make sure you read this first!!
• Princeton Research Computing Knowledge Base on SLURM ← advanced knowledge about SLURM!!
• Useful tips for working on Della ("Removing Tedium", such as getting rid of multiple Duo prompts)

The PLI GPUs on Della

In most cases, you will submit jobs from either the pli, pli-lc or the pli-c partition (not to be confused with the Linux user groups).