Skip to content

Interactive Jobs

Allocation

salloc is used to allocate resources in real time to run an interactive batch job. Typically, this is used to allocate resources and spawn a shell. The shell is then used to execute srun commands to launch parallel tasks.

"interactive" QOS on Perlmutter

Perlmutter has a dedicated interactive QOS to support medium-length interactive work. This QOS is intended to deliver nodes for interactive use within 6 minutes of the job request.

Warning

On Perlmutter, if you have not set a default account, salloc may fail with the following error message:

salloc: error: Job request does not match any supported policy.
salloc: error: Job submit/allocate failed: Unspecified error

Avoid frequent short-lived interactive sessions

The interactive QOS is meant to be used interactively. If you need to submit many short jobs, use a single active salloc session rather than repeated independent salloc commands.

Perlmutter GPU nodes

The -C (constraint flag) must be set to GPU for any GPU-enabled interactive jobs (-C gpu or --constraint gpu). Users can initiate an interactive session with access to GPU nodes by running a command like the following:

salloc --nodes 1 --qos interactive --time 01:00:00 --constraint gpu --gpus 4 --account mxxxx

When using srun in a salloc session, you must also explicitly request GPU resources. Use the --gpus (-G), --gpus-per-node, or --gpus-per-task flag to make the allocated node's GPUs visible to your srun command.

Use the shared_interactive QOS if you only need 1 or 2 GPUs by setting
--qos shared_interactive in your salloc command.

Otherwise, you will see errors / complaints similar to the following:

no CUDA-capable device is detected

No Cuda device found

Perlmutter CPU nodes

Users can initiate an interactive session with access to CPU-only nodes by issuing a command like the following:

salloc --nodes 1 --qos interactive --time 01:00:00 --constraint cpu --account mxxxx

Limits

If resources are not readily available for the requested interactive job, it is automatically canceled after 6 minutes. To allow a job to wait for resources for a longer time, one can use the optional --immediate flag to specify the number of seconds that the job should wait for available resources:

# wait for up to 10 minutes
salloc --nodes 1 --qos interactive --time 01:00:00 --constraint <node type> --account mxxxx --immediate=600

There is a maximum node limit of 4 nodes for interactive jobs on both cpu and gpu partitions. For more details see QOS Limits and Charges.