Reservations¶

Users can request a scheduled reservation of machine resources if their jobs have special needs that cannot be accommodated through the regular batch system. A reservation brings some portion of the machine to a specific user or project for an agreed upon duration. Typically this is used for interactive debugging at scale or real time processing linked to some experiment or event.

Charging¶

For normal batch jobs, charging against a project's allocation is done on a per job basis. For scheduled reservations the entire block of reserved time is charged regardless of the number of nodes used or time spent running jobs.

Requesting a reservation¶

To reserve compute nodes, a request must be sent in with at least 1 week notice. Please ask for the least amount of resources you need and try to schedule reservations so as to minimize impact on other users. It is also recommended that reservations be scheduled to start during NERSC business hours to ensure availability of staff in case any issues arise.

Reservation Request Form

Cancellations¶

Cancellation of a reservation must be done with a minimum of 4 days notice. If you do not receive a confirmation that your cancellation was received and it is less than 4 days until your start time you must contact NERSC operations via 1-800-666-3772 (or 1-510-486-8600) menu option 1 to confirm.

Viewing reservations¶

To view all reservations run scontrol show reservations, the output will consist of one entry per reservation name that can be used with option --reservation to access the reservation. Take a close look at the reservation fields such as StartTime EndTime, Duration, Nodes,Users, Accounts
to understand what the reservation.

$ scontrol show reservations
ReservationName=debug StartTime=2023-01-15T09:34:30 EndTime=2024-01-15T09:34:30 Duration=365-00:00:00
   Nodes=nid[001080-001081,003220-003221,003764-003765,004196-004199,005284-005287,005368-005371,006184-006187,006296,006520-006523,006792-006795,006852-006855,006920-006923,006996-006999] NodeCnt=43 CoreCnt=5120 Features=(null) PartitionName=(null) Flags=MAINT,OVERLAP,IGNORE_JOBS,SPEC_NODES
   TRES=cpu=10240
   Users=root Groups=(null) Accounts=(null) Licenses=(null) State=ACTIVE BurstBuffer=(null) Watts=n/a
   MaxStartDelay=(null)

ReservationName=intro_cpu StartTime=2023-06-08T14:00:00 EndTime=2023-06-08T15:30:00 Duration=01:30:00
   Nodes=nid[004174-004185,004187-004195,004200-004213] NodeCnt=35 CoreCnt=4480 Features=(null) PartitionName=regular_milan_ss11 Flags=
   TRES=cpu=8960
   Users=(null) Groups=(null) Accounts=ntrain3 Licenses=(null) State=INACTIVE BurstBuffer=(null) Watts=n/a
   MaxStartDelay=(null)

ReservationName=intro_gpu StartTime=2023-06-08T14:00:00 EndTime=2023-06-08T15:30:00 Duration=01:30:00
   Nodes=nid[001004,001008-001009,001012-001013,001016-001017,001020-001021,001028] NodeCnt=10 CoreCnt=640 Features=(null) PartitionName=gpu_ss11 Flags=
   TRES=cpu=1280
   Users=(null) Groups=(null) Accounts=ntrain3_g Licenses=(null) State=INACTIVE BurstBuffer=(null) Watts=n/a
   MaxStartDelay=(null)

In order to use a reservation, the system administrators will grant access to individual users or by accounts. To filter by reservation name you can do one of the following

scontrol show reservations=<ReservationName>
scontrol show reservations <ReservationName>

Shown below is a summary for reservation intro_cpu

$ scontrol show reservation=intro_cpu
ReservationName=intro_cpu StartTime=2023-06-08T14:00:00 EndTime=2023-06-08T15:30:00 Duration=01:30:00
   Nodes=nid[004174-004185,004187-004195,004200-004213] NodeCnt=35 CoreCnt=4480 Features=(null) PartitionName=regular_milan_ss11 Flags=
   TRES=cpu=8960
   Users=(null) Groups=(null) Accounts=ntrain3 Licenses=(null) State=INACTIVE BurstBuffer=(null) Watts=n/a
   MaxStartDelay=(null)

The intro_cpu reservation is accessible to all users belonging to the ntrain3 account because Accounts=ntrain3 is set even though no users are defined Users=(null). Notice STATE will show as ACTIVE (instead of INACTIVE above) during the reservation window.

If you notice an error in your reservation, please reach out to us in order to make a correction. Take note of all fields listed in the reservation to ensure we have allocated the resources that matches your request.

Using a reservation¶

Once your reservation request is approved and a reservation is placed on the system, to run jobs in the reservation, you can use the --reservation option on the command line:

nersc$ sbatch --reservation=<reservation_name>

nersc$ salloc --reservation=<reservation_name>

or add #SBATCH --reservation=<reservation_name> to your job script.

Ending a reservation¶

All running jobs under a reservation will be terminated when the reservation ends. There are two ways to end a reservation earlier than scheduled:

• When requesting the reservation, you can ask us to activate a setting that will terminate the reservation a few minutes after all jobs in the reservation queue have completed.

• If your reservation does not have the above setting and you complete all planned computations before the reservation ends, please call NERSC operations at 1-800-666-3772 (or 1-510-486-8600) menu option 1 to cancel the reservation.