Perlmutter is a HPE (Hewlett Packard Enterprise) Cray EX supercomputer, named in honor of Saul Perlmutter, an astrophysicist at Berkeley Lab who shared the 2011 Nobel Prize in Physics for his contributions to research showing that the expansion of the universe is accelerating. Dr. Perlmutter has been a NERSC user for many years, and part of his Nobel Prize-winning work was carried out on NERSC machines and the system name reflects and highlights NERSC's commitment to advancing scientific research.
Perlmutter, based on the HPE Cray Shasta platform, is a heterogeneous system comprising both CPU-only and GPU-accelerated nodes, with a performance of 3-4 times Cori when the installation completes. The system is scheduled to be delivered in two phases: Phase 1, with 12 GPU-accelerated cabinets housing over 1,500 nodes, and 35PB of all-flash storage, was delivered by early 2021, and Phase 2 with 12 CPU cabinets will be delivered later in 2021.
System Overview - Phase 1¶
|System Partition||# of cabinets||# of nodes||CPU Aggregate Theoretical Peak (FP64 in PFlops)||CPU Aggregate Memory (TiB)||GPU Aggregate Theoretical Peak (PFlops)||GPU Aggregate Memory (TiB)|
|GPU-accelerated compute nodes||12||1,536||3.9||384||FP64: 59.9 |
TF32 Tensor: 958.1
|Login Nodes||-||15||0.07||7.5||FP64: 0.1 |
TF32 Tensor: 2.3
System Specification - Phase 1¶
|System Partition||Processor||Clock Rate (MHz)||Cores per Socket||Threads/Core||Sockets per Node||Memory per Node (GiB)||Local Scratch per Node (GB)|
|GPU-accelerated compute nodes||AMD EPYC 7763 (Milan)||2450||64||2||1||256||-|
|Login Nodes||AMD EPYC 7742 (Rome)||2250||64||2||2||512||960|
|System Partition||Processor||Clock Rate (MHz)||SMs per GPU||INT32, FP32, FP64, Tensor cores per GPU||GPUs per Node||Memory per Node (GiB)|
|GPU-accelerated compute nodes||NVIDIA A100 GPU||1410||108||6912, 6912, 3456, 432||4||160|
|Login Nodes||NVIDIA A100 GPU||1410||108||6912, 6912, 3456, 432||1||40|
Detailed System Information¶
Please check here for detailed system info.
Perlmutter is not yet available for general user access. Once users have access, they can ssh to
saul.nersc.gov to log on to Perlmutter.
Perlmutter will be available to users in several stages, in the following order:
- The NESAP (NERSC Exascale Science Applications Program) tier 1 and ECP (Exascale Computing Project) teams
- The NESAP tier 2 and Superfacility teams
- Selected general users running GPU applications
- Remaining general users running GPU applications
- Remaining users
Preparing for Perlmutter¶
Please check the Transitioning Applications to Perlmutter webpage for a wealth of useful information on how to transition your applications for Perlmutter.
Perlmutter uses Slurm for batch job scheduling. Below you can find info on the queue policies, how to submit jobs using Slurm and monitor jobs, etc.:
- Queue Policies on Perlmutter
- Running Jobs on Perlmutter's GPU nodes
- Monitoring Jobs
- Interactive Jobs
During Allocation Year 2021 jobs run on Perlmutter will be free of charge.
Current Known Issues¶
For known system issues, check here.