NPE and PrgEnv-llvm¶
NPE or NERSC Programming Environment complements and extends the Cray Programming Environment provided by HPE.
NPE and PrgEnv-LLVM are currently Experimental
These are new efforts under active development. Long term support options are being planned and evaluated, but for now significant breaking changes may occur at any time.
Your feedback is welcome!
NPE is not yet part of the default set of modules and must be added via:
module use /global/cfs/cdirs/nstaff/cookbg/pe/modulefiles
after which NPE can be loaded
Additional versions of the
npe stack will be made available as additional
LLVM compilers with support for OpenMP offload and SYCL with support for MPI via MPICH.
If GPU support is desired then
ml cudatoolkit craype-accel-nvidia80 ml PrgEnv-llvm
mpicc to compile, not
flang) is not supported at this time.
intel-llvm is the default version and supports OpenMP offload and SYCL for A100. Nightly versions are also available (see
ml avail intel-llvm).
llvm is based on llvm.org released versions and supports OpenMP offload for A100.
mpich module is available for:
mpif90 should be used instead of
ftn in this release.
craype-accel-nvidia80 modules must be loaded before
mpich to enable CUDA aware MPI.
Flux is a flexible framework for resource management consisting of a suite of projects, tools, and libraries which may be used to build site-custom resource managers for HPC centers.
Reminder that everything in NPE is considered "experimental" NERSC's current workflow tool recommendations.
Start multi-node interactive job
salloc -C gpu -A <account> -N 2 --ntasks-per-node=1 --gpus-per-node 4 -q debug -t 30
module load flux-sched
srun --mpi=pmi2 --pty flux start
Verify that flux can see gpu resources
$ flux resource list STATE NNODES NCORES NGPUS NODELIST free 2 128 8 nid[001704-001705] allocated 0 0 0 down 0 0 0
Please report any issues, comments or feedback via the NERSC Help Desk.
fluxsupport added (
- Reframe test for flux
- also build
Initial "alpha" release includes
PrgEnv-llvm/0.1a new LLVM based
llvm/14.0.3from llvm.org llvm.org source with OpenMP offload support for A100.
intel-llvm/2021-12from Intel's fork of LLVM with OpenMP offload and SYCL support for A100.
intel-llvm/sycl-nightly/<date>Access to nightly builds of
ninja/1.10.2Ninja build tool
cmake/3.22.1CMake build system
reframe/3.10.1HPC testing framework