Skip to content

Finding and using software on Perlmutter

The modules environment

Perlmutter's modules environment is similar to Cori's but using Lmod in place of the (earlier) TCL modules used on Cori1.

Lmod works in the same way as the TCL environment modules familiar to NERSC users, and provides the same user interface and commands - module list, module avail, module load etc all do the same as they do on Cori. Lmod also can use the same TCL-based modulefiles as are used on Cori (though its native and preferred modulefile language is Lua).

Lmod adds some capabilities over TCL modules, and these lead to some differences in behavior:

Module hierarchies

Lmod supports a module hierarchy, where sets of modules with a common dependency are "under" the required module, and do not appear in the output of module avail. Lmod provides another command, module spider, which also shows modules that can be made available, by loading the dependency module first.

This is visible particularly in the Cray Programming Environment (NERSC-deployed modulefiles currently use a flat structure). For example, the Cray Programming Environment provides netcdf, which uses HDF5 for lower-level data access. If HDF5 is built with support for parallel I/O, then netcdf can also support parallel I/O. So Perlmutter has two modules for netcdf: one dependent on cray-hdf5 and one on cray-hdf5-parallel. In your initial environment you see neither:

perlmutter> module avail cray-netcdf
No module(s) or extension(s) found!
Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".

Using module spider:

perlmutter> module --terse spider cray-netcdf
cray-netcdf/4.7.4.3
perlmutter> module spider cray-netcdf/4.7.4.3

---------------------------------------------------------------------------------------------------------------------------------------------------------
  cray-netcdf: cray-netcdf/4.7.4.3
---------------------------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "cray-netcdf/4.7.4.3" module is available to load.

      aocc/2.2.0.1  cray-hdf5/1.12.0.3
      cce/11.0.4  cray-hdf5/1.12.0.3
      gcc/10.2.0  cray-hdf5/1.12.0.3
      gcc/9.3.0  cray-hdf5/1.12.0.3

Module families

Lmod has a concept of a "family", which can be thought of as a "niche": only one thing can occupy it at a time. As an example, the craype-hugepages* modules all belong to the "craype_hugepages" family. If you have craype-hugepages2M and run module load craype-hugepages16M, then craype-hugepages2M will be automatically unloaded first.

The ml command

Lmod provides an extra command called ml. ml can be used like a short-form of module for most module subcommands, and with no arguments will perform module list.

The Perlmutter software stack

Warning

This is a tentative estimate of software we anticipate will be GPU-ready within the early-access period for Perlmutter. Availability of software may be delayed and our ability to support these packages may be limited.

GPU-ready applications

Please check the Popular applications section for information on the software available or to be available on Perlmutter.

Libraries

Note

Notes for using these libraries with GPUs are still under development.

  • MPI / mpi4py (CPU/GPU)
  • GASNet (CPU)
  • BLAS / LAPACK / ScaLAPACK (CPU/GPU)
  • FFTW/cuFFT (CPU/GPU)
  • HDF5 / h5py (CPU)
  • NetCDF (CPU)
  • Spark (CPU/GPU)
  • R (CPU/GPU)
  • Numpy (CPU/GPU)
  • pyCUDA (CPU/GPU)
  • Scipy (CPU/GPU)
  • Scikit-Learn (CPU/GPU)
  • Numba (CPU/GPU)
  • TensorFlow (CPU/GPU)
  • Keras (CPU/GPU)
  • Caffe (CPU/GPU)
  • pytorch (CPU/GPU)
  • Hypre (CPU/GPU)
  • SuperLU-dist (CPU/GPU)
  • Sundials (CPU/GPU)
  • SLATE (CPU/GPU)
  • MAGMA (CPU/GPU)

Programming models

  • MPI
  • OpenMP
  • OpenACC
  • Kokkos
  • Raja
  • UPC

Tools

Tools supporting GPU development:

Note

Notes for using these tools with GPUs are still under development.

CPU-based tools:


  1. Cori may evenutally transition to also use Lmod, however as such a change will be disruptive to the user environment it will not be made with short notice, and NERSC is not planning this yet.