Skip to content

E4S 22.05 - Perlmutter

The e4s-22.05 stack is built using the Spack branch e4s-22.05 on Perlmutter using GCC and CCE compilers. This stack can be loaded by running:

module load e4s/22.05

You can also load the same software stack via module load spack/e4s-22.05

Note

Going forward, we do not generate modules for Spack. In order to use E4S software you must use spack load to load a package in your user environment.

Here is a breakdown of the install specs by compiler:

Compiler Root Specs Implicit Specs Total Specs
gcc@10.3.0 1 24 25
gcc@11.2.0 67 232 299
cce@13.0.2 8 44 52
Total 76 300 376

Spack Configuration

When you load the e4s/22.05 module you will be in a Spack environment named e4s, you can see your active environment by running the following:

elvis@perlmutter> spack env status
==> In environment e4s

The Spack configuration is accessible on the filesystem by changing directories into the root e4s environment by running the following:

elvis@perlmutter> spack cd -e e4s
elvis@perlmutter> pwd && ls -l
/global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/var/spack/environments/e4s
total 842
-rw-rw-r-- 1 e4s spackecp 851286 Nov 10 13:00 spack.lock
-rw-rw-r-- 1 e4s spackecp   9480 Nov 10 13:24 spack.yaml

The file spack.yaml contains the configuration used to build the production stack. It contains information on compilers, packages and their variations, and groupings of the packages. If you are interested in seeing the production spack configuration for this stack, please see https://github.com/NERSC/spack-infrastructure/blob/main/spack-configs/perlmutter-e4s-22.05/spack.yaml

Overview

You can see a list of all available packages via spack find. Shown below is breakdown of all packages with their variants.

Output of spack find
elvis@perlmutter> spack find --format "{name}@{version}%{compiler} {variants}"
adios2@2.8.0%cce@13.0.2 +blosc+bzip2~cuda~dataman~dataspaces+fortran~hdf5~ipo+mpi+pic+png~python+shared+ssc+sst+sz+zfp build_type=Release
adios2@2.8.0%gcc@11.2.0 +blosc+bzip2~cuda~dataman~dataspaces+fortran~hdf5~ipo+mpi+pic+png~python+shared+ssc+sst+sz+zfp build_type=Release
adios2@2.8.0%gcc@11.2.0 +blosc+bzip2+cuda~dataman~dataspaces+fortran~hdf5~ipo+mpi+pic+png~python+shared+ssc+sst+sz+zfp build_type=Release cuda_arch=80
amrex@22.05%gcc@11.2.0 ~amrdata~cuda~eb~fortran~hdf5~hypre~ipo+linear_solvers+mpi~openmp~particles~petsc~pic~plotfile_tools~rocm~shared~sundials~tiny_profile build_type=RelWithDebInfo dimensions=3 precision=double
antlr@2.7.7%gcc@11.2.0 +cxx~java~python patches=33897ad
apr@1.7.0%gcc@11.2.0  patches=a412848
apr-util@1.6.1%gcc@11.2.0 ~crypto~gdbm~odbc~pgsql~sqlite
arpack-ng@3.8.0%cce@13.0.2 +mpi+shared
arpack-ng@3.8.0%gcc@11.2.0 +mpi+shared
asio@1.21.0%gcc@11.2.0 ~boost_coroutine~boost_regex~separate_compilation cxxstd=17
autoconf@2.69%gcc@11.2.0  patches=35c4492,7793209,a49dd5b
autoconf-archive@2022.02.11%gcc@11.2.0
automake@1.16.5%gcc@11.2.0
bdftopcf@1.0.5%gcc@11.2.0
berkeley-db@18.1.40%gcc@11.2.0 +cxx~docs+stl patches=b231fcc
binutils@2.38%gcc@11.2.0 ~gas+gold+headers~interwork+ld+libiberty~lto~nls+plugins libs=shared,static
bison@3.8.2%gcc@11.2.0
blaspp@2021.04.01%gcc@11.2.0 ~cuda~ipo+openmp~rocm+shared build_type=RelWithDebInfo
blaspp@2021.04.01%gcc@11.2.0 +cuda~ipo+openmp~rocm+shared build_type=RelWithDebInfo cuda_arch=none
blt@0.4.1%gcc@11.2.0
boost@1.79.0%gcc@11.2.0 +atomic+chrono~clanglibcpp~container~context~contract~coroutine+date_time~debug+exception~fiber+filesystem+graph~graph_parallel~icu+iostreams~json+locale+log+math~mpi+multithreaded~nowide~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded~stacktrace+system~taggedlayout+test+thread+timer~type_erasure~versionedlayout+wave cxxstd=17 patches=57a8401,a440f96 visibility=hidden
boost@1.79.0%gcc@11.2.0 +atomic+chrono~clanglibcpp~container~context~contract~coroutine+date_time~debug~exception~fiber+filesystem~graph~graph_parallel~icu~iostreams~json~locale~log~math~mpi+multithreaded~nowide~numpy~pic~program_options~python~random~regex~serialization+shared~signals~singlethreaded~stacktrace+system~taggedlayout~test+thread+timer~type_erasure~versionedlayout~wave cxxstd=98 patches=57a8401,a440f96 visibility=hidden
boost@1.79.0%gcc@11.2.0 +atomic+chrono~clanglibcpp~container~context~contract~coroutine+date_time~debug~exception~fiber+filesystem+graph~graph_parallel~icu~iostreams~json~locale~log~math~mpi+multithreaded~nowide~numpy~pic~program_options~python~random+regex~serialization+shared~signals~singlethreaded~stacktrace+system~taggedlayout~test+thread+timer~type_erasure~versionedlayout~wave cxxstd=98 patches=57a8401,a440f96 visibility=global
boost@1.79.0%gcc@11.2.0 +atomic+chrono~clanglibcpp~container~context~contract~coroutine+date_time~debug+exception~fiber+filesystem+graph~graph_parallel~icu+iostreams~json+locale+log+math~mpi+multithreaded~nowide~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded~stacktrace+system~taggedlayout+test+thread+timer~type_erasure~versionedlayout+wave cxxstd=98 patches=57a8401,a440f96 visibility=hidden
butterflypack@2.1.1%gcc@11.2.0 ~ipo+shared build_type=RelWithDebInfo
bzip2@1.0.6%cce@13.0.2 ~debug~pic+shared
bzip2@1.0.6%gcc@10.3.0 ~debug~pic+shared
bzip2@1.0.6%gcc@11.2.0 ~debug~pic+shared
c-blosc@1.21.1%cce@13.0.2 +avx2~ipo build_type=RelWithDebInfo
c-blosc@1.21.1%gcc@11.2.0 +avx2~ipo build_type=RelWithDebInfo
cairo@1.16.0%gcc@11.2.0 ~X+fc+ft+gobject+pdf~png~svg patches=7c4da77
cairo@1.16.0%gcc@11.2.0 +X~fc+ft~gobject+pdf~png~svg patches=7c4da77
cairo@1.16.0%gcc@11.2.0 +X+fc+ft~gobject+pdf~png~svg patches=7c4da77
camp@0.2.2%gcc@11.2.0 ~cuda~ipo~rocm~tests build_type=RelWithDebInfo
camp@0.2.2%gcc@11.2.0 +cuda~ipo~rocm~tests build_type=RelWithDebInfo cuda_arch=80
ccache@4.5.1%gcc@11.2.0 ~ipo build_type=RelWithDebInfo
cdo@2.0.5%gcc@11.2.0 ~curl~external-grib1+fftw3+hdf5+libxml2~magics+netcdf+openmp+proj+szip+udunits2 grib2=eccodes
cmake@3.23.1%cce@13.0.2 ~doc+ncurses~ownlibs~qt build_type=Release
cmake@3.23.1%cce@13.0.2 ~doc+ncurses~ownlibs~qt build_type=Release
cmake@3.23.1%gcc@11.2.0 ~doc+ncurses~ownlibs~qt build_type=Release
cmake@3.23.1%gcc@11.2.0 ~doc+ncurses~ownlibs~qt build_type=Release
conduit@0.8.3%gcc@11.2.0 ~adios+blt_find_mpi~doc~doxygen+fortran+hdf5+hdf5_compat~ipo+mpi+parmetis~python+shared~silo+test~zfp build_type=RelWithDebInfo
cpio@2.13%gcc@11.2.0
cray-libsci@21.08.1.2%cce@13.0.2 ~mpi~openmp+shared
cray-libsci@21.08.1.2%gcc@11.2.0 +mpi~openmp+shared
cray-mpich@8.1.15%cce@13.0.2 +wrappers
cray-mpich@8.1.15%gcc@11.2.0 +wrappers
cub@1.13.1%gcc@11.2.0
cuda@11.5.0%gcc@11.2.0 ~allow-unsupported-compilers~dev
curl@7.66.0%cce@13.0.2 ~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 libs=shared,static tls=openssl
curl@7.66.0%gcc@11.2.0 ~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 libs=shared,static tls=openssl
diffutils@3.6%cce@13.0.2
diffutils@3.6%gcc@11.2.0
dyninst@12.1.0%gcc@11.2.0 ~ipo+openmp~stat_dysect~static build_type=RelWithDebInfo
dyninst@12.1.0%gcc@11.2.0 ~ipo+openmp~stat_dysect~static build_type=RelWithDebInfo
eccodes@2.25.0%gcc@11.2.0 ~aec~fortran~ipo~memfs~netcdf~openmp~png~pthreads~python+shared~tools build_type=RelWithDebInfo definitions=auto jp2k=openjpeg samples=auto
elfutils@0.186%gcc@11.2.0 +bzip2~debuginfod~nls~xz
elfutils@0.186%gcc@11.2.0 +bzip2~debuginfod~nls+xz
esmf@8.2.0%gcc@11.2.0 ~debug~external-lapack+mpi+netcdf+pio+pnetcdf+xerces
expat@2.4.8%cce@13.0.2 +libbsd
expat@2.4.8%gcc@10.3.0 +libbsd
expat@2.4.8%gcc@11.2.0 +libbsd
ffmpeg@4.4.1%gcc@11.2.0 ~X~avresample+bzlib~drawtext+gpl~libaom~libmp3lame~libopenjpeg~libopus~libsnappy~libspeex~libssh~libvorbis~libvpx~libwebp~libx264~libzmq~lzma~nonfree~openssl~sdl2+shared+version3
fftw@3.3.10%gcc@11.2.0 +mpi~openmp~pfft_patches precision=double,float
fftw@3.3.10%gcc@11.2.0 +mpi+openmp~pfft_patches precision=double,float
findutils@4.6.0%gcc@11.2.0
flex@2.6.3%gcc@11.2.0 +lex~nls
font-util@1.3.2%gcc@11.2.0  fonts=encodings,font-adobe-100dpi,font-adobe-75dpi,font-adobe-utopia-100dpi,font-adobe-utopia-75dpi,font-adobe-utopia-type1,font-alias,font-arabic-misc,font-bh-100dpi,font-bh-75dpi,font-bh-lucidatypewriter-100dpi,font-bh-lucidatypewriter-75dpi,font-bh-type1,font-bitstream-100dpi,font-bitstream-75dpi,font-bitstream-speedo,font-bitstream-type1,font-cronyx-cyrillic,font-cursor-misc,font-daewoo-misc,font-dec-misc,font-ibm-type1,font-isas-misc,font-jis-misc,font-micro-misc,font-misc-cyrillic,font-misc-ethiopic,font-misc-meltho,font-misc-misc,font-mutt-misc,font-schumacher-misc,font-screen-cyrillic,font-sun-misc,font-winitzki-cyrillic,font-xfree86-type1
fontconfig@2.13.1%gcc@11.2.0
fontconfig@2.13.94%gcc@11.2.0
fontconfig@2.13.94%gcc@11.2.0
fontsproto@2.1.3%gcc@11.2.0
fortrilinos@2.0.0%gcc@11.2.0 +hl~ipo+shared build_type=RelWithDebInfo
freetype@2.11.1%gcc@11.2.0
fribidi@1.0.12%gcc@11.2.0
gasnet@2022.3.0%gcc@11.2.0 ~cuda~debug~rocm conduits=smp
gawk@5.1.1%gcc@11.2.0 ~nls
gdbm@1.19%gcc@11.2.0
gdbm@1.23%cce@13.0.2
gdbm@1.23%gcc@10.3.0
gdbm@1.23%gcc@11.2.0
gettext@0.21%cce@13.0.2 +bzip2+curses+git~libunistring+libxml2+tar+xz
gettext@0.21%gcc@10.3.0 +bzip2+curses+git~libunistring+libxml2+tar+xz
gettext@0.21%gcc@11.2.0 +bzip2+curses+git~libunistring+libxml2+tar+xz
gettext@0.21%gcc@11.2.0 +bzip2+curses+git~libunistring+libxml2+tar+xz
git@2.26.2%gcc@11.2.0 +man+nls+perl+subtree~svn~tcltk
glib@2.72.1%gcc@11.2.0 ~libmount patches=b3fd450 tracing=none
glib@2.72.1%gcc@11.2.0 ~libmount patches=b3fd450 tracing=none
glib@2.72.1%gcc@11.2.0 ~libmount patches=b3fd450 tracing=none
gmake@4.3%gcc@11.2.0 ~guile+nls
gmp@6.2.1%gcc@11.2.0  libs=shared,static
gnuplot@5.4.3%gcc@11.2.0 +X+cairo+gd+libcerf~pbm~qt~wx patches=ad89f23
gobject-introspection@1.56.1%gcc@11.2.0  patches=6f90bb2,7700828
googletest@1.10.0%gcc@11.2.0 ~gmock~ipo+pthreads+shared build_type=RelWithDebInfo
gperf@3.1%gcc@11.2.0
gperftools@2.9.1%gcc@11.2.0 +debugalloc~dynamic_sized_delete_support+libunwind~sized_delete
grads@2.2.1%gcc@11.2.0 +geotiff+shapefile
gromacs@2021.5%gcc@11.2.0 ~blas~cuda~cycle_subcounters~double+hwloc~ipo~lapack~mdrun_only+mpi~nosuffix~opencl+openmp~plumed~relaxed_double_precision+shared~sycl build_type=RelWithDebInfo
gsl@2.7%gcc@11.2.0 ~external-cblas
gsl@2.7.1%gcc@11.2.0 ~external-cblas
harfbuzz@4.2.1%gcc@11.2.0 ~graphite2~strip buildtype=debugoptimized default_library=shared
hdf5@1.8.22%gcc@11.2.0 ~cxx+fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=v18 build_type=RelWithDebInfo
hdf5@1.10.7%cce@13.0.2 ~cxx+fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=v18 build_type=RelWithDebInfo patches=2a1e311
hdf5@1.10.7%gcc@11.2.0 ~cxx+fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=v18 build_type=RelWithDebInfo patches=2a1e311
hdf5@1.10.8%gcc@11.2.0 ~cxx+fortran+hl~ipo~java+mpi+shared+szip~threadsafe+tools api=v18 build_type=RelWithDebInfo patches=ee351eb
hdf5@1.12.1%cce@13.0.2 ~cxx+fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=v18 build_type=RelWithDebInfo patches=ee351eb
hdf5@1.12.1%gcc@11.2.0 ~cxx+fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=v18 build_type=RelWithDebInfo patches=ee351eb
hdf5@1.12.1%gcc@11.2.0 ~cxx+fortran+hl~ipo~java+mpi+shared~szip+threadsafe+tools api=v18 build_type=RelWithDebInfo patches=ee351eb
heffte@2.2.0%gcc@11.2.0 ~cuda+fftw~fortran~ipo~magma~mkl~python~rocm+shared build_type=RelWithDebInfo
hiredis@0.14.1%gcc@11.2.0
hpctoolkit@2022.04.15%gcc@11.2.0 ~all-static~cray~cuda~debug~level_zero~mpi+papi~rocm+viewer
hpctoolkit@2022.04.15%gcc@11.2.0 ~all-static~cray+cuda~debug~level_zero+mpi+papi~rocm+viewer
hpcviewer@2022.03%gcc@11.2.0
hpx@1.7.1%gcc@11.2.0 ~async_cuda~async_mpi~cuda~examples~generic_coroutines~ipo~rocm~tools build_type=RelWithDebInfo cxxstd=17 instrumentation=none malloc=tcmalloc max_cpu_count=64 networking=mpi
hwloc@2.7.1%gcc@11.2.0 ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared
hypre@2.24.0%cce@13.0.2 ~complex~cuda~debug+fortran~gptune~int64~internal-superlu~mixedint+mpi+openmp~rocm+shared+superlu-dist~unified-memory
hypre@2.24.0%gcc@11.2.0 ~complex~cuda~debug+fortran~gptune~int64~internal-superlu~mixedint+mpi+openmp~rocm+shared+superlu-dist~unified-memory
hypre@2.24.0%gcc@11.2.0 ~complex~cuda~debug+fortran~gptune~int64~internal-superlu~mixedint+mpi+openmp~rocm+shared+superlu-dist~unified-memory
hypre@2.24.0%gcc@11.2.0 ~complex+cuda~debug+fortran~gptune~int64~internal-superlu~mixedint+mpi+openmp~rocm+shared+superlu-dist~unified-memory cuda_arch=80
icu4c@67.1%gcc@11.2.0  cxxstd=11
inputproto@2.3.2%gcc@11.2.0
intel-mkl@2020.4.304%gcc@11.2.0 ~ilp64+shared threads=none
intel-tbb@2020.3%gcc@11.2.0 ~ipo+shared+tm build_type=RelWithDebInfo cxxstd=default patches=62ba015,ce1fb16,d62cb66
intel-tbb@2020.3%gcc@11.2.0 ~ipo+shared+tm build_type=RelWithDebInfo cxxstd=default patches=62ba015,ce1fb16,d62cb66
intel-xed@2022.04.17%gcc@11.2.0 ~debug~pic
json-glib@1.6.6%gcc@11.2.0 ~strip buildtype=debugoptimized default_library=shared
kbproto@1.0.7%gcc@11.2.0
kim-api@2.2.1%gcc@11.2.0 ~ipo build_type=RelWithDebInfo
kokkos@3.6.00%cce@13.0.2 ~aggressive_vectorization~compiler_warnings~cuda~cuda_constexpr~cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl+openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning~wrapper build_type=RelWithDebInfo std=14
kokkos@3.6.00%gcc@11.2.0 ~aggressive_vectorization~compiler_warnings~cuda~cuda_constexpr~cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl+openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning~wrapper build_type=RelWithDebInfo std=14
kokkos@3.6.00%gcc@11.2.0 ~aggressive_vectorization~compiler_warnings+cuda~cuda_constexpr~cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning+wrapper build_type=RelWithDebInfo cuda_arch=80 std=14
kokkos-kernels@3.6.00%cce@13.0.2 ~blas~cblas~cublas~cuda~cusparse~ipo~lapack~lapacke~mkl+openmp~pthread~serial+shared~superlu build_type=RelWithDebInfo execspace_cuda=auto execspace_openmp=auto execspace_serial=auto execspace_threads=auto layouts=left memspace_cudaspace=auto memspace_cudauvmspace=auto offsets=int,size_t ordinals=int scalars=double
kokkos-kernels@3.6.00%gcc@11.2.0 ~blas~cblas~cublas~cuda~cusparse~ipo~lapack~lapacke~mkl+openmp~pthread~serial+shared~superlu build_type=RelWithDebInfo execspace_cuda=auto execspace_openmp=auto execspace_serial=auto execspace_threads=auto layouts=left memspace_cudaspace=auto memspace_cudauvmspace=auto offsets=int,size_t ordinals=int scalars=double
kokkos-kernels@3.6.00%gcc@11.2.0 ~blas~cblas~cublas+cuda~cusparse~ipo~lapack~lapacke~mkl~openmp~pthread~serial+shared~superlu build_type=RelWithDebInfo cuda_arch=80 execspace_cuda=auto execspace_openmp=auto execspace_serial=auto execspace_threads=auto layouts=left memspace_cudaspace=auto memspace_cudauvmspace=auto offsets=int,size_t ordinals=int scalars=double
kokkos-nvcc-wrapper@3.2.00%gcc@11.2.0 +mpi
lammps@20220107%gcc@11.2.0 ~asphere~body~class2~colloid~compress~coreshell~cuda~cuda_mps~dipole~exceptions+ffmpeg~granular~ipo+jpeg+kim~kokkos~kspace~latte+lib~manybody~mc~meam~misc~mliap~molecule+mpi~mpiio~opencl+openmp~opt~peri+png~poems~python~qeq~replica~rigid~shock~snap~spin~srd~user-adios~user-atc~user-awpmd~user-bocs~user-cgsdk~user-colvars~user-diffraction~user-dpd~user-drude~user-eff~user-fep~user-h5md~user-lb~user-manifold~user-meamc~user-mesodpd~user-mesont~user-mgpt~user-misc~user-mofff~user-netcdf~user-omp~user-phonon~user-plumed~user-ptm~user-qtb~user-reaction~user-reaxc~user-sdpd~user-smd~user-smtbq~user-sph~user-tally~user-uef~user-yaff~voronoi build_type=RelWithDebInfo
lammps@20220107%gcc@11.2.0 ~asphere~body~class2~colloid~compress~coreshell+cuda+cuda_mps~dipole~exceptions+ffmpeg~granular~ipo+jpeg+kim~kokkos~kspace~latte+lib~manybody~mc~meam~misc~mliap~molecule+mpi~mpiio~opencl+openmp~opt~peri+png~poems~python~qeq~replica~rigid~shock~snap~spin~srd~user-adios~user-atc~user-awpmd~user-bocs~user-cgsdk~user-colvars~user-diffraction~user-dpd~user-drude~user-eff~user-fep~user-h5md~user-lb~user-manifold~user-meamc~user-mesodpd~user-mesont~user-mgpt~user-misc~user-mofff~user-netcdf~user-omp~user-phonon~user-plumed~user-ptm~user-qtb~user-reaction~user-reaxc~user-sdpd~user-smd~user-smtbq~user-sph~user-tally~user-uef~user-yaff~voronoi build_type=RelWithDebInfo cuda_arch=80
lapackpp@2021.04.00%gcc@11.2.0 ~ipo+shared build_type=RelWithDebInfo
lapackpp@2021.04.00%gcc@11.2.0 ~ipo+shared build_type=RelWithDebInfo
libaec@1.0.5%gcc@11.2.0 ~ipo+shared build_type=RelWithDebInfo
libarchive@3.5.2%cce@13.0.2 +iconv compression=bz2lib,lz4,lzma,lzo2,zlib crypto=mbedtls libs=shared,static programs=none xar=expat
libarchive@3.5.2%cce@13.0.2 +iconv compression=bz2lib,lz4,lzma,lzo2,zlib,zstd crypto=mbedtls libs=shared,static programs=none xar=expat
libarchive@3.5.2%gcc@11.2.0 +iconv compression=bz2lib,lz4,lzma,lzo2,zlib,zstd crypto=mbedtls libs=shared,static programs=none xar=expat
libarchive@3.5.2%gcc@11.2.0 +iconv compression=bz2lib,lz4,lzma,lzo2,zlib,zstd crypto=mbedtls libs=shared,static programs=none xar=expat
libbsd@0.11.5%cce@13.0.2
libbsd@0.11.5%gcc@10.3.0
libbsd@0.11.5%gcc@11.2.0
libcerf@1.3%gcc@11.2.0
libdwarf@20180129%gcc@11.2.0
libdwarf@20180129%gcc@11.2.0
libfabric@1.15.0.0%cce@13.0.2 ~debug~kdreg fabrics=rxm,sockets,tcp,udp
libfabric@1.15.0.0%gcc@11.2.0 ~debug~kdreg fabrics=rxm,sockets,tcp,udp
libffi@3.3%gcc@11.2.0  patches=26f26c6
libffi@3.4.2%cce@13.0.2
libffi@3.4.2%gcc@10.3.0
libffi@3.4.2%gcc@11.2.0
libfontenc@1.1.3%gcc@11.2.0
libgd@2.2.4%gcc@11.2.0
libgd@2.2.4%gcc@11.2.0
libgeotiff@1.6.0%gcc@11.2.0 +jpeg+proj+zlib
libiberty@2.37%gcc@11.2.0 +pic
libice@1.0.9%gcc@11.2.0
libiconv@1.16%cce@13.0.2  libs=shared,static
libiconv@1.16%gcc@10.3.0  libs=shared,static
libiconv@1.16%gcc@11.2.0  libs=shared,static
libjpeg-turbo@2.1.3%gcc@11.2.0
libmd@1.0.4%cce@13.0.2
libmd@1.0.4%gcc@10.3.0
libmd@1.0.4%gcc@11.2.0
libmonitor@2021.11.08%gcc@11.2.0 ~commrank~dlopen+hpctoolkit
libpciaccess@0.16%gcc@11.2.0
libpng@1.6.37%cce@13.0.2
libpng@1.6.37%gcc@11.2.0
libpthread-stubs@0.4%gcc@11.2.0
libquo@1.3.1%gcc@11.2.0
libsigsegv@2.13%gcc@11.2.0
libsm@1.2.3%gcc@11.2.0
libtiff@4.3.0%gcc@11.2.0 ~jbig~jpeg~jpeg12~lerc~libdeflate~lzma~old-jpeg~pixarlog~webp~zlib~zstd
libtool@2.4.7%gcc@11.2.0
libunwind@1.6.2%gcc@11.2.0 ~block_signals~conservative_checks~cxx_exceptions~debug~debug_frame+docs+pic+tests+weak_backtrace+xz~zlib components=none libs=shared,static
libunwind@1.6.2%gcc@11.2.0 ~block_signals~conservative_checks~cxx_exceptions~debug~debug_frame+docs+pic+tests+weak_backtrace+xz~zlib components=none libs=shared,static
libuv@1.44.1%cce@13.0.2
libuv@1.44.1%gcc@11.2.0
libx11@1.7.0%gcc@11.2.0
libxau@1.0.8%gcc@11.2.0
libxaw@1.0.13%gcc@11.2.0
libxcb@1.14%gcc@11.2.0
libxdmcp@1.1.2%gcc@11.2.0
libxext@1.3.3%gcc@11.2.0
libxfont@1.5.2%gcc@11.2.0
libxft@2.3.2%gcc@11.2.0
libxinerama@1.1.3%gcc@11.2.0
libxml2@2.9.13%cce@13.0.2 ~python
libxml2@2.9.13%gcc@10.3.0 ~python
libxml2@2.9.13%gcc@11.2.0 ~python
libxml2@2.9.13%gcc@11.2.0 ~python
libxmu@1.1.2%gcc@11.2.0
libxpm@3.5.12%gcc@11.2.0
libxrender@0.9.10%gcc@11.2.0
libxt@1.1.5%gcc@11.2.0
likwid@5.2.1%gcc@11.2.0 +cuda+fortran
lua@5.3.5%gcc@11.2.0 ~pcfile+shared fetcher=curl
lz4@1.9.3%cce@13.0.2  libs=shared,static
lz4@1.9.3%gcc@11.2.0  libs=shared,static
lzo@2.10%cce@13.0.2  libs=shared,static
lzo@2.10%gcc@11.2.0  libs=shared,static
m4@1.4.18%gcc@11.2.0 +sigsegv patches=3877ab5,fc9b616
makedepend@1.0.5%gcc@11.2.0
mbedtls@2.28.0%cce@13.0.2 +pic build_type=Release libs=static
mbedtls@2.28.0%gcc@11.2.0 +pic build_type=Release libs=static
memkind@1.13.0%gcc@11.2.0
meson@0.62.1%gcc@11.2.0  patches=aa6c50d
meson@0.62.1%gcc@11.2.0  patches=aa6c50d
meson@0.62.1%gcc@11.2.0  patches=aa6c50d
metis@5.1.0%cce@13.0.2 ~gdb~int64~real64+shared build_type=Release patches=4991da9
metis@5.1.0%gcc@11.2.0 ~gdb~int64~real64+shared build_type=Release patches=4991da9,b1225da
mkfontdir@1.0.7%gcc@11.2.0
mkfontscale@1.1.2%gcc@11.2.0
mpark-variant@1.4.0%gcc@11.2.0 ~ipo build_type=RelWithDebInfo patches=21a4f8d,4e173fe,b3501f7
mpfr@4.1.0%gcc@11.2.0  libs=shared,static
nano@4.9%gcc@11.2.0
nasm@2.15.05%gcc@11.2.0
nccmp@1.9.0.1%gcc@11.2.0 ~ipo build_type=RelWithDebInfo
ncl@6.6.2%gcc@11.2.0 ~gdal~hdf4+openmp+triangle+udunits2 patches=64f3502,a2f7ac8,a612d41
nco@5.0.1%gcc@11.2.0 ~doc
ncurses@6.1%cce@13.0.2 ~symlinks+termlib abi=none
ncurses@6.1%gcc@10.3.0 ~symlinks+termlib abi=none
ncurses@6.1%gcc@11.2.0 ~symlinks+termlib abi=none
ncview@2.1.8%gcc@11.2.0
netcdf-c@4.8.1%gcc@11.2.0 ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared patches=de556da
netcdf-c@4.8.1%gcc@11.2.0 ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared patches=de556da
netcdf-c@4.8.1%gcc@11.2.0 ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared patches=de556da
netcdf-fortran@4.5.4%gcc@11.2.0 ~doc+pic+shared
netlib-scalapack@2.2.0%gcc@11.2.0 ~ipo~pic+shared build_type=Release patches=072b006,1c9ce5f
ninja@1.10.2%gcc@11.2.0
ninja@1.10.2%gcc@11.2.0
nlohmann-json@3.10.5%gcc@11.2.0 ~ipo~multiple_headers build_type=RelWithDebInfo
numactl@2.0.14%gcc@11.2.0  patches=4e1d78c,62fc8a8,ff37630
nwchem@7.0.2%gcc@11.2.0 ~mpipr~openmp
openblas@0.3.20%gcc@11.2.0 ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared symbol_suffix=none threads=none
openblas@0.3.20%gcc@11.2.0 ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared symbol_suffix=none threads=openmp
openjdk@11.0.14.1_1%gcc@11.2.0
openjpeg@2.3.1%gcc@11.2.0 ~codec~ipo build_type=RelWithDebInfo
openpmd-api@0.14.4%gcc@11.2.0 ~adios1+adios2+hdf5~ipo+mpi~python+shared build_type=RelWithDebInfo
otf2@2.3%gcc@11.2.0
pango@1.50.7%gcc@11.2.0 ~X~strip buildtype=debugoptimized default_library=shared
papi@6.0.0.1%gcc@11.2.0 ~cuda+example~infiniband~lmsensors~nvml~powercap~rapl~rocm~rocm_smi~sde+shared~static_tools
papi@6.0.0.1%gcc@11.2.0 +cuda+example~infiniband~lmsensors~nvml~powercap~rapl~rocm~rocm_smi~sde+shared~static_tools
parallel@20210922%gcc@11.2.0
parallel-netcdf@1.12.2%gcc@11.2.0 ~burstbuffer+cxx+fortran+pic+shared
parmetis@4.0.3%cce@13.0.2 ~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f89253,50ed208,704b84f
parmetis@4.0.3%gcc@11.2.0 ~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f89253,50ed208,704b84f
pcre@8.45%gcc@11.2.0 ~jit+multibyte+utf
pdt@3.25.1%gcc@11.2.0 ~pic
perl@5.34.1%gcc@11.2.0 +cpanm+shared+threads
petsc@3.17.1%cce@13.0.2 ~X~batch~cgns~complex~cuda~debug+double~exodusii~fftw+fortran~giflib+hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack+shared~strumpack~suite-sparse+superlu-dist~tetgen~trilinos~valgrind clanguage=C
petsc@3.17.1%gcc@11.2.0 ~X~batch~cgns~complex~cuda~debug+double~exodusii~fftw+fortran~giflib+hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack+shared+strumpack~suite-sparse+superlu-dist~tetgen~trilinos~valgrind clanguage=C
petsc@3.17.1%gcc@11.2.0 ~X~batch~cgns~complex+cuda~debug+double~exodusii~fftw+fortran~giflib+hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack+shared+strumpack~suite-sparse+superlu-dist~tetgen~trilinos~valgrind clanguage=C cuda_arch=80
petsc@3.17.1%gcc@11.2.0 ~X~batch~cgns~complex+cuda~debug+double~exodusii~fftw+fortran~giflib+hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack+shared+strumpack~suite-sparse+superlu-dist~tetgen~trilinos~valgrind clanguage=C cuda_arch=none
pixman@0.40.0%gcc@11.2.0
pkgconf@1.8.0%cce@13.0.2
pkgconf@1.8.0%gcc@10.3.0
pkgconf@1.8.0%gcc@11.2.0
plasma@21.8.29%gcc@11.2.0 ~ipo~lua+shared build_type=RelWithDebInfo
proj@8.2.1%gcc@11.2.0 +curl+tiff
py-beniget@0.4.1%gcc@11.2.0
py-cython@0.29.24%gcc@10.3.0
py-cython@0.29.24%gcc@11.2.0
py-gast@0.5.3%gcc@11.2.0
py-libensemble@0.9.1%gcc@10.3.0 ~deap~mpi~mpmath~nlopt~petsc4py~pyyaml~scipy~tasmanian
py-mpi4py@3.1.2%gcc@11.2.0
py-numpy@1.22.3%gcc@10.3.0 ~blas~lapack patches=873745d,e9508c3
py-numpy@1.22.3%gcc@11.2.0 +blas+lapack patches=873745d,e9508c3
py-periodictable@1.5.0%gcc@11.2.0
py-picmistandard@0.0.19%gcc@11.2.0
py-pip@21.3.1%gcc@10.3.0
py-pip@21.3.1%gcc@11.2.0
py-pip@21.3.1%gcc@11.2.0
py-pip@21.3.1%gcc@11.2.0
py-ply@3.11%gcc@11.2.0
py-psutil@5.8.0%gcc@10.3.0
py-pybind11@2.8.1%gcc@11.2.0 ~ipo build_type=RelWithDebInfo
py-pyparsing@3.0.6%gcc@11.2.0
py-pythran@0.10.0%gcc@11.2.0
py-scipy@1.8.0%gcc@11.2.0
py-setuptools@59.4.0%gcc@10.3.0
py-setuptools@59.4.0%gcc@11.2.0
py-setuptools@59.4.0%gcc@11.2.0
py-setuptools@59.4.0%gcc@11.2.0
py-warpx@22.05%gcc@11.2.0 +mpi
py-warpx@22.05%gcc@11.2.0 +mpi
py-warpx@22.05%gcc@11.2.0 +mpi
py-wheel@0.37.0%gcc@10.3.0
py-wheel@0.37.0%gcc@11.2.0
py-wheel@0.37.0%gcc@11.2.0
py-wheel@0.37.0%gcc@11.2.0
python@3.8.13%cce@13.0.2 +bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3~ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,ebdca64,f2fd060
python@3.8.13%gcc@10.3.0 +bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3~ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,ebdca64,f2fd060
python@3.8.13%gcc@11.2.0 +bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3~ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,ebdca64,f2fd060
python@3.8.13%gcc@11.2.0 +bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3~ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,ebdca64,f2fd060
python@3.8.13%gcc@11.2.0 +bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3~ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,ebdca64,f2fd060
python@3.8.13%gcc@11.2.0 +bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3~ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93,4c24573,ebdca64,f2fd060
qthreads@1.16%gcc@11.2.0 +hwloc~spawn_cache+static scheduler=distrib stack_size=4096
quantum-espresso@7.0%gcc@11.2.0 +cmake~elpa~environ+epw~gipaw~ipo~libxc+mpi~openmp+patch~qmcpack+scalapack build_type=RelWithDebInfo hdf5=none
raja@0.14.0%gcc@11.2.0 ~cuda+examples+exercises~ipo+openmp~rocm+shared~tests build_type=RelWithDebInfo
raja@0.14.0%gcc@11.2.0 +cuda+examples+exercises~ipo+openmp~rocm+shared~tests build_type=RelWithDebInfo cuda_arch=80
readline@7.0%cce@13.0.2
readline@7.0%gcc@10.3.0
readline@7.0%gcc@11.2.0
renderproto@0.11.1%gcc@11.2.0
rhash@1.4.2%cce@13.0.2  patches=093518c,3fbfe46
rhash@1.4.2%gcc@11.2.0  patches=093518c,3fbfe46
shapelib@1.5.0%gcc@11.2.0 ~ipo build_type=RelWithDebInfo
slate@2021.05.02%gcc@11.2.0 ~cuda~ipo+mpi+openmp~rocm+shared build_type=RelWithDebInfo
slate@2021.05.02%gcc@11.2.0 +cuda~ipo+mpi+openmp~rocm+shared build_type=RelWithDebInfo cuda_arch=80
slepc@3.17.1%cce@13.0.2 +arpack~blopex~cuda~rocm
slepc@3.17.1%gcc@11.2.0 +arpack~blopex~cuda~rocm
slepc@3.17.1%gcc@11.2.0 +arpack~blopex+cuda~rocm cuda_arch=80
snappy@1.1.8%cce@13.0.2 ~ipo+pic+shared build_type=RelWithDebInfo patches=c9cfecb
snappy@1.1.8%gcc@11.2.0 ~ipo+pic+shared build_type=RelWithDebInfo patches=c9cfecb
sqlite@3.38.5%cce@13.0.2 +column_metadata+dynamic_extensions+fts~functions+rtree
sqlite@3.38.5%gcc@10.3.0 +column_metadata+dynamic_extensions+fts~functions+rtree
sqlite@3.38.5%gcc@11.2.0 +column_metadata+dynamic_extensions+fts~functions+rtree
strumpack@6.3.1%gcc@11.2.0 ~butterflypack+c_interface~count_flops~cuda~ipo+mpi+openmp+parmetis~rocm~scotch+shared~slate~task_timers+zfp build_type=RelWithDebInfo
strumpack@6.3.1%gcc@11.2.0 ~butterflypack+c_interface~count_flops+cuda~ipo+mpi+openmp+parmetis~rocm~scotch+shared~slate~task_timers+zfp build_type=RelWithDebInfo cuda_arch=80
subversion@1.14.1%gcc@11.2.0 +nls~perl~serf
sundials@6.2.0%cce@13.0.2 +ARKODE+CVODE+CVODES+IDA+IDAS+KINSOL~cuda+examples+examples-install~f2003~fcmix+generic-math~hypre~int64~ipo~klu~lapack~magma~monitoring+mpi~openmp~petsc~profiling~pthread~raja~rocm+shared+static~superlu-dist~superlu-mt~sycl~trilinos build_type=RelWithDebInfo cstd=99 cxxstd=14 logging-level=0 precision=double
sundials@6.2.0%gcc@11.2.0 +ARKODE+CVODE+CVODES+IDA+IDAS+KINSOL~cuda+examples+examples-install~f2003~fcmix+generic-math~hypre~int64~ipo~klu~lapack~magma~monitoring+mpi~openmp~petsc~profiling~pthread~raja~rocm+shared+static~superlu-dist~superlu-mt~sycl~trilinos build_type=RelWithDebInfo cstd=99 cxxstd=14 logging-level=0 precision=double
superlu@5.3.0%cce@13.0.2 ~ipo+pic build_type=RelWithDebInfo
superlu@5.3.0%gcc@11.2.0 ~ipo+pic build_type=RelWithDebInfo
superlu-dist@7.2.0%cce@13.0.2 ~cuda~int64~ipo+openmp~rocm+shared build_type=RelWithDebInfo patches=8da9e21
superlu-dist@7.2.0%gcc@11.2.0 ~cuda~int64~ipo+openmp~rocm+shared build_type=RelWithDebInfo patches=8da9e21
superlu-dist@7.2.0%gcc@11.2.0 ~cuda~int64~ipo+openmp~rocm+shared build_type=RelWithDebInfo patches=8da9e21
sz@2.1.12%cce@13.0.2 ~fortran~hdf5~ipo~netcdf~pastri~python~random_access+shared~stats~time_compression build_type=RelWithDebInfo
sz@2.1.12%gcc@11.2.0 ~fortran~hdf5~ipo~netcdf~pastri~python~random_access+shared~stats~time_compression build_type=RelWithDebInfo
tar@1.30%cce@13.0.2
tar@1.30%gcc@10.3.0
tar@1.30%gcc@11.2.0
tasmanian@7.7%gcc@11.2.0 ~blas~cuda~fortran~ipo~magma~mpi+openmp~python~rocm~xsdkflags build_type=Release
tau@2.31.1%gcc@11.2.0 ~adios2+binutils~comm~craycnl+cuda+elf+fortran~gasnet+io~level_zero+libdwarf+libunwind~likwid+mpi~ompt~opari~opencl~openmp+otf2+papi+pdt~phase~ppc64le~profileparam+pthreads~python~rocm~rocprofiler~roctracer~scorep~shmem~sqlite~x86_64
tcsh@6.24.00%gcc@11.2.0  patches=3a4e60f
termcap@1.3.1%gcc@11.2.0
texinfo@6.5%gcc@11.2.0  patches=12f6edb,1732115
trilinos@13.0.1%gcc@11.2.0 ~adios2+amesos+amesos2+anasazi+aztec~basker+belos+boost~chaco~complex~cuda~cuda_rdc~debug~dtk+epetra+epetraext~epetraextbtf~epetraextexperimental~epetraextgraphreorderings~exodus+explicit_template_instantiation~float+fortran~gtest~hdf5~hypre+ifpack+ifpack2+intrepid+intrepid2~ipo+isorropia+kokkos~mesquite+minitensor+ml+mpi+muelu~mumps+nox~openmp~panzer+phalanx+piro~python~rocm~rocm_rdc+rol+rythmos+sacado~scorec+shards+shared+shylu+stk+stokhos+stratimikos~strumpack~suite-sparse~superlu+superlu-dist+teko+tempus+thyra+tpetra+trilinoscouplings~wrapper~x11+zoltan+zoltan2 build_type=RelWithDebInfo cxxstd=14 gotype=long_long
udunits@2.2.28%gcc@11.2.0
unzip@6.0%gcc@11.2.0
utf8proc@2.6.1%gcc@11.2.0 ~ipo build_type=RelWithDebInfo
util-linux-uuid@2.33.1%cce@13.0.2
util-linux-uuid@2.33.1%gcc@10.3.0
util-linux-uuid@2.33.1%gcc@11.2.0
util-macros@1.19.3%gcc@11.2.0
vtk-m@1.7.1%gcc@11.2.0 ~64bitids+ascent_types~cuda+doubleprecision~ipo~kokkos~logging~mpi+openmp+rendering~rocm~shared~tbb~testlib~virtuals build_type=Release
warpx@22.05%gcc@11.2.0 +app~ascent~eb~ipo+lib+mpi+mpithreadmultiple+openpmd+psatd+qed~qedtablegen+shared+tprof build_type=RelWithDebInfo compute=omp dims=2 precision=double
warpx@22.05%gcc@11.2.0 +app~ascent~eb~ipo+lib+mpi+mpithreadmultiple+openpmd+psatd+qed~qedtablegen+shared+tprof build_type=RelWithDebInfo compute=omp dims=3 precision=double
warpx@22.05%gcc@11.2.0 +app~ascent~eb~ipo+lib+mpi+mpithreadmultiple+openpmd+psatd+qed~qedtablegen+shared+tprof build_type=RelWithDebInfo compute=omp dims=rz precision=double
xcb-proto@1.14.1%gcc@11.2.0
xerces-c@3.2.3%gcc@11.2.0  cxxstd=default netaccessor=curl transcoder=gnuiconv
xerces-c@3.2.3%gcc@11.2.0  cxxstd=default netaccessor=curl transcoder=iconv
xextproto@7.3.0%gcc@11.2.0
xineramaproto@1.2.1%gcc@11.2.0
xproto@7.0.31%gcc@11.2.0
xterm@353%gcc@11.2.0
xtrans@1.3.5%gcc@11.2.0
xz@5.2.5%cce@13.0.2 ~pic libs=shared,static
xz@5.2.5%gcc@10.3.0 ~pic libs=shared,static
xz@5.2.5%gcc@11.2.0 ~pic libs=shared,static
xz@5.2.5%gcc@11.2.0 +pic libs=shared,static
yasm@1.3.0%gcc@11.2.0
zfp@0.5.5%cce@13.0.2 ~aligned~c~cuda~fasthash~fortran~ipo~openmp~profile~python+shared~strided~twoway bsws=64 build_type=RelWithDebInfo
zfp@0.5.5%gcc@11.2.0 ~aligned~c~cuda~fasthash~fortran~ipo~openmp~profile~python+shared~strided~twoway bsws=64 build_type=RelWithDebInfo
zfp@0.5.5%gcc@11.2.0 ~aligned~c+cuda~fasthash~fortran~ipo~openmp~profile~python+shared~strided~twoway bsws=64 build_type=RelWithDebInfo cuda_arch=80
zlib@1.2.12%cce@13.0.2 +optimize+pic+shared patches=0d38234
zlib@1.2.12%gcc@10.3.0 +optimize+pic+shared patches=0d38234
zlib@1.2.12%gcc@11.2.0 +optimize+pic+shared patches=0d38234
zstd@1.5.2%cce@13.0.2 ~programs libs=shared,static
zstd@1.5.2%gcc@11.2.0 ~programs libs=shared,static

Shown below are a few useful spack find commands you can use to filter output.

All root specs built with the CCE compiler
elvis@perlmutter> spack find -x --format "{name}" %cce
adios2  hdf5  hypre  kokkos  kokkos-kernels  petsc  slepc  sundials  superlu

We have optimized our CUDA builds for the NVIDIA A100 GPUS which is set in Spack packages using the cuda_arch=80 variant. If you want to find all specs built for CUDA you can use the following command shown below.

All specs built with variant cuda_arch=80
elvis@perlmutter> spack find --format "{name}@{version}%{compiler.name}@{compiler.version}: {variants}" cuda_arch=80
adios2@2.8.0%gcc@11.2.0: +blosc+bzip2+cuda~dataman~dataspaces+fortran~hdf5~ipo+mpi+pic+png~python+shared+ssc+sst+sz+zfp build_type=Release cuda_arch=80
camp@0.2.2%gcc@11.2.0: +cuda~ipo~rocm~tests build_type=RelWithDebInfo cuda_arch=80
hypre@2.24.0%gcc@11.2.0: ~complex+cuda~debug+fortran~gptune~int64~internal-superlu~mixedint+mpi+openmp~rocm+shared+superlu-dist~unified-memory cuda_arch=80
kokkos@3.6.00%gcc@11.2.0: ~aggressive_vectorization~compiler_warnings+cuda~cuda_constexpr~cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning+wrapper build_type=RelWithDebInfo cuda_arch=80 std=14
kokkos-kernels@3.6.00%gcc@11.2.0: ~blas~cblas~cublas+cuda~cusparse~ipo~lapack~lapacke~mkl~openmp~pthread~serial+shared~superlu build_type=RelWithDebInfo cuda_arch=80 execspace_cuda=auto execspace_openmp=auto execspace_serial=auto execspace_threads=auto layouts=left memspace_cudaspace=auto memspace_cudauvmspace=auto offsets=int,size_t ordinals=int scalars=double
lammps@20220107%gcc@11.2.0: ~asphere~body~class2~colloid~compress~coreshell+cuda+cuda_mps~dipole~exceptions+ffmpeg~granular~ipo+jpeg+kim~kokkos~kspace~latte+lib~manybody~mc~meam~misc~mliap~molecule+mpi~mpiio~opencl+openmp~opt~peri+png~poems~python~qeq~replica~rigid~shock~snap~spin~srd~user-adios~user-atc~user-awpmd~user-bocs~user-cgsdk~user-colvars~user-diffraction~user-dpd~user-drude~user-eff~user-fep~user-h5md~user-lb~user-manifold~user-meamc~user-mesodpd~user-mesont~user-mgpt~user-misc~user-mofff~user-netcdf~user-omp~user-phonon~user-plumed~user-ptm~user-qtb~user-reaction~user-reaxc~user-sdpd~user-smd~user-smtbq~user-sph~user-tally~user-uef~user-yaff~voronoi build_type=RelWithDebInfo cuda_arch=80
petsc@3.17.1%gcc@11.2.0: ~X~batch~cgns~complex+cuda~debug+double~exodusii~fftw+fortran~giflib+hdf5~hpddm~hwloc+hypre~int64~jpeg~knl~kokkos~libpng~libyaml~memkind+metis~mkl-pardiso~mmg~moab~mpfr+mpi~mumps+openmp~p4est~parmmg~ptscotch~random123~rocm~saws~scalapack+shared+strumpack~suite-sparse+superlu-dist~tetgen~trilinos~valgrind clanguage=C cuda_arch=80
raja@0.14.0%gcc@11.2.0: +cuda+examples+exercises~ipo+openmp~rocm+shared~tests build_type=RelWithDebInfo cuda_arch=80
slate@2021.05.02%gcc@11.2.0: +cuda~ipo+mpi+openmp~rocm+shared build_type=RelWithDebInfo cuda_arch=80
slepc@3.17.1%gcc@11.2.0: +arpack~blopex+cuda~rocm cuda_arch=80
strumpack@6.3.1%gcc@11.2.0: ~butterflypack+c_interface~count_flops+cuda~ipo+mpi+openmp+parmetis~rocm~scotch+shared~slate~task_timers+zfp build_type=RelWithDebInfo cuda_arch=80
zfp@0.5.5%gcc@11.2.0: ~aligned~c+cuda~fasthash~fortran~ipo~openmp~profile~python+shared~strided~twoway bsws=64 build_type=RelWithDebInfo cuda_arch=80

Loading E4S Packages in Your User Environment

We do not generate modules for Spack packages, therefore one must use spack load to load the appropriate package in your user environment. Please refer to the help message (spack load --help) for a list of options.

To see a list of Spack packages loaded in your user environment type spack load --list. The first time you module load e4s no packages will be loaded.

elvis@perlmutter> spack load --list
==> 0 loaded packages

To load a package type spack load and the name of the package. The command spack load works similarly to module load in the sense that it will dynamically change your user environment to access software.

Use Case Example: Loading HDF5

Let's assume you want to use HDF5, first you need to determine which Spack package you want to load. We can run the following which shows the installed versions of HDF5.

elvis@perlmutter> spack find hdf5

This will output a list of installed packages as shown:

==> 7 installed packages
-- cray-sles15-zen3 / cce@13.0.2 --------------------------------
hdf5@1.10.7  hdf5@1.12.1

-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
hdf5@1.8.22  hdf5@1.10.7  hdf5@1.10.8  hdf5@1.12.1  hdf5@1.12.1

Let's load HDF5 version 1.10.7 that was compiled with GCC version 11.2.0 into our user environment. To specify the version use @ and to specify the compiler use %. These symbols can also be combined as is the case with loading GCC version 11.2.0. Take note that Spack will load the dependencies in your user environment which is the default behavior.

elvis@perlmutter> spack load hdf5@1.10.7%gcc@11.2.0
elvis@perlmutter> spack load --list
==> 20 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6   cray-mpich@8.1.15  expat@2.4.8  libarchive@3.5.2  libiconv@1.16  libuv@1.44.1  lzo@2.10        ncurses@6.1    rhash@1.4.2  zlib@1.2.12
cmake@3.23.1  curl@7.66.0        hdf5@1.10.7  libbsd@0.11.5     libmd@1.0.4    lz4@1.9.3     mbedtls@2.28.0  pkgconf@1.8.0  xz@5.2.5     zstd@1.5.2

Your user environment will be updated. That includes PATH, LD_LIBRARY_PATH, and other variables relevant for loading the package. The HDF5 package provides the h5cc program which will appear in your environment as shown below.

elvis@perlmutter> which h5cc
/global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-sfm6kvjvvobwsmdmgfb5r5tnfno3mkf5/bin/h5cc

elvis@perlmutter> h5cc --version
/global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-sfm6kvjvvobwsmdmgfb5r5tnfno3mkf5/h5cc
dir is /global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-sfm6kvjvvobwsmdmgfb5r5tnfno3mkf5
gcc (GCC) 11.2.0 20210728 (Cray Inc.)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

This confirms that HDF5 was successfully loaded.

View Shell Commands to Load the Package

It can be informative to view the commands that Spack uses when a package is loaded. For example, to see the shell commands used when zlib is loaded, use spack load --sh zlib%gcc.

epalmer@perlmutter:login34:~> $ spack load --sh zlib%gcc
export CMAKE_PREFIX_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5;
export LD_LIBRARY_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib:/opt/cray/pe/gcc/11.2.0/snos/lib64:/opt/cray/pe/papi/6.0.0.15/lib64:/opt/cray/libfabric/1.15.0.0/lib64;
export MANPATH=/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/share/man:/opt/cray/pe/libsci/21.08.1.2/man:/opt/cray/pe/man/csmlversion:/opt/cray/pe/mpich/8.1.17/ofi/man:/opt/cray/pe/mpich/8.1.17/man/mpich:/opt/cray/pe/dsmml/0.2.2/dsmml/man:/opt/cray/pe/craype/2.7.16/man:/opt/cray/pe/gcc/11.2.0/snos/share/man:/opt/cray/pe/perftools/22.06.0/man:/opt/cray/pe/papi/6.0.0.15/share/pdoc/man:/opt/cray/libfabric/1.15.0.0/share/man:/usr/share/lmod/lmod/share/man:/usr/local/man:.:;
export PKG_CONFIG_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/software/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib/pkgconfig:/opt/cray/pe/dsmml/0.2.2/dsmml/lib/pkgconfig:/opt/cray/pe/craype/2.7.16/pkg-config:/opt/cray/libfabric/1.15.0.0/lib64/pkgconfig;
export SPACK_LOADED_HASHES=ozmcyfjfv7i5gjjgklfsh43h67vzsuc5;

Find the Install Prefix for a Package

For your application you may want to specify the location of the installed package as an environment variable. In this case, let's set HDF5_DIR to the parent directory of libs and include --the install prefix for our HDF5 package. A convenient way to do this is as follows,

elvis@perlmutter> export HDF5_DIR=$(spack location -i hdf5@1.10.7%gcc@11.2.0)

Unloading Spack Packages From Your Environment

You can use spack unload to remove packages from your environment which functions similarly to module unload. spack unload will only work on installed specs that were loaded in your environment. In the example below, we show how to unload the xz Spack package from the user environment.

elvis@perlmutter> spack load --list
==> 21 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6   cray-mpich@8.1.15  expat@2.4.8  hdf5@1.10.7       libbsd@0.11.5  libmd@1.0.4   lz4@1.9.3  mbedtls@2.28.0  pkgconf@1.8.0  xz@5.2.5     zstd@1.5.2
cmake@3.23.1  curl@7.66.0        hdf5@1.8.22  libarchive@3.5.2  libiconv@1.16  libuv@1.44.1  lzo@2.10   ncurses@6.1     rhash@1.4.2    zlib@1.2.12

elvis@perlmutter> spack unload xz@5.2.5

elvis@perlmutter> spack load --list
==> 20 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6   cray-mpich@8.1.15  expat@2.4.8  hdf5@1.10.7       libbsd@0.11.5  libmd@1.0.4   lz4@1.9.3  mbedtls@2.28.0  pkgconf@1.8.0  zlib@1.2.12
cmake@3.23.1  curl@7.66.0        hdf5@1.8.22  libarchive@3.5.2  libiconv@1.16  libuv@1.44.1  lzo@2.10   ncurses@6.1     rhash@1.4.2    zstd@1.5.2

If you want to remove all packages from your user environment you can run the following:

elvis@perlmutter> spack unload --all
elvis@perlmutter> spack load --list
==> 0 loaded packages

Using Hashes for Packages with Multiple Versions

Sometimes, you will need to use hashes instead of specs to load a specific package variant. Let's assume we want to load CMake and we get the error, "Use a more specific spec". Note that in this example we have three specs for same version but each has a different hash (The 7 character code proceeding the spec).

Error

elvis@perlmutter> spack load cmake@3.23.1%gcc@11.2.0
==> Error: cmake@3.23.1%gcc@11.2.0 matches multiple packages.
  Matching packages:
    p23fzuo cmake@3.23.1%gcc@11.2.0 arch=cray-sles15-zen3
    6v4d2hu cmake@3.23.1%gcc@11.2.0 arch=cray-sles15-zen3
  Use a more specific spec.

In this case, we can use the hash to uniquely identify the desired variant of CMake. For example, if we want to load the spec with hash p23fzuo, we would run the following:

elvis@perlmutter> spack load /p23fzuo

Loading a Spack Package Without Dependencies

The command spack load will load a package and its dependencies which is recommended when using software, however sometimes you don't want to load all the extra dependencies in your environment. Spack provides a way to load only the package via --only package which will load the package and not its dependencies. In the example shown below, we load one package. Take note of this compared to its normal behavior which loads all dependencies.

elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> spack load --only package /p23fzuo
elvis@perlmutter> spack load --list
==> 1 loaded package
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
cmake@3.23.1

elvis@perlmutter> spack unload -a
elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> spack load /p23fzuo
elvis@perlmutter> spack load --list
==> 17 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6  cmake@3.23.1  curl@7.66.0  expat@2.4.8  libarchive@3.5.2  libbsd@0.11.5  libiconv@1.16  libmd@1.0.4  libuv@1.44.1  lz4@1.9.3  lzo@2.10  mbedtls@2.28.0  ncurses@6.1  rhash@1.4.2  xz@5.2.5  zlib@1.2.12  zstd@1.5.2

Spack Views

Spack environments can define filesystem views to directly access the software without having to load software.

When you load e4s module module load e4s we activate a spack environment without a view (spack env activate --without-view e4s) which means the filesystem view is not enabled. This means you must use spack load in order to use the software.

In order to activate the view, you will need to reactivate the e4s environment, make sure you deactivate from any existing environment by running despactivate:

elvis@perlmutter> despacktivate
elvis@perlmutter> spack env activate --with-view e4s

Spack will automatically load everything in the view, Take note, we have 0 loaded packages in the output of spack load --list, however we can access software from the view since $PATH, $LD_LIBRARY_PATH, etc... has been updated in your user environment. Spack will present software in a view similar to how one access software in Linux in /usr/local or /usr/bin.

elvis@perlmutter> spack load --list
==> 0 loaded packages

elvis@perlmutter> which h5cc
/global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/views/default/hdf5/1.10.7-gcc-11.2.0/bin/h5cc

elvis@perlmutter> which cmake
/global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/views/default/cmake/3.23.1-gcc-11.2.0/bin/cmake

Now you may be wondering which packages you can access with spack views. We have selected all gcc specs in the view with some exceptions, for instance we can't load two instances of same package such as cmake in the same view or two packages built with gcc and cce compiler or two packages one built with CUDA and without.

Developer Workflow

Shown below is a guide to get you started building packages with our Spack instance. First, we recommend you source the following file which will activate a python environment and install clingo in your environment.

# bash, sh, zsh users
source $SPACK_ROOT/bin/spack-setup.sh

# csh or tcsh users
source $SPACK_ROOT/bin/spack-setup.csh

Upon sourcing the file you will see the following message, note that clingo is installed as a Python package which can be seen by running pip list.

elvis@perlmutter> source $SPACK_ROOT/bin/spack-setup.sh
Creating python virtual environment in /global/homes/s/elvis/.spack-pyenv
WARNING: You are using pip version 20.2.3; however, version 21.3.1 is available.
You should consider upgrading via the '/global/homes/s/elvis/.spack-pyenv/bin/python3 -m pip install --upgrade pip' command.
Package       Version
------------- -------
cffi          1.15.1
clingo        5.5.2
mpi4py        3.1.2
numpy         1.22.3
periodictable 1.5.0
picmistandard 0.0.19
pip           20.2.3
pycparser     2.21
pyelftools    0.26
pyparsing     3.0.6
scipy         1.8.0
setuptools    59.4.0
toml          0.10.2
WARNING: You are using pip version 20.2.3; however, version 21.3.1 is available.
You should consider upgrading via the '/global/homes/s/elvis/.spack-pyenv/bin/python3 -m pip install --upgrade pip' command.
Your python interpreter used by spack is /global/homes/s/elvis/.spack-pyenv/bin/python

We recommend you run the following to see if Spack will concretize a spec and not try to bootstrap clingo. Note we must exit the e4s environment via despacktivate otherwise Spack will use the production path which will cause permission issues. You can confirm your active environment by running spack env status. Next try running spack spec zlib to check if spack will concretize zlib which will use clingo installed in your Python environment.

(.spack-pyenv) elvis@perlmutter> despacktivate
(.spack-pyenv) elvis@perlmutter> spack env status
==> No active environment

(.spack-pyenv) elvis@perlmutter> spack spec zlib
Input spec
--------------------------------
zlib

Concretized
--------------------------------
zlib@1.2.12%gcc@11.2.0+optimize+pic+shared patches=0d38234 arch=cray-sles15-zen3

We recommend you create a Spack environment in your user space to install your Spack stack. This can be done using spack env create command using the -d option to specify a directory. This will create a directory with an empty spack.yaml. To activate the environment you can run spack env activate and name of directory.

(.spack-pyenv) elvis@perlmutter> spack env create -d $HOME/e4s-22.05-demo
==> Created environment in /global/homes/s/elvis/e4s-22.05-demo
==> You can activate this environment with:
==>   spack env activate /global/homes/s/elvis/e4s-22.05-demo

(.spack-pyenv) elvis@perlmutter> spack env activate $HOME/e4s-22.05-demo

(.spack-pyenv) elvis@perlmutter> ls -l $HOME/e4s-22.05-demo
total 1
-rw-rw---- 1 elvis elvis 199 Jul 19 12:10 spack.yaml

You can retrieve the site scope via spack config --scope site get packages.

This Spack instance is configured to install packages in your HOME directory, the Spack site configuration can be found in this directory $SPACK_ROOT/etc/spack/ which applies configuration to all users but this can be overridden in your spack.yaml. Take note of the following settings below for build stage, and installation tree. Note that HOME directory is shared between Cori and Perlmutter, therefore we organize user installation by system. We recommend you adopt a similar naming scheme if you plan on building Spack packages on both systems.

(.spack-pyenv) elvis@perlmutter> cat $SPACK_ROOT/etc/spack/config.yaml
config:
  build_stage: $HOME/spack-workspace/perlmutter/build_stage
  misc_cache: $HOME/spack-workspace/perlmutter/misc_cache
  install_tree:
    root: $HOME/spack-workspace/perlmutter/software
  source_cache: $HOME/.spack/cache

Shown below is the site configuration for modules.yaml, we have configured the module root to be $HOME/spack-workspace/perlmutter/modules where TCL modules will be generated. Furthermore we have customized the TCL module setting when generating modules. You are welcome to override these configuration in your spack.yaml.

(.spack-pyenv) elvis@perlmutter> cat $SPACK_ROOT/etc/spack/modules.yaml
modules:
  default:
    roots:
      tcl: $HOME/spack-workspace/perlmutter/modules
    tcl:
      naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
      all:
        conflict:
        - '{name}'
        environment:
          set:
            '{name}_ROOT': '{prefix}'

Now let's add the following package in your spack.yaml and try building it via Spack. We will build papi. Shown below is the content of spack.yaml, please copy this content to your spack.yaml.

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs:
  - papi
  view: true

Note we don't provide a buildcache so papi will be built from source. Now let's install all packages in your environment by running spack install.

(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack install
==> Starting concretization
==> Environment concretized in 13.19 seconds.
==> Concretized papi
 -   s2y4nrv  papi@6.0.0.1%gcc@11.2.0~cuda+example~infiniband~lmsensors~nvml~powercap~rapl~rocm~rocm_smi~sde+shared~static_tools arch=cray-sles15-zen3

==> Installing environment /global/homes/s/elvis/e4s-22.05-demo
==> Installing papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
==> No binary for papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af found: installing from source
==> Using cached archive: /global/homes/s/elvis/.spack/cache/_source-cache/archive/3c/3cd7ed50c65b0d21d66e46d0ba34cd171178af4bbf9d94e693915c1aca1e287f.tar.gz
==> No patches needed for papi
==> papi: Executing phase: 'autoreconf'
==> papi: Executing phase: 'configure'
==> papi: Executing phase: 'build'
==> papi: Executing phase: 'install'
==> papi: Successfully installed papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
  Fetch: 0.01s.  Build: 25.80s.  Total: 25.81s.
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
==> Updating view at /global/homes/s/elvis/e4s-22.05-demo/.spack-env/view

We can find the root directory for papi by running the following. We see the package was installed in our home directory.

(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack location -i papi
/global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af

We can create the module tree and retrieve the location of our generated module as shown below.

(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack module tcl find --full-path papi
/global/homes/s/elvis/spack-workspace/perlmutter/modules/cray-sles15-zen3/papi/6.0.0.1-gcc-11.2.0-s2y4nrv

If you want to add this to MODULEPATH you can simply run the following:

(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> module use $HOME/spack-workspace/perlmutter/modules/$(spack arch)

Now we see the modulefile papi/6.0.0.1-gcc-11.2.0-s2y4nrv is visible and can be loaded as any other software package.

(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> ml -t av papi
/global/homes/p/pratham/spack-workspace/perlmutter/modules/cray-sles15-zen3:
papi/6.0.0.1-gcc-11.2.0-s2y4nrv
/global/common/software/spackecp/perlmutter/e4s-22.05/modules/cray-sles15-zen3:
papi/6.0.0.1-gcc-11.2.0-cuda
papi/6.0.0.1-gcc-11.2.0
/opt/cray/pe/lmod/modulefiles/core:
papi/6.0.0.12
papi/6.0.0.14
papi/6.0.0.15

Binary Cache Mirror

We have setup a binary cache mirror named perlmutter-e4s-22.05 that can be added into your spack.yaml. This spack instance includes the binary cache mirror configuration.

You can check this by running spack mirror list to see the list of mirrors.

elvis@perlmutter>  spack mirror list
perlmutter-e4s-22.05    file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05
source_mirror           file:///global/cfs/cdirs/m3503/mirrors/source_mirror
spack-public            https://mirror.spack.io

Spack has support signing and verifying packages via GPG Key which is required when installing packages from the binary cache. To learn more about the binary cache, please see https://spack-tutorial.readthedocs.io/en/latest/tutorial_binary_cache.html#setting-up-a-binary-cache-mirror

First, you will need to install the GPG key locally, note this is only required once. Please run the following command.

elvis@perlmutter> spack buildcache keys -it
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/_pgp/B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F.pub
gpg: key 3BD6B0E9935AEB8F: public key "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: inserting ownertrust of 6

You can see list of gpg keys by running gpg -k make sure you see the following entry in the output.

pub   rsa2048 2022-02-23 [SC]
      B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
uid           [ultimate] GPG Key - e4s <shahzebsiddiqui@lbl.gov>
sub   rsa2048 2022-02-23 [E]

If you have successfully imported the gpg key you should be able to run spack buildcache list to see all the specs in the binary cache.

We have set the environment variable SPACK_GNUPGHOME which points to $HOME/.gnupg in case you run any commands like spack gpg which will typically fail since spack attempts to write GPG keys into the spack instance at $SPACK_ROOT/spack/opt/gpg which you won't have access to.

If you are still in your spack environment that you created in section Developer Workflow, let's install a package from the binary cache, you will notice that spack will install packages from the binary cache instead of building from source.

elvis@perlmutter> spack install raja@0.14.0~cuda+examples+exercises~ipo+openmp~rocm+shared~tests build_type=RelWithDebInfo
==> Installing pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0/cray-sles15-zen3-gcc-11.2.0-pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q.spack
==> Extracting pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q from binary cache
gpg: Signature made Fri 14 Oct 2022 08:12:17 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q
==> Installing berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/cray-sles15-zen3/gcc-11.2.0/berkeley-db-18.1.40/cray-sles15-zen3-gcc-11.2.0-berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3.spack
==> Extracting berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3 from binary cache
gpg: Signature made Thu 13 Oct 2022 11:52:15 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3

...