Spack¶

Spack is a package manager designed to support multiple versions and configurations of software on a wide variety of platforms and environments. Spack is non-destructive: installing a new version of a package does not break existing installations, so many configurations of the same package can coexist.

Using the NERSC-provided Spack¶

NERSC provides the Spack package manager via a modulefile. This Spack instance is preconfigured by NERSC to integrate with the Perlmutter software environment.

In order to use the default Spack instance, run

module load spack

Spack Commands¶

Shown below are some of the most common Spack commands you will use. For a complete list of commands, see the Spack Command Reference.

Command	Description
`spack install <spec>`	Install a package by spec.
`spack find`	List installed packages.
`spack list`	List available packages.
`spack load <spec>`	Load a package into the current environment.
`spack unload <spec>`	Unload a package from the current environment.
`spack load --list`	List packages loaded into the current environment.
`spack env create <name>`	Create a new environment.
`spack env activate <name>`	Activate an environment.
`spack env deactivate`	Deactivate the current environment.
`spack env status`	Show the status of the current environment.
`spack env create -d <dir>`	Create an environment in a directory.
`spack env activate -d <dir>`	Activate an environment in a directory.
`spack concretize -f`	Force concretize an environment.
`spack spec <spec>`	Show what would be installed given a spec.
`spack spec -Il <spec>`	Show the install status and dependencies with hashes.
`spack buildcache list`	List available binary packages.
`spack info <spec>`	Show information about a package.
`spack edit <spec>`	Open a package file in an editor.
`spack config get <section>`	Show the YAML configuration for a section.
`spack config list`	List all configuration scopes.
`spack config blame <section>`	Show the YAML configuration file and line numbers for a section.

In this coming section we will show you how to load and unload Spack packages into your user environment. There will be several examples which can be run on Perlmutter by loading the E4S module.

E4S Setup¶

In this coming section we will show you how to load and unload Spack packages into your user environment. There will be several examples which can be run on Perlmutter by loading E4S module.

Note

You can follow these examples by running the following commands to ensure you have a clean environment:

module load e4s
spack env activate gcc
spack unload --all

Loading Spack Packages¶

Note

Please see E4S Setup before running the examples below.

Unlike module load that is used for loading modules into your user environment, in Spack you must use spack load to load a Spack package into your user environment. In this exercise we will show how to load PAPI into your user environment.

First, let's activate the gcc environment by running the following:

spack env activate gcc

Let's assume you want to use PAPI. First, you need to determine which Spack package you want to load. We can run the following to show the installed versions of PAPI.

spack find papi

This will output a list of installed packages as shown:

==> Installed packages
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
papi@6.0.0.1
==> 1 installed package

Let's load PAPI into our user environment. To specify the version use @ and to specify the compiler use %. These symbols can also be combined as is the case with loading GCC version 11.2.0. Take note that Spack will load the dependencies in your user environment which is the default behavior.

elvis@perlmutter> spack load papi%gcc@11.2.0
elvis@perlmutter> spack load --list
==> 13 loaded packages
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
berkeley-db@18.1.40  kbproto@1.0.7  libuv@1.44.1  ncurses@6.1   pkg-config@0.29.2  util-linux-uuid@2.36.2  xproto@7.0.31
inputproto@2.3.2     libmd@1.0.4    lzo@2.10      papi@6.0.0.1  rhash@1.4.2        xextproto@7.3.0

Your user environment will be updated. That includes PATH, LD_LIBRARY_PATH, and other variables relevant for loading the package. Let's retrieve the papi version, we can do this using the papi_version program which will appear in your environment if you run which papi as shown below.

elvis@perlmutter> which papi_version
/global/common/software/spackecp/perlmutter/e4s-23.05/89639/spack/opt/spack/linux-sles15-zen3/gcc-11.2.0/papi-6.0.0.1-frkekrtdhzrwqsr2nvuntrddpju7etpt/bin/papi_version

Let's retrieve the PAPI version by running the program

elvis@perlmutter> papi_version
PAPI Version: 6.0.0.1

View Shell Commands to Load the Package¶

Note

Please see E4S Setup before running the examples below.

It can be informative to view the commands that Spack uses when a package is loaded. For example, to see the shell commands used when zlib is loaded, use spack load --sh zlib%gcc@11.2.0.

elvis@perlmutter> spack load --sh zlib%gcc@11.2.0
export CMAKE_PREFIX_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q:/opt/cray/pe/mpich/8.1.22/ofi/gnu/9.1:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/cmake-3.23.1-p23fzuowp4yuitemelic7f65nwybthxd:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libuv-1.44.1-gitz4v6soymrlpymzvygqlqibktfsp7a:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zstd-1.5.2-rtaozj7xdfhxengzppbqbth4mya3edpq:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/mbedtls-2.28.0-ucarxpp47pj7yo7abt5povg3swygev3q:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lzo-2.10-rlpwurcqberlfjlpk62go3arnneonv6u:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libiconv-1.16-3xwztgyaaco4thqqhacdst65mmnr5is3:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/expat-2.4.8-yvchkv23jw7po7meyuxckpre5uroipmd:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog:/global/common/software/nersc/pm-2021q4/easybuild/software/Nsight-Systems/2022.2.1:/global/common/software/nersc/pm-2021q4/easybuild/software/Nsight-Compute/2022.1.1;
export LD_LIBRARY_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q/lib:/opt/cray/pe/mpich/8.1.22/ofi/gnu/9.1/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libuv-1.44.1-gitz4v6soymrlpymzvygqlqibktfsp7a/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zstd-1.5.2-rtaozj7xdfhxengzppbqbth4mya3edpq/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/mbedtls-2.28.0-ucarxpp47pj7yo7abt5povg3swygev3q/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lzo-2.10-rlpwurcqberlfjlpk62go3arnneonv6u/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libiconv-1.16-3xwztgyaaco4thqqhacdst65mmnr5is3/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/expat-2.4.8-yvchkv23jw7po7meyuxckpre5uroipmd/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog/lib:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/math_libs/11.7/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/extras/CUPTI/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/extras/Debugger/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/nvvm/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/lib64:/global/common/software/nersc/pm-2022q3/sw/darshan/3.4.0/lib:/opt/cray/pe/papi/6.0.0.16/lib64:/opt/cray/pe/gcc/11.2.0/snos/lib64:/opt/cray/libfabric/1.15.2.0/lib64;
export MANPATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm/share/man:/usr/share/man:/usr/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libiconv-1.16-3xwztgyaaco4thqqhacdst65mmnr5is3/share/man:/usr/share/man:/usr/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog/share/man:/usr/share/man:/usr/man:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/doc/man:/opt/cray/pe/perftools/22.09.0/man:/opt/cray/pe/papi/6.0.0.16/share/pdoc/man:/opt/cray/pe/gcc/11.2.0/snos/share/man:/opt/cray/pe/craype/2.7.19/man:/opt/cray/pe/mpich/8.1.22/ofi/man:/opt/cray/pe/mpich/8.1.22/man/mpich:/opt/cray/pe/libsci/22.11.1.2/man:/opt/cray/pe/man/csmlversion:/opt/cray/pe/dsmml/0.2.2/dsmml/man:/opt/cray/libfabric/1.15.2.0/share/man:/usr/share/lmod/lmod/share/man:/usr/local/man:/usr/share/man:/usr/man:;
export PKG_CONFIG_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q/lib/pkgconfig:/opt/cray/pe/mpich/8.1.22/ofi/gnu/9.1/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm/lib/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libuv-1.44.1-gitz4v6soymrlpymzvygqlqibktfsp7a/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zstd-1.5.2-rtaozj7xdfhxengzppbqbth4mya3edpq/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lzo-2.10-rlpwurcqberlfjlpk62go3arnneonv6u/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa/lib/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/expat-2.4.8-yvchkv23jw7po7meyuxckpre5uroipmd/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog/lib/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/opt/modulefiles/cudatoolkit:/global/common/software/nersc/pm-2022q3/sw/darshan/3.4.0/lib/pkgconfig:/opt/cray/pe/craype/2.7.19/pkg-config:/opt/cray/pe/dsmml/0.2.2/dsmml/lib/pkgconfig:/opt/cray/libfabric/1.15.2.0/lib64/pkgconfig;
export SPACK_LOADED_HASHES=ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:6opvtfcoaxox7oskb7dzisnb32fnk5ne:nugfov2wsvpllczfy6ff7bytzt4oi34q:6yvclv72nfvhlx2fj72n5npigrgouojd:p23fzuowp4yuitemelic7f65nwybthxd:xjxbsefahiue4isqlzwcobgc7sgfhmqm:i6ri5efvxr243dakip6bk7n6b3qfruyb:gitz4v6soymrlpymzvygqlqibktfsp7a:d4gpdx2rxj4calconu7cd6kgq2ev5roe:rtaozj7xdfhxengzppbqbth4mya3edpq:ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:tyswlp4epqidsxgtgxyefciqoz2ju6oe:ucarxpp47pj7yo7abt5povg3swygev3q:rlpwurcqberlfjlpk62go3arnneonv6u:xxbkaphnofa6gxugxszqr5kzew7chrqa:3xwztgyaaco4thqqhacdst65mmnr5is3:4g7s6qpmt2nsclyi2d2ndfek2ptey4m6:yvchkv23jw7po7meyuxckpre5uroipmd:uomjj7io3hhggbfyzofrw5jpw5chlv74:iynqjjkz4jgtxagqy2o2tvspqfn3yvog:zkrv7nh3jglxsimtfzld7vo4mhdqlh3u;

Find the Install Prefix for a Package¶

For your application you may want to specify the location of the installed package as an environment variable. For instance, let's set PAPI_DIR to the install prefix for the papi package. This can be done by running the spack location -i command which returns the install prefix for the package.

elvis@perlmutter> export PAPI_DIR=$(spack location -i papi%gcc@11.2.0)

Unloading Spack Packages From Your Environment¶

Note

Please see E4S Setup before running the examples below.

You can use spack unload to remove packages from your environment which functions similarly to module unload. spack unload will only work on installed specs that were loaded in your environment. In the example below, we show how to unload the cmake Spack package from the user environment. Take note as we unload cmake via spack unload cmake the user environment was updated and it now resorts to the system cmake /usr/bin/cmake

elvis@perlmutter> spack load --list
==> 17 loaded packages
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
bzip2@1.0.6   curl@7.66.0  libarchive@3.6.2  libiconv@1.17  libuv@1.44.1  lzo@2.10        ncurses@6.1  xz@5.4.1     zstd@1.5.5
cmake@3.26.3  expat@2.5.0  libbsd@0.11.7     libmd@1.0.4    lz4@1.9.4     mbedtls@2.28.2  rhash@1.4.2  zlib@1.2.13

elvis@perlmutter> which cmake
/global/common/software/spackecp/perlmutter/e4s-23.05/89639/spack/opt/spack/linux-sles15-zen3/gcc-11.2.0/cmake-3.26.3-j6xprhqvjwkyngr534kqdxnp5uhwf6j4/bin/cmake

elvis@perlmutter> cmake --version
cmake version 3.26.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

Now let's unload cmake and take note that your user environment is using the system installed cmake.

elvis@perlmutter> spack unload cmake
elvis@perlmutter> which cmake
/usr/bin/cmake

elvis@login40> cmake --version
cmake version 3.20.4

CMake suite maintained and supported by Kitware (kitware.com/cmake).

elvis@perlmutter> spack load --list
==> 16 loaded packages
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
bzip2@1.0.6  expat@2.5.0       libbsd@0.11.7  libmd@1.0.4   lz4@1.9.4  mbedtls@2.28.2  rhash@1.4.2  zlib@1.2.13
curl@7.66.0  libarchive@3.6.2  libiconv@1.17  libuv@1.44.1  lzo@2.10   ncurses@6.1     xz@5.4.1     zstd@1.5.5

If you want to remove all packages from your user environment you can run the following:

elvis@perlmutter> spack unload --all
elvis@perlmutter> spack load --list
==> 0 loaded packages

Using Hashes for Packages with Multiple Versions¶

Note

Please see E4S Setup before running the examples below.

Sometimes, you will need to use hashes instead of specs to load a specific package variant. Let's assume we want to load HDF5 and we get the error, "Use a more specific spec". Note that in this example we have three specs for same version but each has a different hash (The 7 character code proceeding the spec).

Error

elvis@perlmutter> spack load hdf5
==> Error: hdf5 matches multiple packages.
  Matching packages:
    poditdt hdf5@1.14.1-2%gcc@=11.2.0 arch=linux-sles15-zen3
    74dfegs hdf5@1.14.1-2%gcc@=11.2.0 arch=linux-sles15-zen3
    2mtt4fm hdf5@1.14.1-2%gcc@=11.2.0 arch=linux-sles15-zen3
  Use a more specific spec (e.g., prepend '/' to the hash).

In this case, we can use the hash to uniquely identify the desired variant of HDF5. For example, if we want to load the spec with hash poditdt, we would run the following:

elvis@perlmutter> spack load /poditdt

Loading a Spack Package Without Dependencies¶

Note

Please see E4S Setup before running the examples below.

The command spack load will load a package and its dependencies which is recommended when using software, however sometimes you don't want to load all the extra dependencies in your environment. Spack provides a way to load only the package via --only package which will load the package and not its dependencies. In the example shown below, we load one package. Take note of this compared to its normal behavior which loads all dependencies.

elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> spack load --only package /poditdt
elvis@perlmutter> spack load --list
==> 1 loaded package
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
hdf5@1.14.1-2

elvis@perlmutter> spack unload -a
elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> spack load /poditdt
elvis@perlmutter> spack load --list
==> 21 loaded packages
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
bzip2@1.0.6   cray-mpich@os  expat@2.5.0  hdf5@1.14.1-2     libbsd@0.11.7  libmd@1.0.4   lz4@1.9.4  mbedtls@2.28.2  pkg-config@0.29.2  xz@5.4.1     zstd@1.5.5
cmake@3.26.3  curl@7.66.0    gmake@4.4.1  libarchive@3.6.2  libiconv@1.17  libuv@1.44.1  lzo@2.10   ncurses@6.1     rhash@1.4.2        zlib@1.2.13

Recommended setup using environments¶

The e4s/<version> modules comes with several environments such as: gcc, nvhpc, cce, cuda. The cuda environment provides packages built with CUDA support with the gcc@11.2.0 compiler. Depending on the environment you are using, please consider loading the appropriate modules

Load the PrgEnv-gnu, cpu module when using gcc environment
Load the PrgEnv-cce, cpu module when using cce environment
Load the PrgEnv-nvhpc, cpu module when using nvhpc environment
Load the PrgEnv-gnu, gpu module when using cuda environment

When in doubt, please refer to the compiler settings to determine the list of modules used. To retrieve a list of compilers you can do spack compiler list. To query a compiler instance like gcc@11.2.0 you can run spack compiler info gcc@11.2.0.

In the cuda environment, you will see packages are built with variants +cuda cuda_arch=80 that are optimized for NVIDIA A100 GPUs (+cuda cuda_arch=80). You should make sure the craype-accel-nvidia modulefile is loaded. Usually this is loaded with the gpu modulefile at login. You can check the accelerator target by running the following:

elvis@perlmutter> echo $CRAY_ACCEL_TARGET
nvidia80

Developer Workflow¶

Please see example below illustrating how to install a spack packages in your user environment.

Example building software and generating modules via spack

Shown below is an example script used to create your environment to install a spack package and generate moudlefiles. You can copy this script and name if workflow.sh. Please make sure the script and spack.yaml are present and in same directory.

#!/bin/bash
rm -rf $HOME/spack-workspace/perlmutter/modules
rm -rf $HOME/spack-workspace/perlmutter/software
module load spack/0.22
source $SPACK_ROOT/bin/spack-setup.sh
spack env status
spack spec zlib
rm -rf $HOME/e4s-demo
spack env create -d $HOME/e4s-demo spack.yaml
spack env activate -d $HOME/e4s-demo
ls -l $HOME/e4s-demo
spack config list
echo  $SPACK_SYSTEM_CONFIG_PATH
ls -l $SPACK_SYSTEM_CONFIG_PATH
cat $SPACK_ROOT/etc/spack/config.yaml
spack clean -m
spack install
spack location -i papi
cat $SPACK_ROOT/etc/spack/modules.yaml
spack module tcl refresh --delete-tree -y
spack module tcl find --full-path papi
module use $HOME/spack-workspace/perlmutter/modules/$(spack arch)
ml -t av papi

Shown below is the spack configuration (spack.yaml) for installing papi

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs:
  - papi
  view: true

Shown below is the output of the script, we named this file workflow.sh. Notice at end of output you will see the modulefile for papi is created in the user $HOME directory.

elvis@login40> bash workflow.sh
    _______________________________________________________________________________________________________
 Welcome to Spack!

 In order to access the production stack, you will need to load a spack environment. Here are some tips to get started:


 'spack env list' - List all Spack environments
 'spack env activate gcc' - Activate the "gcc" Spack environment
 'spack env status' - Display the active Spack environment
 'spack load amrex' - Load the "amrex" Spack package into your user environment

 For additional support, please refer to the following references:

   NERSC E4S Documentation: https://docs.nersc.gov/applications/e4s/
   E4S Documentation: https://e4s.readthedocs.io
   Spack Documentation: https://spack.readthedocs.io/en/latest/
   Spack Slack: https://spackpm.slack.com

______________________________________________________________________________________________________

/global/common/software/spackecp/perlmutter/v0.22/default/spack/bin/spack
Creating python virtual environment in /global/homes/e/elvis/.spack-pyenv

[notice] A new release of pip is available: 23.2.1 -> 24.1
[notice] To update, run: pip install --upgrade pip
Package     Version
----------- -------
cffi        1.16.0
clingo      5.7.1
nersc-pymon 0.1.1
pip         23.2.1
pycparser   2.22
setuptools  65.5.0

[notice] A new release of pip is available: 23.2.1 -> 24.1
[notice] To update, run: pip install --upgrade pip
Your python interpreter used by spack is /global/homes/e/elvis/.spack-pyenv/bin/python
==> No active environment
==> Warning: cannot detect libc from gcc@=13.2.0. The compiler will not be used during concretization.
==> Warning: cannot detect libc from gcc@=10.3.0. The compiler will not be used during concretization.
==> Warning: cannot detect libc from gcc@=11.2.0. The compiler will not be used during concretization.
Input spec
--------------------------------
-   zlib

Concretized
--------------------------------
-   zlib@1.3.1%gcc@12.3.0+optimize+pic+shared build_system=makefile arch=linux-sles15-zen3
-       ^gcc-runtime@12.3.0%gcc@12.3.0 build_system=generic arch=linux-sles15-zen3
[e]      ^glibc@2.31%gcc@12.3.0 build_system=autotools arch=linux-sles15-zen3
     -       ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-sles15-zen3

==> Created independent environment in: /global/homes/e/elvis/e4s-demo
==> Activate with: spack env activate /global/homes/e/elvis/e4s-demo
total 1
-rw-rw-r-- 1 elvis elvis 205 Jun 21 12:08 spack.yaml
compilers concretizer definitions view develop mirrors repos packages modules config upstreams bootstrap ci cdash
/global/common/software/spackecp/perlmutter/spack_settings
total 12
-rw-rw-r-- 1 e4s spackecp 4147 Jun 10 12:59 compilers.yaml
-rw-rw-r-- 1 e4s spackecp 4817 Jun 10 12:59 packages.yaml
config:
  build_stage: $HOME/spack-workspace/perlmutter/build_stage
  misc_cache: $HOME/spack-workspace/perlmutter/misc_cache
  install_tree:
    root: $HOME/spack-workspace/perlmutter/software
  source_cache: $HOME/.spack/cache

==> Removing cached information on repositories
==> Warning: cannot detect libc from gcc@=13.2.0. The compiler will not be used during concretization.
==> Warning: cannot detect libc from gcc@=10.3.0. The compiler will not be used during concretization.
==> Warning: cannot detect libc from gcc@=11.2.0. The compiler will not be used during concretization.
==> Concretized papi
-   mpl3rc6  papi@7.1.0%gcc@12.3.0~cuda~debug+example~infiniband~lmsensors~nvml~powercap~rapl~rocm~rocm_smi~sde+shared~static_tools build_system=autotools patches=48cb202 arch=linux-sles15-zen3
-   brkks4g      ^gcc-runtime@12.3.0%gcc@12.3.0 build_system=generic arch=linux-sles15-zen3
[e]  ea6wt6w      ^glibc@2.31%gcc@12.3.0 build_system=autotools arch=linux-sles15-zen3
    -   bxti4iw      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-sles15-zen3

[+] /usr (external glibc-2.31-ea6wt6wyyi3je7yf5vmcdg6chgi7bi6f)
==> Installing gcc-runtime-12.3.0-brkks4glphmqjotqtuc652bbuh4t2qae [2/4]
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-v0.22/build_cache/linux-sles15-zen3-gcc-12.3.0-gcc-runtime-12.3.0-brkks4glphmqjotqtuc652bbuh4t2qae.spec.json.sig
gpg: Signature made Wed 22 May 2024 08:59:25 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-v0.22/build_cache/linux-sles15-zen3/gcc-12.3.0/gcc-runtime-12.3.0/linux-sles15-zen3-gcc-12.3.0-gcc-runtime-12.3.0-brkks4glphmqjotqtuc652bbuh4t2qae.spack
==> Extracting gcc-runtime-12.3.0-brkks4glphmqjotqtuc652bbuh4t2qae from binary cache
==> gcc-runtime: Successfully installed gcc-runtime-12.3.0-brkks4glphmqjotqtuc652bbuh4t2qae
  Search: 0.00s.  Fetch: 0.32s.  Install: 0.36s.  Extract: 0.30s.  Relocate: 0.04s.  Total: 0.68s
[+] /global/homes/e/elvis/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-12.3.0/gcc-runtime-12.3.0-brkks4glphmqjotqtuc652bbuh4t2qae
==> Installing gmake-4.4.1-bxti4iwawnm7ba76iin6qcnumhmfatud [3/4]
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-v0.22/build_cache/linux-sles15-zen3-gcc-12.3.0-gmake-4.4.1-bxti4iwawnm7ba76iin6qcnumhmfatud.spec.json.sig
gpg: Signature made Wed 22 May 2024 09:01:38 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-v0.22/build_cache/linux-sles15-zen3/gcc-12.3.0/gmake-4.4.1/linux-sles15-zen3-gcc-12.3.0-gmake-4.4.1-bxti4iwawnm7ba76iin6qcnumhmfatud.spack
==> Extracting gmake-4.4.1-bxti4iwawnm7ba76iin6qcnumhmfatud from binary cache
==> gmake: Successfully installed gmake-4.4.1-bxti4iwawnm7ba76iin6qcnumhmfatud
  Search: 0.00s.  Fetch: 0.02s.  Install: 0.07s.  Extract: 0.04s.  Relocate: 0.03s.  Total: 0.10s
[+] /global/homes/e/elvis/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-12.3.0/gmake-4.4.1-bxti4iwawnm7ba76iin6qcnumhmfatud
==> Installing papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r [4/4]
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-v0.22/build_cache/linux-sles15-zen3-gcc-12.3.0-papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r.spec.json.sig
gpg: Signature made Wed 22 May 2024 09:19:22 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-v0.22/build_cache/linux-sles15-zen3/gcc-12.3.0/papi-7.1.0/linux-sles15-zen3-gcc-12.3.0-papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r.spack
==> Extracting papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r from binary cache
==> papi: Successfully installed papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r
  Search: 0.00s.  Fetch: 0.04s.  Install: 1.12s.  Extract: 1.03s.  Relocate: 0.08s.  Total: 1.16s
[+] /global/homes/e/elvis/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-12.3.0/papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r
==> Updating view at /global/homes/e/elvis0/e4s-demo/.spack-env/view
/global/homes/e/elvis/spack-workspace/perlmutter/software/linux-sles15-zen3/gcc-12.3.0/papi-7.1.0-mpl3rc6bu7wk6v3kvfhp5pqj4veefk3r
modules:
  default:
    roots:
      tcl: $HOME/spack-workspace/perlmutter/modules
    tcl:
      naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
      all:
        conflict:
        - '{name}'
        environment:
          set:
            '{name}_ROOT': '{prefix}'
==> Regenerating tcl module files
/global/homes/e/elvis/spack-workspace/perlmutter/modules/linux-sles15-zen3/papi/7.1.0-gcc-12.3.0-mpl3rc6

Due to MODULEPATH changes, the following have been reloaded:
1) gpu/1.0

/global/homes/e/elvis/spack-workspace/perlmutter/modules/linux-sles15-zen3:
papi/7.1.0-gcc-12.3.0-mpl3rc6
/opt/cray/pe/lmod/modulefiles/core:
papi/7.0.0.1
papi/7.0.0.2
papi/7.0.1.1
papi/7.0.1.2

Bootstrapping Spack¶

There is a known issue with Spack unable to bootstrap clingo on Perlmutter see https://github.com/spack/spack/issues/28315. The main issue is getting clingo installed which must be in your python environment. In order to get around this we have provided a script you can source which will create a python environment in your $HOME directory and install clingo. You can source the following script based on your shell type.

Note

If you already have clingo in your python environment you don't need to source this script.

# bash, sh, zsh users
source $SPACK_ROOT/bin/spack-setup.sh

# csh or tcsh users
source $SPACK_ROOT/bin/spack-setup.csh

We recommend you run the following to see if Spack will concretize a spec and not try to bootstrap clingo. If you are in a Spack environment, please deactivate the environment using despacktivate command. You can confirm your active environment by running spack env status. Next try running spack spec zlib to check if Spack will concretize zlib which will use clingo installed in your Python environment.

(.spack-pyenv) elvis@perlmutter> despacktivate
(.spack-pyenv) elvis@perlmutter> spack env status
==> No active environment

(.spack-pyenv) elvis@perlmutter> spack spec zlib
==> Warning: cannot detect libc from gcc@=13.2.0. The compiler will not be used during concretization.
==> Warning: cannot detect libc from gcc@=10.3.0. The compiler will not be used during concretization.
==> Warning: cannot detect libc from gcc@=11.2.0. The compiler will not be used during concretization.
Input spec
--------------------------------
 -   zlib

Concretized
--------------------------------
 -   zlib@1.3.1%gcc@12.3.0+optimize+pic+shared build_system=makefile arch=linux-sles15-zen3
[+]      ^gcc-runtime@12.3.0%gcc@12.3.0 build_system=generic arch=linux-sles15-zen3
[e]      ^glibc@2.31%gcc@12.3.0 build_system=autotools arch=linux-sles15-zen3
[+]      ^gmake@4.4.1%gcc@12.3.0~guile build_system=generic arch=linux-sles15-zen3

We recommend you create an environment in your user space to install your Spack stack. This can be done using the spack env create -d <dir> command. This will create a directory with an empty spack.yaml.

In example below, we will create an environment in our home directory called e4s-demo and activate the spack environment.

(.spack-pyenv) elvis@perlmutter> spack env create -d $HOME/e4s-demo
==> Created environment in /global/homes/s/elvis/e4s-demo
==> You can activate this environment with:
==>   spack env activate /global/homes/s/elvis/e4s-demo

(.spack-pyenv) elvis@perlmutter> spack env activate $HOME/e4s-demo

(.spack-pyenv) elvis@perlmutter> ls -l $HOME/e4s-demo
total 1
-rw-rw---- 1 elvis elvis 199 Jul 19 12:10 spack.yaml

Querying Spack Configuration¶

The spack config command can be used to query Spack configuration. Spack has several configuration scopes where configuration can be pulled. The configuration scope that is most relevant to you are site, user and defaults.

You can use spack config get [section] to query the configuration section. If you are unsure of all the configuration sections, you can run

spack config list

If you want to query the package configuration, you can run spack config get packages. The spack config --scope option can be useful to query by configuration scope. We have set environment $SPACK_SYSTEM_CONFIG_PATH in our e4s modules that provides our global spack settings for all e4s stacks. The environment should be set to the following which should contain the following configuration files:

elvis@perlmutter> echo  $SPACK_SYSTEM_CONFIG_PATH
/global/common/software/spackecp/perlmutter/spack_settings

elvis@perlmutter> ls -l $SPACK_SYSTEM_CONFIG_PATH
total 10
-rw-rw-r-- 1 e4s spackecp 3609 Mar  7 07:27 compilers.yaml
-rw-rw-r-- 1 e4s spackecp 4494 Mar  7 07:27 packages.yaml

This Spack instance is configured to install packages in your HOME directory, the Spack site configuration can be found in this directory $SPACK_ROOT/etc/spack/ which applies configuration to all users but this can be overridden in your spack.yaml. Take note of the following settings below for build stage, and installation tree.

(.spack-pyenv) elvis@perlmutter> cat $SPACK_ROOT/etc/spack/config.yaml
config:
  build_stage: $HOME/spack-workspace/perlmutter/build_stage
  misc_cache: $HOME/spack-workspace/perlmutter/misc_cache
  install_tree:
    root: $HOME/spack-workspace/perlmutter/software
  source_cache: $HOME/.spack/cache

If you plan on installing a large software stack, we recommend you use $CFS since it will have more storage space.

Installing packages¶

In the example above, we had the following spack configuration that was responsible for installing papi as shown below.

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs:
  - papi
  view: true

If you want to add more packages in your environment, just add them under the specs keyword. The spack configuration file (spack.yaml) is meant to be customized to your preference. You can override any global configurations in your spack.yaml.

Creating Modulefiles via spack¶

In this section, we will explain the module generation step via spack. We recommend you review the modules documentation in spack. The spack configuration for modules is defined in modules.yaml.

Shown below is the site configuration for modules.yaml, we have configured the module root to be $HOME/spack-workspace/perlmutter/modules where TCL modules will be generated. Furthermore, we have customized the TCL module setting when generating modules. You are welcome to override these configurations in your spack.yaml.

(.spack-pyenv) elvis@perlmutter> cat $SPACK_ROOT/etc/spack/modules.yaml
modules:
  default:
    roots:
      tcl: $HOME/spack-workspace/perlmutter/modules
    tcl:
      naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
      all:
        conflict:
        - '{name}'
        environment:
          set:
            '{name}_ROOT': '{prefix}'

We can generate the modules using spack module tcl refresh, this will generate the modules in the directory. The --delete-tree will delete the existing modules in the directory. The -y option will automatically answer yes to all questions.

(.spack-pyenv) elvis@perlmutter> spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files
(.spack-pyenv) elvis@perlmutter> spack module tcl find --full-path papi
/global/homes/e/elvis/spack-workspace/perlmutter/modules/linux-sles15-zen3/papi/6.0.0.1-gcc-11.2.0-frkekrt

Note

You should not copy or update modulefiles in this directory since they will be overwritten when you run spack module tcl refresh.

If you want to add view the newly created modules, just add the directory path to MODULEPATH by running the following:

(.spack-pyenv) elvis@perlmutter> module use $HOME/spack-workspace/perlmutter/modules/$(spack arch)

Now we see the modulefile papi/6.0.0.1-gcc-11.2.0-frkekrt is visible and can be loaded as any other software package.

(.spack-pyenv) elvis@perlmutter> ml -t av papi
/global/homes/e/elvis/spack-workspace/perlmutter/modules/linux-sles15-zen3:
papi/6.0.0.1-gcc-11.2.0-frkekrt
/opt/cray/pe/lmod/modulefiles/core:
papi/6.0.0.16
papi/7.0.0.1

Binary Cache Mirror¶

We have setup a binary cache mirror named perlmutter-e4s-23.05 that can be added into your spack.yaml.

You can check this by running spack mirror list to see the list of mirrors.

elvis@perlmutter>  spack mirror list
perlmutter-e4s-23.05    file:///global/common/software/spackecp/mirrors/perlmutter-e4s-23.05
source_mirror           file:///global/cfs/cdirs/m3503/mirrors/source_mirror
spack-public            https://mirror.spack.io

Spack has support signing and verifying packages via GPG Key which is required when installing packages from the binary cache. To learn more about the binary cache, please see https://spack-tutorial.readthedocs.io/en/latest/tutorial_binary_cache.html#setting-up-a-binary-cache-mirror

First, you will need to install the GPG key locally, note this is only required once. Please run the following command.

elvis@perlmutter> spack buildcache keys -it
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/_pgp/B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F.pub
gpg: key 3BD6B0E9935AEB8F: public key "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: inserting ownertrust of 6

You can see list of gpg keys by running gpg -k make sure you see the following entry in the output.

pub   rsa2048 2022-02-23 [SC]
      B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
uid           [ultimate] GPG Key - e4s <shahzebsiddiqui@lbl.gov>
sub   rsa2048 2022-02-23 [E]

If you have successfully imported the gpg key you should be able to run spack buildcache list to see all the specs in the binary cache.

We have set the environment variable SPACK_GNUPGHOME which points to $HOME/.gnupg in case you run any commands like spack gpg which will typically fail since Spack attempts to write GPG keys into the Spack instance at $SPACK_ROOT/spack/opt/gpg which you won't have access to.

If you are still in your environment that you created in section Developer Workflow, let's install a package from the binary cache, you will notice that Spack will install packages from the binary cache instead of building from source.

elvis@perlmutter> spack install raja@0.14.0~cuda+examples+exercises~ipo+openmp~rocm+shared~tests build_type=RelWithDebInfo
==> Installing pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0/cray-sles15-zen3-gcc-11.2.0-pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q.spack
==> Extracting pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q from binary cache
gpg: Signature made Fri 14 Oct 2022 08:12:17 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q
==> Installing berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/cray-sles15-zen3/gcc-11.2.0/berkeley-db-18.1.40/cray-sles15-zen3-gcc-11.2.0-berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3.spack
==> Extracting berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3 from binary cache
gpg: Signature made Thu 13 Oct 2022 11:52:15 AM PDT
gpg:                using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3

...

Need Help with Spack¶

If you need help with Spack, please join the Spack Slack Channel, once you have registered you can go to slack workspace at https://spackpm.slack.com/.