Spack¶
Spack is a package manager for supercomputers, Linux, and macOS. Spack is designed to support multiple versions and configurations of software on a wide variety of platforms and environments. Spack is non-destructive: installing a new version of a package does not break existing installations, so many configurations of the same package can coexist.
Spack Commands¶
Shown below are some of the most common Spack commands you will use. For a complete list of commands, see the Spack Command Reference.
Command | Description |
---|---|
spack install <spec> | Install a package by spec. |
spack find | List installed packages. |
spack list | List available packages. |
spack load <spec> | Load a package into the current environment. |
spack unload <spec> | Unload a package from the current environment. |
spack load --list | List packages loaded into the current environment. |
spack env create <name> | Create a new environment. |
spack env activate <name> | Activate an environment. |
spack env deactivate | Deactivate the current environment. |
spack env status | Show the status of the current environment. |
spack env create -d <dir> | Create an environment in a directory. |
spack env activate -d <dir> | Activate an environment in a directory. |
spack concretize -f | Force concretize an environment. |
spack spec <spec> | Show what would be installed given a spec. |
spack spec -Il <spec> | Show the install status and dependencies with hashes. |
spack buildcache list | List available binary packages. |
spack info <spec> | Show information about a package. |
spack edit <spec> | Open a package file in an editor. |
spack config get <section> | Show the YAML configuration for a section. |
spack config list | List all configuration scopes. |
spack config blame <section> | Show the YAML configuration file and line numbers for a section. |
Loading spack packages into your user environment¶
Unlike module load
that is used for loading modules into your user environment, in Spack you must use spack load
to load a Spack package into your user environment.
Use Case Example: Loading HDF5¶
First, let's activate the gcc
environment by running the following:
spack env activate -V gcc
Let's assume you want to use HDF5, first you need to determine which Spack package you want to load. We can run the following to show the installed versions of HDF5.
spack find hdf5
This will output a list of installed packages as shown:
==> 5 installed packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
hdf5@1.8.22 hdf5@1.10.7 hdf5@1.10.8 hdf5@1.12.2 hdf5@1.12.2
Let's load HDF5 version 1.10.7 that was compiled with GCC version 11.2.0 into our user environment. To specify the version use @
and to specify the compiler use %
. These symbols can also be combined as is the case with loading GCC version 11.2.0. Take note that Spack will load the dependencies in your user environment which is the default behavior.
elvis@perlmutter> spack load hdf5@1.10.7%gcc@11.2.0
elvis@perlmutter> spack load --list
==> 20 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6 cray-mpich@8.1.22 expat@2.4.8 libarchive@3.5.2 libiconv@1.16 libuv@1.44.1 lzo@2.10 ncurses@6.1 rhash@1.4.2 zlib@1.2.12
cmake@3.23.1 curl@7.66.0 hdf5@1.10.7 libbsd@0.11.5 libmd@1.0.4 lz4@1.9.3 mbedtls@2.28.0 pkgconf@1.8.0 xz@5.2.5 zstd@1.5.2
Your user environment will be updated. That includes PATH
, LD_LIBRARY_PATH
, and other variables relevant for loading the package. The HDF5 package provides the h5cc
program which will appear in your environment as shown below.
elvis@perlmutter> which h5cc
/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne/h5cc
dir is /global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne
gcc (GCC) 11.2.0 20210728 (Cray Inc.)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
This confirms that HDF5 was successfully loaded.
View Shell Commands to Load the Package¶
It can be informative to view the commands that Spack uses when a package is loaded. For example, to see the shell commands used when zlib
is loaded, use spack load --sh zlib%gcc@11.2.0
.
elvis@perlmutter> spack load --sh zlib%gcc@11.2.0
export CMAKE_PREFIX_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q:/opt/cray/pe/mpich/8.1.22/ofi/gnu/9.1:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/cmake-3.23.1-p23fzuowp4yuitemelic7f65nwybthxd:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libuv-1.44.1-gitz4v6soymrlpymzvygqlqibktfsp7a:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zstd-1.5.2-rtaozj7xdfhxengzppbqbth4mya3edpq:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/mbedtls-2.28.0-ucarxpp47pj7yo7abt5povg3swygev3q:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lzo-2.10-rlpwurcqberlfjlpk62go3arnneonv6u:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libiconv-1.16-3xwztgyaaco4thqqhacdst65mmnr5is3:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/expat-2.4.8-yvchkv23jw7po7meyuxckpre5uroipmd:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog:/global/common/software/nersc/pm-2021q4/easybuild/software/Nsight-Systems/2022.2.1:/global/common/software/nersc/pm-2021q4/easybuild/software/Nsight-Compute/2022.1.1;
export LD_LIBRARY_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q/lib:/opt/cray/pe/mpich/8.1.22/ofi/gnu/9.1/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libuv-1.44.1-gitz4v6soymrlpymzvygqlqibktfsp7a/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zstd-1.5.2-rtaozj7xdfhxengzppbqbth4mya3edpq/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/mbedtls-2.28.0-ucarxpp47pj7yo7abt5povg3swygev3q/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lzo-2.10-rlpwurcqberlfjlpk62go3arnneonv6u/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libiconv-1.16-3xwztgyaaco4thqqhacdst65mmnr5is3/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/expat-2.4.8-yvchkv23jw7po7meyuxckpre5uroipmd/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74/lib:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog/lib:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/math_libs/11.7/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/extras/CUPTI/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/extras/Debugger/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/nvvm/lib64:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/lib64:/global/common/software/nersc/pm-2022q3/sw/darshan/3.4.0/lib:/opt/cray/pe/papi/6.0.0.16/lib64:/opt/cray/pe/gcc/11.2.0/snos/lib64:/opt/cray/libfabric/1.15.2.0/lib64;
export MANPATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm/share/man:/usr/share/man:/usr/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libiconv-1.16-3xwztgyaaco4thqqhacdst65mmnr5is3/share/man:/usr/share/man:/usr/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74/share/man:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog/share/man:/usr/share/man:/usr/man:/opt/nvidia/hpc_sdk/Linux_x86_64/22.5/cuda/11.7/doc/man:/opt/cray/pe/perftools/22.09.0/man:/opt/cray/pe/papi/6.0.0.16/share/pdoc/man:/opt/cray/pe/gcc/11.2.0/snos/share/man:/opt/cray/pe/craype/2.7.19/man:/opt/cray/pe/mpich/8.1.22/ofi/man:/opt/cray/pe/mpich/8.1.22/man/mpich:/opt/cray/pe/libsci/22.11.1.2/man:/opt/cray/pe/man/csmlversion:/opt/cray/pe/dsmml/0.2.2/dsmml/man:/opt/cray/libfabric/1.15.2.0/share/man:/usr/share/lmod/lmod/share/man:/usr/local/man:/usr/share/man:/usr/man:;
export PKG_CONFIG_PATH=/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/hdf5-1.10.7-6opvtfcoaxox7oskb7dzisnb32fnk5ne/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q/lib/pkgconfig:/opt/cray/pe/mpich/8.1.22/ofi/gnu/9.1/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/rhash-1.4.2-xjxbsefahiue4isqlzwcobgc7sgfhmqm/lib/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libuv-1.44.1-gitz4v6soymrlpymzvygqlqibktfsp7a/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libarchive-3.5.2-d4gpdx2rxj4calconu7cd6kgq2ev5roe/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zstd-1.5.2-rtaozj7xdfhxengzppbqbth4mya3edpq/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/zlib-1.2.12-ozmcyfjfv7i5gjjgklfsh43h67vzsuc5/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/xz-5.2.5-tyswlp4epqidsxgtgxyefciqoz2ju6oe/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lzo-2.10-rlpwurcqberlfjlpk62go3arnneonv6u/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/lz4-1.9.3-xxbkaphnofa6gxugxszqr5kzew7chrqa/lib/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/expat-2.4.8-yvchkv23jw7po7meyuxckpre5uroipmd/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libbsd-0.11.5-uomjj7io3hhggbfyzofrw5jpw5chlv74/lib/pkgconfig:/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/libmd-1.0.4-iynqjjkz4jgtxagqy2o2tvspqfn3yvog/lib/pkgconfig:/usr/share/pkgconfig:/usr/lib64/pkgconfig:/usr/lib/pkgconfig:/opt/modulefiles/cudatoolkit:/global/common/software/nersc/pm-2022q3/sw/darshan/3.4.0/lib/pkgconfig:/opt/cray/pe/craype/2.7.19/pkg-config:/opt/cray/pe/dsmml/0.2.2/dsmml/lib/pkgconfig:/opt/cray/libfabric/1.15.2.0/lib64/pkgconfig;
export SPACK_LOADED_HASHES=ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:6opvtfcoaxox7oskb7dzisnb32fnk5ne:nugfov2wsvpllczfy6ff7bytzt4oi34q:6yvclv72nfvhlx2fj72n5npigrgouojd:p23fzuowp4yuitemelic7f65nwybthxd:xjxbsefahiue4isqlzwcobgc7sgfhmqm:i6ri5efvxr243dakip6bk7n6b3qfruyb:gitz4v6soymrlpymzvygqlqibktfsp7a:d4gpdx2rxj4calconu7cd6kgq2ev5roe:rtaozj7xdfhxengzppbqbth4mya3edpq:ozmcyfjfv7i5gjjgklfsh43h67vzsuc5:tyswlp4epqidsxgtgxyefciqoz2ju6oe:ucarxpp47pj7yo7abt5povg3swygev3q:rlpwurcqberlfjlpk62go3arnneonv6u:xxbkaphnofa6gxugxszqr5kzew7chrqa:3xwztgyaaco4thqqhacdst65mmnr5is3:4g7s6qpmt2nsclyi2d2ndfek2ptey4m6:yvchkv23jw7po7meyuxckpre5uroipmd:uomjj7io3hhggbfyzofrw5jpw5chlv74:iynqjjkz4jgtxagqy2o2tvspqfn3yvog:zkrv7nh3jglxsimtfzld7vo4mhdqlh3u;
Find the Install Prefix for a Package¶
For your application you may want to specify the location of the installed package as an environment variable. In this case, let's set HDF5_DIR
to the parent directory of libs
and include
--the install prefix for our HDF5 package. A convenient way to do this is as follows,
elvis@perlmutter> export HDF5_DIR=$(spack location -i hdf5@1.10.7%gcc@11.2.0)
Unloading Spack Packages From Your Environment¶
You can use spack unload
to remove packages from your environment which functions similarly to module unload
. spack unload
will only work on installed specs that were loaded in your environment. In the example below, we show how to unload the cmake
Spack package from the user environment. Take note as we unload cmake via spack unload cmake
the user environment was updated and it now resorts to the system cmake /usr/bin/cmake
elvis@perlmutter> spack load --list
==> 20 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6 cray-mpich@8.1.22 expat@2.4.8 libarchive@3.5.2 libiconv@1.16 libuv@1.44.1 lzo@2.10 ncurses@6.1 rhash@1.4.2 zlib@1.2.12
cmake@3.23.1 curl@7.66.0 hdf5@1.10.7 libbsd@0.11.5 libmd@1.0.4 lz4@1.9.3 mbedtls@2.28.0 pkgconf@1.8.0 xz@5.2.5 zstd@1.5.2
elvis@perlmutter> which cmake
/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/opt/spack/cray-sles15-zen3/gcc-11.2.0/cmake-3.23.1-p23fzuowp4yuitemelic7f65nwybthxd/bin/cmake
elvis@perlmutter> spack unload xz@5.2.5
elvis@perlmutter> which cmake
/usr/bin/cmake
elvis@perlmutter> spack load --list
==> 19 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6 cray-mpich@8.1.22 curl@7.66.0 expat@2.4.8 hdf5@1.10.7 libarchive@3.5.2 libbsd@0.11.5 libiconv@1.16 libmd@1.0.4 libuv@1.44.1 lz4@1.9.3 lzo@2.10 mbedtls@2.28.0 ncurses@6.1 pkgconf@1.8.0 rhash@1.4.2 xz@5.2.5 zlib@1.2.12 zstd@1.5.2
If you want to remove all packages from your user environment you can run the following:
elvis@perlmutter> spack unload --all
elvis@perlmutter> spack load --list
==> 0 loaded packages
Using Hashes for Packages with Multiple Versions¶
Sometimes, you will need to use hashes instead of specs to load a specific package variant. Let's assume we want to load CMake and we get the error, "Use a more specific spec
". Note that in this example we have three specs for same version but each has a different hash (The 7 character code proceeding the spec).
Error
elvis@perlmutter> spack load cmake@3.23.1%gcc@11.2.0
==> Error: cmake@3.23.1%gcc@11.2.0 matches multiple packages.
Matching packages:
p23fzuo cmake@3.23.1%gcc@11.2.0 arch=cray-sles15-zen3
6v4d2hu cmake@3.23.1%gcc@11.2.0 arch=cray-sles15-zen3
Use a more specific spec.
In this case, we can use the hash to uniquely identify the desired variant of CMake. For example, if we want to load the spec with hash p23fzuo, we would run the following:
elvis@perlmutter> spack load /p23fzuo
Loading a Spack Package Without Dependencies¶
The command spack load
will load a package and its dependencies which is recommended when using software, however sometimes you don't want to load all the extra dependencies in your environment. Spack provides a way to load only the package via --only package
which will load the package and not its dependencies. In the example shown below, we load one package. Take note of this compared to its normal behavior which loads all dependencies.
elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> spack load --only package /p23fzuo
elvis@perlmutter> spack load --list
==> 1 loaded package
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
cmake@3.23.1
elvis@perlmutter> spack unload -a
elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> spack load /p23fzuo
elvis@perlmutter> spack load --list
==> 17 loaded packages
-- cray-sles15-zen3 / gcc@11.2.0 --------------------------------
bzip2@1.0.6 cmake@3.23.1 curl@7.66.0 expat@2.4.8 libarchive@3.5.2 libbsd@0.11.5 libiconv@1.16 libmd@1.0.4 libuv@1.44.1 lz4@1.9.3 lzo@2.10 mbedtls@2.28.0 ncurses@6.1 rhash@1.4.2 xz@5.2.5 zlib@1.2.12 zstd@1.5.2
Recommended setup using environments¶
The e4s/<version>
modules comes with several environments such as: gcc
, nvhpc
, cce
, cuda
. The cuda
environment provides packages built with CUDA support with the gcc@11.2.0
compiler. Depending on the environment you are using, please consider loading the appropriate modules
- Load the
PrgEnv-gnu
,cpu
module when usinggcc
environment - Load the
PrgEnv-cce
,cpu
module when usingcce
environment - Load the
PrgEnv-nvhpc
,cpu
module when usingnvhpc
environment - Load the
PrgEnv-gnu
,gpu
module when usingcuda
environment
When in doubt, please refer to the compiler settings to determine the list of modules used. To retrieve a list of compilers you can do spack compiler list
. To query a compiler instance like gcc@11.2.0
you can run spack compiler info gcc@11.2.0
.
In the cuda
environment, you will see packages are built with variants +cuda cuda_arch=80
that are optimized for NVIDIA A100 GPUs (+cuda cuda_arch=80
). You should make sure the craype-accel-nvidia
modulefile is loaded. Usually this is loaded with the gpu
modulefile at login. You can check the accelerator target by running the following:
elvis@perlmutter> echo $CRAY_ACCEL_TARGET
nvidia80
Developer Workflow¶
Shown below is a guide to get you started building packages with our Spack instance. First, we recommend you source the following file which will activate a python environment and install clingo
.
# bash, sh, zsh users
source $SPACK_ROOT/bin/spack-setup.sh
# csh or tcsh users
source $SPACK_ROOT/bin/spack-setup.csh
We recommend you run the following to see if Spack will concretize a spec and not try to bootstrap clingo. If you are in a Spack environment, please deactivate the environment using despacktivate
command. You can confirm your active environment by running spack env status
. Next try running spack spec zlib
to check if Spack will concretize zlib
which will use clingo
installed in your Python environment.
(.spack-pyenv) elvis@perlmutter> despacktivate
(.spack-pyenv) elvis@perlmutter> spack env status
==> No active environment
(.spack-pyenv) elvis@perlmutter> spack spec zlib
Input spec
--------------------------------
zlib
Concretized
--------------------------------
zlib@1.2.12%gcc@11.2.0+optimize+pic+shared patches=0d38234 arch=cray-sles15-zen3
We recommend you create a environment in your user space to install your Spack stack. This can be done using the spack env create -d <dir>
command. This will create a directory with an empty spack.yaml
. Now you can activate your newly created environment using the -d
option to specify a directory. This will create a directory with an empty spack.yaml
.
In example below, we will create a environment in our home directory called e4s-22.05-demo
.
(.spack-pyenv) elvis@perlmutter> spack env create -d $HOME/e4s-22.05-demo
==> Created environment in /global/homes/s/elvis/e4s-22.05-demo
==> You can activate this environment with:
==> spack env activate /global/homes/s/elvis/e4s-22.05-demo
(.spack-pyenv) elvis@perlmutter> spack env activate $HOME/e4s-22.05-demo
(.spack-pyenv) elvis@perlmutter> ls -l $HOME/e4s-22.05-demo
total 1
-rw-rw---- 1 elvis elvis 199 Jul 19 12:10 spack.yaml
Querying Spack Configuration¶
The spack config
command can be used to query Spack configuration. Spack has several configuration scopes where configuration can be pulled. The configuration scope that is most relevant to you are site
, user
and defaults
.
You can use spack config get [section]
to query the configuration section. If you are unsure of all the configuration sections, you can run
spack config list
If you want to query the package configuration, you can run spack config get packages
. The spack config --scope
option can be useful to query by configuration scope. We have set environment $SPACK_SYSTEM_CONFIG_PATH
in our e4s modules that provides our global spack settings for all e4s stacks. The environment should be set to the following which should contain the following configuration files:
elvis@perlmutter> echo $SPACK_SYSTEM_CONFIG_PATH
/global/common/software/spackecp/perlmutter/spack_settings
elvis@perlmutter> ls -l $SPACK_SYSTEM_CONFIG_PATH
total 10
-rw-rw-r-- 1 e4s spackecp 3609 Mar 7 07:27 compilers.yaml
-rw-rw-r-- 1 e4s spackecp 4494 Mar 7 07:27 packages.yaml
This Spack instance is configured to install packages in your HOME
directory, the Spack site configuration can be found in this directory $SPACK_ROOT/etc/spack/
which applies configuration to all users but this can be overridden in your spack.yaml
. Take note of the following settings below for build stage, and installation tree. Note that HOME
directory is shared between Cori and Perlmutter, therefore we organize user installation by system. We recommend you adopt a similar naming scheme if you plan on building Spack packages on both systems.
(.spack-pyenv) elvis@perlmutter> cat $SPACK_ROOT/etc/spack/config.yaml
config:
build_stage: $HOME/spack-workspace/perlmutter/build_stage
misc_cache: $HOME/spack-workspace/perlmutter/misc_cache
install_tree:
root: $HOME/spack-workspace/perlmutter/software
source_cache: $HOME/.spack/cache
Installing packages¶
Now let's add the following package in your spack.yaml
and try building it via Spack. We will build papi
. Shown below is the content of spack.yaml
, please copy this content to your spack.yaml
.
# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
# add package specs to the `specs` list
specs:
- papi
view: true
Note we don't provide a buildcache so papi
will be built from source. Now let's install all packages in your environment by running spack install
.
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack install
==> Starting concretization
==> Environment concretized in 13.19 seconds.
==> Concretized papi
- s2y4nrv papi@6.0.0.1%gcc@11.2.0~cuda+example~infiniband~lmsensors~nvml~powercap~rapl~rocm~rocm_smi~sde+shared~static_tools arch=cray-sles15-zen3
==> Installing environment /global/homes/s/elvis/e4s-22.05-demo
==> Installing papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
==> No binary for papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af found: installing from source
==> Using cached archive: /global/homes/s/elvis/.spack/cache/_source-cache/archive/3c/3cd7ed50c65b0d21d66e46d0ba34cd171178af4bbf9d94e693915c1aca1e287f.tar.gz
==> No patches needed for papi
==> papi: Executing phase: 'autoreconf'
==> papi: Executing phase: 'configure'
==> papi: Executing phase: 'build'
==> papi: Executing phase: 'install'
==> papi: Successfully installed papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
Fetch: 0.01s. Build: 25.80s. Total: 25.81s.
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
==> Updating view at /global/homes/s/elvis/e4s-22.05-demo/.spack-env/view
We can find the root directory for papi
by running the following. We see the package was installed in our home directory.
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack location -i papi
/global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/papi-6.0.0.1-s2y4nrvu6whr6hhgi63aa3nqwz2d35af
Creating Modulefiles via spack¶
In this section, we will create modules via spack. We recommend you review the modules documentation in spack. The spack configuration for modules is defined in modules.yaml
.
Shown below is the site configuration for modules.yaml
, we have configured the module root to be $HOME/spack-workspace/perlmutter/modules
where TCL modules will be generated. Furthermore we have customized the TCL module setting when generating modules. You are welcome to override these configurations in your spack.yaml
.
(.spack-pyenv) elvis@perlmutter> cat $SPACK_ROOT/etc/spack/modules.yaml
modules:
default:
roots:
tcl: $HOME/spack-workspace/perlmutter/modules
tcl:
naming_scheme: '{name}/{version}-{compiler.name}-{compiler.version}'
all:
conflict:
- '{name}'
environment:
set:
'{name}_ROOT': '{prefix}'
We can create the module tree and retrieve the location of our generated module as shown below.
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack module tcl refresh --delete-tree -y
==> Regenerating tcl module files
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> spack module tcl find --full-path papi
/global/homes/s/elvis/spack-workspace/perlmutter/modules/cray-sles15-zen3/papi/6.0.0.1-gcc-11.2.0-s2y4nrv
If you want to add this to MODULEPATH you can simply run the following:
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> module use $HOME/spack-workspace/perlmutter/modules/$(spack arch)
Now we see the modulefile papi/6.0.0.1-gcc-11.2.0-s2y4nrv
is visible and can be loaded as any other software package.
(.spack-pyenv) elvis@perlmutter:~/e4s-22.05-demo> ml -t av papi
/global/homes/e/elvis/spack-workspace/perlmutter/modules/cray-sles15-zen3:
papi/6.0.0.1-gcc-11.2.0-s2y4nrv
/opt/cray/pe/lmod/modulefiles/core:
papi/6.0.0.16
papi/7.0.0.1
Spack Views¶
Spack environments can define filesystem views to directly access the software without having to load software.
If you are in an existing environment, you will want to deactivate and reactivate environment with view.
elvis@perlmutter> despacktivate
In this example, let's enable Spack view for nvhpc
spack environment
Note
Activating an environment with view may take some time depending on size of environment. If you run spack env activate <env>
, it will activate the environment with a view. We recommend you don't enable view since it may end up corrupting your user environment with the entire stack.
Spack View Enabled¶
elvis@perlmutter> spack env activate --with-view nvhpc
or
elvis@perlmutter> spack env activate nvhpc
Spack View Disabled¶
elvis@perlmutter> spack env activate -V nvhpc
Spack will automatically load everything in the view, Take note, we have 0 loaded packages in the output of spack load --list
, however we can access software from the view since $PATH, $LD_LIBRARY_PATH, etc... has been updated in your user environment. Spack will present software in a view similar to how one access software in Linux in /usr/local
or /usr/bin
.
elvis@perlmutter> spack load --list
==> 0 loaded packages
elvis@perlmutter> which h5cc
/global/common/software/spackecp/perlmutter/e4s-22.05/78535/spack/var/spack/environments/nvhpc/views/default/hdf5/1.10.7-nvhpc-22.5/bin/h5cc
elvis@perlmutter> which cmake
/global/common/software/spackecp/perlmutter/e4s-22.05/75197/spack/views/default/cmake/3.23.1-gcc-11.2.0/bin/cmake
Now you may be wondering which packages you can access with Spack views. We have selected packages in view to avoid conflicts during view generation. For instance you can't have two instances of cmake
in the same view.
Binary Cache Mirror¶
We have setup a binary cache mirror named perlmutter-e4s-22.05 that can be added into your spack.yaml
.
You can check this by running spack mirror list
to see the list of mirrors.
elvis@perlmutter> spack mirror list
perlmutter-e4s-22.05 file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05
source_mirror file:///global/cfs/cdirs/m3503/mirrors/source_mirror
spack-public https://mirror.spack.io
Spack has support signing and verifying packages via GPG Key which is required when installing packages from the binary cache. To learn more about the binary cache, please see https://spack-tutorial.readthedocs.io/en/latest/tutorial_binary_cache.html#setting-up-a-binary-cache-mirror
First, you will need to install the GPG key locally, note this is only required once. Please run the following command.
elvis@perlmutter> spack buildcache keys -it
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/_pgp/B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F.pub
gpg: key 3BD6B0E9935AEB8F: public key "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" imported
gpg: Total number processed: 1
gpg: imported: 1
gpg: inserting ownertrust of 6
You can see list of gpg keys by running gpg -k
make sure you see the following entry in the output.
pub rsa2048 2022-02-23 [SC]
B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
uid [ultimate] GPG Key - e4s <shahzebsiddiqui@lbl.gov>
sub rsa2048 2022-02-23 [E]
If you have successfully imported the gpg key you should be able to run spack buildcache list
to see all the specs in the binary cache.
We have set the environment variable SPACK_GNUPGHOME
which points to $HOME/.gnupg
in case you run any commands like spack gpg
which will typically fail since Spack attempts to write GPG keys into the Spack instance at $SPACK_ROOT/spack/opt/gpg
which you won't have access to.
If you are still in your environment that you created in section Developer Workflow, let's install a package from the binary cache, you will notice that Spack will install packages from the binary cache instead of building from source.
elvis@perlmutter> spack install raja@0.14.0~cuda+examples+exercises~ipo+openmp~rocm+shared~tests build_type=RelWithDebInfo
==> Installing pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0/cray-sles15-zen3-gcc-11.2.0-pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q.spack
==> Extracting pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q from binary cache
gpg: Signature made Fri 14 Oct 2022 08:12:17 AM PDT
gpg: using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/pkgconf-1.8.0-nugfov2wsvpllczfy6ff7bytzt4oi34q
==> Installing berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3
==> Fetching file:///global/common/software/spackecp/mirrors/perlmutter-e4s-22.05/build_cache/cray-sles15-zen3/gcc-11.2.0/berkeley-db-18.1.40/cray-sles15-zen3-gcc-11.2.0-berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3.spack
==> Extracting berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3 from binary cache
gpg: Signature made Thu 13 Oct 2022 11:52:15 AM PDT
gpg: using RSA key B5FDE18F615783AF078ED29C3BD6B0E9935AEB8F
gpg: Good signature from "GPG Key - e4s <shahzebsiddiqui@lbl.gov>" [ultimate]
[+] /global/homes/s/elvis/spack-workspace/perlmutter/software/cray-sles15-zen3/gcc-11.2.0/berkeley-db-18.1.40-hafeanvncfcwgstenlf5cc7ttyobmgt3
...
Need Help with Spack¶
If you need help with Spack, please join the Spack Slack Channel, once you have registered you can go to slack workspace at https://spackpm.slack.com/.