Skip to content

Spack - A package manager

Spack is a flexible package manager for HPC. It includes the following features:

  • Simple package installation - Spack installs the latest version of a package and all of its dependencies by default.

  • Custom versions and configurations - Spack allows installation to be customized. It offers a simple spec syntax so that users can specify the version, build compiler, compile-time options, cross-compile platform, and dependencies, all on the command line.

  • Non-destructive installs - Spack installs every unique package/dependency configuration into its own prefix (hash), so new installs will not break existing ones.

  • Peaceful coexistence of packages - Spack avoids library misconfiguration by using RPATH to link dependencies. When a user links a library or runs a program, it is tied to the dependencies it was built with, so there is no need to manipulate LD_LIBRARY_PATH at runtime.

  • Easy package creation - Spack stores the recipes for building packages in the package files that are written in pure Python, and specs allow package authors to maintain a single file for many different builds of the same package.

Spack module

While users can use their own installation of Spack (git clone https://github.com/spack/spack), we provide a Spack instance for users, which provides NERSC’s recommended, up-to-date Spack configurations on our systems. We have also configured it so that users can make use of the Spack installations by NERSC staff instead of building every dependent package redundantly.

To access the shared Spack instance, do

module load spack

This will do the following to your environment:

  • Allow access to the spack command and the recommended Spack configuration files at the $SPACK_ROOT/etc/spack directory.

  • Add Spack's shell support by sourcing the file $SPACK_ROOT/share/spack/setup-env.{sh,csh}. Note that this will also make the Spack generated modulefiles available in your shell environment.

  • Create a directory (at the first time invocation only), $HOME/sw, and a few subdirectories under it on your account to build and install software on. E.g, software will be installed on your $HOME/sw/opt/spack directory, and the Spack generated modulefiles will be available at the $HOME/sw/share/spack/modules directory. You can create your own package files under the directory $HOME/sw/var/spack/repos/$USER/packages. The directory $HOME/sw/build is for you to run the spack install command on and to save the build logs.

Note

Note that you can overwrite this global setting by placing your configurations on your ~/.spack directory. Please refer to the Spack Configuration Scope Page for scoping rules.

Building software using the Spack module

Spack is a powerful package management tool, and the learning curve for Spack can be steep. Please refer to the Spack tutorial and Spack documentation pages to learn how to use Spack to build software. In this section, we will cover how to use the Spack module provided by NERSC to build software on NERSC systems (currently only Cori is configured).

Spack supports about 1000s of packages. To see the full list of supported software by the Spack module on Cori, do

module load spack
spack list
Use the spack list <package name> to check if a specific package is supported or not.

To see the full list of Spack installed software on Cori, do

spack find
Use the spack find -p <package name> command to check if a specific package is already installed or not; the -p option of spack find shows where the package of interest is installed.

If the package of interest is not installed, you can install it with the following commands:

cd $HOME/sw/build
spack install <package name>
This will build the latest version of the package using the default Intel compiler for Cori Haswell and install it on your $HOME/sw/opt/spack directory. A modulefile will be created for the install at the $HOME/sw/share/spack/modules/cray-cnl7-haswell directory as well. Here was the output from installing the xerces-c package on Cori:

spack install xerces-c output on Cori
train467@cori09:~/sw/build> spack install xerces-c
[+] /global/common/sw/cray/cnl7/haswell/libiconv/1.16/intel/19.0.3.199/vr7hfhz
[+] /global/common/sw/cray/cnl7/haswell/libiconv/1.16/intel/19.0.3.199/vr7hfhz
==> 41416: Installing xerces-c
==> Fetching https://archive.apache.org/dist/xerces/c/3/sources/xerces-c-3.2.2.tar.bz2
################################################################################################################################################ 100.0%
==> Staging archive: /global/cscratch1/sd/train467/sw/spackbuild/spack-stage-xerces-c-3.2.2-ggdbuk52jm72rebq67s3ro2vxzaeozg4/xerces-c-3.2.2.tar.bz2
==> Created stage in /global/cscratch1/sd/train467/sw/spackbuild/spack-stage-xerces-c-3.2.2-ggdbuk52jm72rebq67s3ro2vxzaeozg4
==> No patches needed for xerces-c
==> 41416: xerces-c: Building xerces-c [AutotoolsPackage]
==> 41416: xerces-c: Executing phase: 'autoreconf'
==> 41416: xerces-c: Executing phase: 'configure'
==> 41416: xerces-c: Executing phase: 'build'
==> 41416: xerces-c: Executing phase: 'install'
==> 41416: xerces-c: Successfully installed xerces-c
  Fetch: 2.30s.  Build: 2m 34.05s.  Total: 2m 36.35s.
[+] /global/homes/t/train467/sw/opt/spack/cray-cnl7-haswell/intel-19.0.3.199/xerces-c-3.2.2-ggdbuk5

Each build has a unique hash associated with it. To see the detailed build info including hashes and dependencies, use the spack spec -l <package name> command. Here is an example output of spack spec -l.

Tip

You are recommended to link your installatinkson directory $HOME/sw/opt/spack to a directory under your /global/common/software/<your project> for performance (/global/common is a read-only file system on the compute nodes) and convenience (e.g., to share your installations with your group members). You can also link the $HOME/sw/share/spack/modules directory to /global/common/software/<your project> as well.

If spack install fails

While we expect many packages from the builtin package repository ($SPACK_ROOT/var/sparck/repos/builtin) can be built out of the box on our systems, your spack install command may fail with some packages (you are encouraged to open a github issue at the Spack github site). If this happens, you may have to modify the package.py files for the packages that fail to build. You can view a package’s package.py file in the bulitin repository with the spack edit <package name> command, but you don't have the write permission to directly modify it (note that this is a shared instance). You can copy the packge.py file over to your directory, $HOME/sw/var/spack/repos/$USER/packages, which is one of the directories that Spack searches for the package files, and make changes to your local copy using the spack edit command or directly using a text editor.

For example, if you need to install the cp2k package, you may need to do the following:

cp -pr $SPACK_ROOT/var/spack/repos/builtin/packages/cp2k $HOME/sw/var/spack/repos/$USER/packages
spack edit cp2k #or vi $HOME/sw/var/spack/repos/$USER/packages/cp2k/package.py 
#e.g, change line 448 to  mkf.write('AR = xiar -r\n\n'.format(intel_bin_dir))
cd $HOME/sw/build
spack install cp2k %intel blas=mkl +mpi ^intel-mkl arch=cray-cnl7-haswell

Maintenance of the NERSC package repository

The Spack module is configured to search for the package files in the following three directories (repositories):

train467@cori09:~/sw/build> spack config get repos
repos:
- $HOME/sw/var/spack/repos/$USER
- /global/common/sw/spack/0.14.2/var/spack/repos/nersc
- /global/common/sw/spack/0.14.2d/var/spack/repos/builtin
where the NERSC repository, /global/common/sw/spack/0.14.2/var/spack/repos/nersc directory, is for the packages that needed changes to work on Cori. You are encouraged to contribute the package.py files that you made to work on Cori. You can send us the package.py file using the give command,

tar -cvf packagename_username.tar $HOME/sw/var/spack/repos/$USER/packages/packagename
give -u swowner packagename_username.tar
Optionally, you can include a README file in the packagename directory and briefly describe the changes made and the full spack install command lines that were successful after the changes. We will review the package, then add it to the NERSC repository. This directory will be updated weekly if modified packages are available.

To minimize the maintenance effort, we will not maintain the NERSC repo across different Spack versions. If you would like to contribute to the Spack project, please follow the developer guide at here.

Spack support

For NESRC-specific issues, you are welcome to file a consulting ticket at https://help@nersc.gov. For general Spack questions, please consult the Spack resources listed below. For bug reports, please open an issue at the Spack github site.

Resources