Skip to content

E4S 21.02 - Cori

The E4S 21.02 stack is built using e4s-21.02 tag release in spack that is built for Haswell architecture. The spack configuration can be found at https://software.nersc.gov/NERSC/e4s-2102.

In order to access this stack you need to load the following module:

module load e4s/21.02

This module will setup a source spack in your user shell and provide the spack command in your $PATH.

Overview

This release comes with 45 E4S products, with most packages built with intel@19.1.2.254 compiler and a subset of packages were installed using gcc@10.1.0. Some of these packages were installed with gcc because spack could not concretize or install the package with intel compiler.

Shown below is the list of installed packages provided by spack.

Note

The spack find --explicit will show installed specs explicitly which is all of the E4S products which can be useful to filter output by E4S products. If you to see all packages (explicit and implicit) then please use spack find.

Output of spack find --explicit
$ spack find --explicit
==> In environment e4s
==> Root specs
-- no arch / gcc@10.1.0 -----------------------------------------
darshan-runtime@3.2.1%gcc@10.1.0 +slurm  dyninst@10.2.1%gcc@10.1.0   plasma@20.9.20%gcc@10.1.0 
darshan-util@3.2.1%gcc@10.1.0 +bzip2     legion@20.03.0%gcc@10.1.0   slate@2020.10.00%gcc@10.1.0 ~cuda

-- no arch / intel@19.1.2.254 -----------------------------------
adios2@2.7.1%intel@19.1.2.254 +hdf5
aml@0.1.0%intel@19.1.2.254 
arborx@0.9-beta%intel@19.1.2.254 +openmp
bolt@2.0%intel@19.1.2.254 
caliper@2.5.0%intel@19.1.2.254 +fortran
faodel@1.1906.1%intel@19.1.2.254 
flecsi@1.4%intel@19.1.2.254 +caliper+cinch+graphviz+tutorial
flit@2.1.0%intel@19.1.2.254 
gasnet@2020.3.0%intel@19.1.2.254 +udp
ginkgo@1.3.0%intel@19.1.2.254 
gotcha@1.0.3%intel@19.1.2.254 +test
hdf5@1.10.7%intel@19.1.2.254 
hypre@2.20.0%intel@19.1.2.254 +mixedint+openmp+superlu-dist
libnrm@0.1.0%intel@19.1.2.254 
libquo@1.3.1%intel@19.1.2.254 
mercury@2.0.0%intel@19.1.2.254 +udreg
mfem@4.2.0%intel@19.1.2.254 +examples+gnutls+gslib+lapack+libunwind+openmp+pumi+threadsafe+umpire
ninja@1.10.2%intel@19.1.2.254 
omega-h@9.32.5%intel@19.1.2.254 ~trilinos
openpmd-api@0.13.2%intel@19.1.2.254 
papi@6.0.0.1%intel@19.1.2.254 +example+infiniband+powercap+static_tools
papyrus@1.0.1%intel@19.1.2.254 
pdt@3.25.1%intel@19.1.2.254 +pic
precice@2.2.0%intel@19.1.2.254 +python
pumi@2.2.5%intel@19.1.2.254 +fortran
qthreads@1.16%intel@19.1.2.254 ~hwloc
raja@0.13.0%intel@19.1.2.254 +tests
slepc@3.14.2%intel@19.1.2.254 
strumpack@5.1.1%intel@19.1.2.254 +shared
sundials@5.7.0%intel@19.1.2.254 +examples-cxx+hypre+klu+lapack
superlu@5.2.1%intel@19.1.2.254 
superlu-dist@6.4.0%intel@19.1.2.254 +openmp
swig@4.0.2-fortran%intel@19.1.2.254 
tasmanian@7.3%intel@19.1.2.254 +blas+fortran+mpi+python+xsdkflags
tau@2.30.1%intel@19.1.2.254 +mpi~pdt
turbine@1.2.3%intel@19.1.2.254 +hdf5+python
umap@2.1.0%intel@19.1.2.254 +tests
umpire@4.1.2%intel@19.1.2.254 +fortran+numa+openmp
upcxx@2020.10.0%intel@19.1.2.254 
zfp@0.5.5%intel@19.1.2.254 +aligned+c+fortran+openmp+profile

==> 45 installed packages
-- cray-cnl7-haswell / gcc@10.1.0 -------------------------------
darshan-runtime@3.2.1  darshan-util@3.2.1  dyninst@10.2.1  legion@20.03.0  plasma@20.9.20  slate@2020.10.00

-- cray-cnl7-haswell / intel@19.1.2.254 -------------------------
adios2@2.7.1     flecsi@1.4       libnrm@0.1.0    openpmd-api@0.13.2  qthreads@1.16    superlu-dist@6.4.0  umpire@4.1.2
aml@0.1.0        flit@2.1.0       libquo@1.3.1    papi@6.0.0.1        raja@0.13.0      swig@4.0.2-fortran  upcxx@2020.10.0
arborx@0.9-beta  gasnet@2020.3.0  mercury@2.0.0   papyrus@1.0.1       slepc@3.14.2     tasmanian@7.3       zfp@0.5.5
bolt@2.0         ginkgo@1.3.0     mfem@4.2.0      pdt@3.25.1          strumpack@5.1.1  tau@2.30.1
caliper@2.5.0    gotcha@1.0.3     ninja@1.10.2    precice@2.2.0       sundials@5.7.0   turbine@1.2.3
faodel@1.1906.1  hypre@2.20.0     omega-h@9.32.5  pumi@2.2.5          superlu@5.2.1    umap@2.1.0

Accessing E4S Software Stack

When you load e4s/21.02 we expose a MODULEPATH to the location where spack generates modules. The full path to module root is /global/common/software/spackecp/e4s-21.02/modules/cray-cnl7-haswell/. In this release we provide all modules without hash in the form: {name}/{version}-{compiler.name}-{compiler.version}.

Note

We only generate modules for root specs which means packages installed implicitly won't have a generated module.

Output of module tree via module av
--------------------------------------------------------------------- /global/common/software/spackecp/e4s-21.02/modules/cray-cnl7-haswell/ ----------------------------------------------------------------------
adios2/2.7.1-intel-19.1.2.254       flecsi/1.4-intel-19.1.2.254         mercury/2.0.0-intel-19.1.2.254      precice/2.2.0-intel-19.1.2.254      superlu-dist/6.4.0-intel-19.1.2.254
aml/0.1.0-intel-19.1.2.254          flit/2.1.0-intel-19.1.2.254         mfem/4.2.0-intel-19.1.2.254         pumi/2.2.5-intel-19.1.2.254         swig/4.0.2-fortran-intel-19.1.2.254
arborx/0.9-beta-intel-19.1.2.254    gasnet/2020.3.0-intel-19.1.2.254    ninja/1.10.2-intel-19.1.2.254       qthreads/1.16-intel-19.1.2.254      tasmanian/7.3-intel-19.1.2.254
bolt/2.0-intel-19.1.2.254           ginkgo/1.3.0-intel-19.1.2.254       omega-h/9.32.5-intel-19.1.2.254     raja/0.13.0-intel-19.1.2.254        tau/2.30.1-intel-19.1.2.254
caliper/2.5.0-intel-19.1.2.254      gotcha/1.0.3-intel-19.1.2.254       openpmd-api/0.13.2-intel-19.1.2.254 slate/2020.10.00-gcc-10.1.0         turbine/1.2.3-intel-19.1.2.254
darshan-runtime/3.2.1-gcc-10.1.0    hypre/2.20.0-intel-19.1.2.254       papi/6.0.0.1-intel-19.1.2.254       slepc/3.14.2-intel-19.1.2.254       umap/2.1.0-intel-19.1.2.254
darshan-util/3.2.1-gcc-10.1.0       legion/20.03.0-gcc-10.1.0           papyrus/1.0.1-intel-19.1.2.254      strumpack/5.1.1-intel-19.1.2.254    umpire/4.1.2-intel-19.1.2.254
dyninst/10.2.1-gcc-10.1.0           libnrm/0.1.0-intel-19.1.2.254       pdt/3.25.1-intel-19.1.2.254         sundials/5.7.0-intel-19.1.2.254     upcxx/2020.10.0-intel-19.1.2.254
faodel/1.1906.1-intel-19.1.2.254    libquo/1.3.1-intel-19.1.2.254       plasma/20.9.20-gcc-10.1.0           superlu/5.2.1-intel-19.1.2.254      zfp/0.5.5-intel-19.1.2.254

If you are unsure about the module file you can use spack module tcl find <spec> to see a correlation of spec with its modulefile. If you want to know the full path to modulefile you can use the --full-path option in spack module tcl find command.

You can access the software via module load or spack load depending on your preference. For instance we load the papi module and you can run papi_mem_info to report Memory Cache details.

$ module load papi/6.0.0.1-intel-19.1.2.254
$ which papi_mem_info
/global/common/software/spackecp/e4s-21.02/software/cray-cnl7-haswell/intel-19.1.2.254/papi-6.0.0.1-ghgds2oiifqz4ulxpfpldp6br5rnwb2w/bin/papi_mem_info
$ papi_mem_info
Memory Cache and TLB Hierarchy Information.
------------------------------------------------------------------------
TLB Information.
  There may be multiple descriptors for each level of TLB
  if multiple page sizes are supported.

L1 Data TLB:
  Page Size:              4 KB
  Number of Entries:     64
  Associativity:          4


Cache Information.

L1 Data Cache:
  Total size:            32 KB
  Line size:             64 B
  Number of Lines:      512
  Associativity:          8

L1 Instruction Cache:
  Total size:            32 KB
  Line size:             64 B
  Number of Lines:      512
  Associativity:          8

L2 Unified Cache:
  Total size:           256 KB
  Line size:             64 B
  Number of Lines:     4096
  Associativity:          8

L3 Unified Cache:
  Total size:         40960 KB
  Line size:             64 B
  Number of Lines:   655360
  Associativity:         20

Alternately, you can use spack load to change your user environment similar to module load but you have access to all the spack packages whereas the modules only expose explitcit package.

Let's say we want to use darshan tool so we can load darshan-util and darshan-runtime package using the following. darshan-parser is one of the tools provided by darshan note that loading the spack package updated the $PATH to point to the E4S install.

$ spack load darshan-util
$ spack load darshan-runtime
$ which darshan-parser 
/global/common/software/spackecp/e4s-21.02/software/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx/bin/darshan-parser
$ darshan-parser --help
Usage: darshan-parser [options] <filename>
    --all   : all sub-options are enabled
    --base  : darshan log field data [default]
    --file  : total file counts
    --file-list  : per-file summaries
    --file-list-detailed  : per-file summaries with additional detail
    --perf  : derived perf data
    --total : aggregated darshan field data

If you want to see list of specs loaded in your user environment use the spack find --loaded command, which is akin to module list command which reports active modules.

Similarly you can use spack unload <spec> to remove spec from user environment or run spack unload -a to remove all loaded spack packages from user environment which is equivalent to module purge.

Installing Software via spack

We provide a mirror named cori-e4s-21.02 that provides a list of specs that can be installed from buildcache. To see the list of mirrors you can run the following

$ spack mirror list
cori-e4s-21.02    /global/common/software/spackecp/mirrors/cori-e4s-21.02
spack-public      https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/

To see a list of packages in the buildcache you can run, the -Lv option can be useful when you want to see specs with hash and list of variants for each spec:

spack buildcache list -Lv

If you want to install some packages, we recommend you create an environment in your user space such as $HOME as follows:

Note

You may need to deactivate from e4s environment by running despactivate or spack env deactivate in order to create new environment.

elvis@cori08:~> spack env create -d $HOME/myproject
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Created environment in /global/u2/y/elvis/myproject
==> You can activate this environment with:
==>   spack env activate /global/u2/y/elvis/myproject

elvis@cori08:~> spack env activate -d $HOME/myproject
elvis@cori10:~> spack env st
==> In environment /global/u2/y/elvis/myproject

Error

You cannot create spack environment using the name format because spack will try to write environment in $SPACK_ROOT/var/spack/environments which is not world writable

elvis@cori03:~/myproject> spack env create demo
==> Error: [Errno 13] Permission denied: '/global/common/software/spackecp/e4s-21.02/spack/var/spack/environments/demo'

We have configured our spack instance with site default in order to help facilitate spack install for multi-user. The spack configuration can be found at $SPACK_ROOT/etc/spack that contains files like compilers.yaml, config.yaml, mirrors.yaml, modules.yaml, and packages.yaml

For instance, we have configured the mirror cori-e4s-21.02 so that user can automatically install specs from buildcache. This configuration is set in $SPACK_ROOT/etc/spack/mirrors.yaml.

elvis@cori08:~> spack mirror list
cori-e4s-21.02    /global/common/software/spackecp/mirrors/cori-e4s-21.02
spack-public    https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/

Now let's try to install darshan-runtime and darshan-util by copying this content into your spack.yaml

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  view: true
  specs:
  - darshan-runtime@3.2.1%gcc@10.1.0 +slurm
  - darshan-util@3.2.1%gcc +bzip2

We need to trust the gpg key in-order for spack to install specs from buildcache. This is a one-time operation, we recommend you set the following which tells spack to look for gpg keys in $HOME/.gnupg instead of $SPACK_ROOT.

export SPACK_GNUPGHOME=$HOME/.gnupg

Next you should trust the gpg key by running the following:

$ spack gpg trust /global/common/software/spackecp/gpgkeys/e4s2010.pub   
gpgconf: socketdir is '/run/user/92503/gnupg'
gpg: key 0140A256659E0CBD: public key "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" imported
gpg: Total number processed: 1
gpg:               imported: 1

Next let's concretize the environment to see package preferences.

Running spack concretize -f
elvis@cori10:~/myproject> spack concretize -f
==> Concretized darshan-runtime@3.2.1%gcc@10.1.0+slurm
[+]  bqigqvv  darshan-runtime@3.2.1%gcc@10.1.0~cobalt+mpi~pbs+slurm arch=cray-cnl7-haswell
[+]  4owqxmf      ^mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell
[+]  5xqcgln      ^zlib@1.2.11%gcc@10.1.0+optimize+pic+shared arch=cray-cnl7-haswell

==> Concretized darshan-util@3.2.1%gcc+bzip2
 -   wgigpno  darshan-util@3.2.1%gcc@10.1.0+bzip2+shared patches=d1a0e581fb72830089302c94f77ccfcbaae416478f5241c4213fbe8b3f63b3b7 arch=cray-cnl7-haswell
 -   ssdgsz7      ^bzip2@1.0.8%gcc@10.1.0+shared arch=cray-cnl7-haswell
 -   3rcgfbw          ^diffutils@3.7%gcc@10.1.0 arch=cray-cnl7-haswell
 -   fu3r6gy              ^libiconv@1.16%gcc@10.1.0 arch=cray-cnl7-haswell
[+]  5xqcgln      ^zlib@1.2.11%gcc@10.1.0+optimize+pic+shared arch=cray-cnl7-haswell

==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Warning: [/global/u2/y/elvis/myproject/.spack-env/view] Skipping external package: mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell/4owqxmf

Please note we have defined some package preferences to help you satisfy some common package dependency (compiler, mpi, mkl), this can be found in $SPACK_ROOT/etc/spack/packages.yaml. We recommend you specify your own packages section in your spack.yaml to help tune your package preference. Note, if your package preference differs from specs in buildcache or slight difference in variants, then spack will end up installing packages from source so please be mindful of your selection. If you want to reuse existing specs in buildcache, you will need to specify preference with the same variants so that spack will reuse spec from buildcache. This can be done by comparing spec and its variant in buildcache spack buildcache list -Lv <spec> with the concretized output via spack concretize -f or spack spec -L <spec>

Running spack install
elvis@cori10:~/myproject> spack install
==> Installing environment /global/u2/y/elvis/myproject
==> Installing libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/libiconv-1.16/cray-cnl7-haswell-gcc-10.1.0-libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi.spack
################################################################################################################################################################### 100.0%
==> Extracting libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi from binary cache
gpgconf: socketdir is '/global/homes/y/elvis/.gnupg'
gpgconf:    no /run/user dir
gpgconf:    using homedir as fallback
gpg: Signature made Tue Jun  1 13:13:04 2021 PDT
gpg:                using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56  522F 0140 A256 659E 0CBD
     Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628  CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/zlib-1.2.11-5xqcglnrl3hl4wvyf4w3eafeefze5gcu
==> Installing diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/diffutils-3.7/cray-cnl7-haswell-gcc-10.1.0-diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx.spack
################################################################################################################################################################### 100.0%
==> Extracting diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx from binary cache
gpg: Signature made Tue Jun  1 14:03:15 2021 PDT
gpg:                using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56  522F 0140 A256 659E 0CBD
     Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628  CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx
==> Installing bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/bzip2-1.0.8/cray-cnl7-haswell-gcc-10.1.0-bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg.spack
################################################################################################################################################################### 100.0%
==> Extracting bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg from binary cache
gpg: Signature made Tue Jun  1 14:35:43 2021 PDT
gpg:                using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56  522F 0140 A256 659E 0CBD
     Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628  CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg
==> Installing darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx
==> Fetching file:///global/common/software/spackecp/mirrors/cori-e4s-21.02/build_cache/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1/cray-cnl7-haswell-gcc-10.1.0-darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx.spack
################################################################################################################################################################### 100.0%
==> Extracting darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx from binary cache
gpg: Signature made Tue Jun  1 16:32:50 2021 PDT
gpg:                using RSA key 1C0641AF18F4EA5A2628CBDDAC248997479D4AB0
gpg: Good signature from "Spack GPG Key (Spack E4S GPG Key) <shahzebsiddiqui@lbl.gov>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: EA17 2EB6 343D 3075 0A56  522F 0140 A256 659E 0CBD
     Subkey fingerprint: 1C06 41AF 18F4 EA5A 2628  CBDD AC24 8997 479D 4AB0
[+] /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Warning: [/global/u2/y/elvis/myproject/.spack-env/view] Skipping external package: mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell/4owqxmf
==> Updating view at /global/u2/y/elvis/myproject/.spack-env/view
==> Warning: [/global/u2/y/elvis/myproject/.spack-env/view] Skipping external package: mpich@3.1%gcc@10.1.0~argobots+fortran+hwloc+hydra+libxml2+pci+romio~slurm~verbs+wrapperrpath device=ch4 netmod=ofi pmi=pmi arch=cray-cnl7-haswell/4owqxmf

Module Generation

Now we have successfully installed packages, if you are curious where they are installed. By default we have set the install location in $HOME/spack-workspace/software which is set in $SPACK_ROOT/etc/spack/config.yaml. Let's do a quick check to see where software is installed by running the following

elvis@cori10:~/myproject> spack find -Lp                
==> In environment /global/u2/y/elvis/myproject
==> Root specs
-- no arch / gcc ------------------------------------------------
-------------------------------- darshan-util@3.2.1%gcc +bzip2

-- no arch / gcc@10.1.0 -----------------------------------------
-------------------------------- darshan-runtime@3.2.1%gcc@10.1.0 +slurm

==> 7 installed packages
-- cray-cnl7-haswell / gcc@10.1.0 -------------------------------
ssdgsz7hansfksagqhzaoxcvb3aug3bg bzip2@1.0.8            /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/bzip2-1.0.8-ssdgsz7hansfksagqhzaoxcvb3aug3bg
bqigqvvhf5rsdwa3yg53bfap7pcodfcw darshan-runtime@3.2.1  /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/darshan-runtime-3.2.1-bqigqvvhf5rsdwa3yg53bfap7pcodfcw
wgigpnoiosrtmizts4pn54of4lkh2ubx darshan-util@3.2.1     /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/darshan-util-3.2.1-wgigpnoiosrtmizts4pn54of4lkh2ubx
3rcgfbwwpvu7oxlsm2kh53trxjyilzdx diffutils@3.7          /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/diffutils-3.7-3rcgfbwwpvu7oxlsm2kh53trxjyilzdx
fu3r6gykbpgc3mayw34uhoh5q6pvjbhi libiconv@1.16          /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/libiconv-1.16-fu3r6gykbpgc3mayw34uhoh5q6pvjbhi
4owqxmfeiur5ctos6iwzfsmevyqvtl7e mpich@3.1              /opt/cray/pe/mpt/7.7.10/gni/mpich-gnu/8.2
5xqcglnrl3hl4wvyf4w3eafeefze5gcu zlib@1.2.11            /global/homes/y/elvis/spack-workspace/software/cray-cnl7-haswell/gcc-10.1.0/zlib-1.2.11-5xqcglnrl3hl4wvyf4w3eafeefze5gcu

Next, let's generate the modules for this environment. We have set a default location for modules in $HOME/spack-workspace/modules. If you want to generate the modules you can run the following:

elvis@cori10:~/myproject> spack module tcl refresh -y 
==> Regenerating tcl module files

Let's check where the module files are located by checking the path to one of the spec.

elvis@cori10:~/myproject> spack module tcl find --full-path darshan-runtime
/global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell/darshan-runtime/3.2.1-gcc-10.1.0

We see that the root of module tree is /global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell/ therefore we should add this path to MODULEPATH by running the following:

elvis@cori10:~/myproject> module use /global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell

The module use will prepend the directory in front of MODULEPATH.

If all is done correctly you should see a module tree as follows:

$ module av
----------------------------------------------------- /global/homes/y/elvis/spack-workspace/modules/cray-cnl7-haswell ------------------------------------------------------
bzip2/1.0.8-gcc-10.1.0           darshan-util/3.2.1-gcc-10.1.0    libiconv/1.16-gcc-10.1.0         zlib/1.2.11-gcc-10.1.0
darshan-runtime/3.2.1-gcc-10.1.0 diffutils/3.7-gcc-10.1.0         mpich/3.1-gcc-10.1.0

Note

We have the same name format as our production stack {name}/{version}-{compiler.name}-{compiler.version}. This is set in our site configuration $SPACK_ROOT/etc/spack/modules.yaml which can be overridden in your spack.yaml