Skip to content

E4S 21.05 - Cori

The e4s/21.05 stack is based on e4s-21.05 branch of spack. This release is built for Haswell using intel@19.1.3.304 compiler. You can load the following module to activate e4s/21.05 stack:

module load e4s/21.05

Overview

Output of spack find
elvis@cori> spack find -x --format "{name}@{version}%{compiler} {variants}"
adiak@0.2.1%intel@19.1.3.304 ~ipo+mpi+shared build_type=RelWithDebInfo
adios@1.13.1%intel@19.1.3.304 +blosc~bzip2~fortran~hdf5~infiniband+lz4+mpi~netcdf+shared+sz~szip+zfp+zlib patches=01113e9efb929d71c28bf33cc8b7f215d85195ec700e99cb41164e2f8f830640,8ae17f655248e87cbab1d1ed794e15364a38d2f5f8d971b1086702f72d79bd42,d24b79b795f66e40ddcd331ea4be896ac9c393d6f68f4318616d23928b0694e9 staging=none
adios2@2.7.1%intel@19.1.3.304 +blosc+bzip2~dataman~dataspaces~endian_reverse+fortran~hdf5~ipo+mpi+pic+png~python+shared+ssc+sst+sz+zfp build_type=Release
adlbx@1.0.0%intel@19.1.3.304
aml@0.1.0%intel@19.1.3.304
amrex@21.05%intel@19.1.3.304 ~amrdata~cuda~eb~fortran~hdf5~hypre~ipo+linear_solvers+mpi~openmp~particles~petsc~pic~rocm~shared~sundials amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none dimensions=3 precision=double
arborx@1.0%intel@19.1.3.304 ~cuda~ipo+mpi~openmp~rocm+serial~trilinos build_type=RelWithDebInfo
argobots@1.1%intel@19.1.3.304 ~affinity~debug+perf~stackunwind~tool~valgrind stackguard=none
arpack-ng@3.8.0%intel@19.1.3.304 +mpi+shared
ascent@0.7.1%intel@19.1.3.304 ~adios~babelflow~cuda~doc~dray~fortran~mfem+mpi+openmp~python+serial+shared+test+vtkh cuda_arch=none
axl@0.4.0%intel@19.1.3.304 ~bbapi_fallback~ipo async_api=daemon build_type=RelWithDebInfo
binutils@2.33.1%intel@19.1.3.304 ~gas+gold+headers~interwork+ld+libiberty~lto~nls+plugins libs=shared,static
blaspp@2021.04.01%intel@19.1.3.304 ~cuda~ipo+openmp~rocm+shared amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none
bolt@2.0%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
boost@1.75.0%intel@19.1.3.304 +atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math+mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=17 patches=57a8401dee8f52b0342e0c8147a5b2db834e8d8f3fbcbbc5950016bd3e9e1ef0,f2409bfa0e69e44817a5f8799e25c2e9e5ee50876a5aaacefd32fa647b80472f visibility=hidden
boost@1.75.0%intel@19.1.3.304 +atomic+chrono~clanglibcpp~container~context~coroutine+date_time~debug+exception~fiber+filesystem+graph~icu+iostreams+locale+log+math+mpi+multithreaded~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded+system~taggedlayout+test+thread+timer~versionedlayout+wave cxxstd=98 patches=57a8401dee8f52b0342e0c8147a5b2db834e8d8f3fbcbbc5950016bd3e9e1ef0,f2409bfa0e69e44817a5f8799e25c2e9e5ee50876a5aaacefd32fa647b80472f visibility=hidden
butterflypack@1.2.1%intel@19.1.3.304 ~ipo+shared build_type=RelWithDebInfo
bzip2@1.0.8%intel@19.1.3.304 ~debug~pic+shared
c-blosc@1.21.0%intel@19.1.3.304 +avx2~ipo build_type=RelWithDebInfo
cabana@0.3.0%intel@19.1.3.304 ~cuda~ipo+mpi~openmp+serial+shared build_type=RelWithDebInfo
caliper@2.5.0%intel@19.1.3.304 +adiak~cuda~fortran+gotcha~ipo+libdw~libpfm+libunwind+mpi+papi+sampler+shared~sosflow build_type=RelWithDebInfo cuda_arch=none
camp@0.1.0%intel@19.1.3.304 ~cuda~ipo~rocm~tests amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none
chai@2.3.0%intel@19.1.3.304 ~benchmarks~cuda~enable_pick+examples~ipo~raja~rocm+shared~tests amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none
cmake@3.20.2%intel@19.1.3.304 ~doc+ncurses+openssl+ownlibs~qt build_type=Release
conduit@0.7.2%intel@19.1.3.304 ~adios~doc~doxygen+fortran+hdf5+hdf5_compat~ipo+mpi~python+shared~silo+test~zfp build_type=RelWithDebInfo
darshan-runtime@3.3.0%intel@19.1.3.304 ~apmpi~apmpi_sync~apxc~cobalt~hdf5+mpi~pbs~slurm
darshan-util@3.3.0%intel@19.1.3.304 ~apmpi~apxc~bzip2+shared
dtcmp@1.1.1%intel@19.1.3.304
eigen@3.3.9%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
elfutils@0.182%intel@19.1.3.304 +bzip2~debuginfod~nls+xz
er@0.0.4%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
exmcutils@0.6.0%intel@19.1.3.304
expat@2.2.10%intel@19.1.3.304 +libbsd
faodel@1.1906.1%intel@19.1.3.304 ~cereal~hdf5~ipo+mpi+shared+tcmalloc build_type=RelWithDebInfo logging=stdout network=nnti patches=0d8604c48c421da1a28e5c23493a55c367fc39ebdf054f2978b4b6f2108bef91,823eff7668eb4ac2bac4b2b337d9edbeb486d60fc5a98177e9c9b1883159ef68
fftw@3.3.9%intel@19.1.3.304 +mpi~openmp~pfft_patches precision=double,float
flecsi@1.4%intel@19.1.3.304 ~caliper+cinch~coverage~debug_backend~doc~doxygen~flecstan~flog~graphviz+hdf5~ipo~minimal+shared~tutorial backend=mpi build_type=Release
flit@2.1.0%intel@19.1.3.304
gasnet@2021.3.0%intel@19.1.3.304 ~debug conduits=smp
gdbm@1.18.1%intel@19.1.3.304
gettext@0.21%intel@19.1.3.304 +bzip2+curses+git~libunistring+libxml2+tar+xz
ginkgo@1.3.0%intel@19.1.3.304 ~cuda~develtools~full_optimizations~hwloc~ipo+openmp~rocm+shared amdgpu_target=none build_type=Release cuda_arch=none
globalarrays@5.8%intel@19.1.3.304 ~scalapack armci=mpi-ts
gmake@4.3%intel@19.1.3.304 ~guile+nls
gmp@6.2.1%intel@19.1.3.304
googletest@1.8.1%intel@19.1.3.304 +gmock~ipo+pthreads+shared build_type=RelWithDebInfo
gotcha@1.0.3%intel@19.1.3.304 ~ipo~test build_type=RelWithDebInfo
hdf5@1.8.22%intel@19.1.3.304 ~cxx~debug+fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none
hdf5@1.10.7%intel@19.1.3.304 ~cxx~debug+fortran+hl~java+mpi+pic+shared~szip~threadsafe api=none
hwloc@2.4.1%intel@19.1.3.304 ~cairo~cuda~gl~libudev+libxml2~netloc~nvml+pci+shared
hypre@2.20.0%intel@19.1.3.304 ~complex~cuda~debug~int64~internal-superlu~mixedint+mpi~openmp+shared~superlu-dist~unified-memory cuda_arch=none patches=6e3336b1d62155f6350dfe42b0f9ea25d4fa0af60c7e540959139deb93a26059
kokkos@3.4.00%intel@19.1.3.304 ~aggressive_vectorization~compiler_warnings~cuda~cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning~wrapper amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none std=14
kokkos@3.4.00%intel@19.1.3.304 ~aggressive_vectorization~compiler_warnings~cuda~cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl+openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning~wrapper amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none std=14
kokkos-kernels@3.2.00%intel@19.1.3.304 ~blas~cblas~cublas~cuda~cusparse~ipo~lapack~lapacke~mkl+openmp~pthread~serial~superlu build_type=RelWithDebInfo cuda_arch=none execspace_cuda=auto execspace_openmp=auto execspace_serial=auto execspace_threads=auto layouts=left memspace_cudaspace=auto memspace_cudauvmspace=auto offsets=int,size_t ordinals=int scalars=double
kvtree@1.1.1%intel@19.1.3.304 ~ipo+mpi build_type=RelWithDebInfo file_lock=FLOCK
lapackpp@2021.04.00%intel@19.1.3.304 ~ipo+shared build_type=RelWithDebInfo
legion@21.03.0%intel@19.1.3.304 ~bindings~bounds_checks~cuda~cuda_hijack~cuda_unsupported_compiler~enable_tls~fortran~gasnet_debug~hdf5~hwloc~ipo~kokkos+libdl~native~openmp~papi~privilege_checks~python~redop_complex~shared~spy+zlib build_type=RelWithDebInfo conduit=none cuda_arch=70 gasnet_root=none max_dims=3 max_fields=512 network=none output_level=warning
libbsd@0.10.0%intel@19.1.3.304  patches=71b49f52a01420ab632f23373135853a2565f88725fb77e625a7fbf7ea812eef
libdwarf@20180129%intel@19.1.3.304
libfabric@1.12.1%intel@19.1.3.304 ~kdreg fabrics=rxm,sockets,tcp,udp
libffi@3.3%intel@19.1.3.304  patches=26f26c6f29a7ce9bf370ad3ab2610f99365b4bdd7b82e7c31df41a3370d685c0
libiconv@1.16%intel@19.1.3.304
libnrm@0.1.0%intel@19.1.3.304
libpciaccess@0.16%intel@19.1.3.304
libpng@1.6.37%intel@19.1.3.304
libquo@1.3.1%intel@19.1.3.304
libsigsegv@2.12%intel@19.1.3.304
libsodium@1.0.18%intel@19.1.3.304
libunwind@1.5.0%intel@19.1.3.304 +pic+xz~zlib
libxml2@2.9.10%intel@19.1.3.304 ~python
libyogrt@1.24%intel@19.1.3.304 ~static scheduler=slurm
libzmq@4.3.3%intel@19.1.3.304 ~drafts+libsodium
loki@0.1.7%intel@19.1.3.304 +shared
lwgrp@1.0.3%intel@19.1.3.304
lz4@1.9.3%intel@19.1.3.304  libs=shared,static
m4@1.4.18%intel@19.1.3.304 +sigsegv patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8
mercury@2.0.1%intel@19.1.3.304 ~bmi+boostsys~cci+checksum~debug~ipo+mpi+ofi+shared+sm~udreg build_type=RelWithDebInfo
metall@0.13%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
metis@5.1.0%intel@19.1.3.304 ~gdb~int64~real64+shared build_type=Release patches=4991da938c1d3a1d3dea78e49bbebecba00273f98df2a656e38b83d55b281da1
mfem@4.2.0%intel@19.1.3.304 ~amgx~conduit~cuda~debug~examples~gnutls~gslib~lapack~libceed~libunwind+metis~miniapps~mpfr+mpi~netcdf~occa~openmp~petsc~pumi~raja~shared+static~strumpack~suite-sparse~sundials~superlu-dist~threadsafe~umpire+zlib cuda_arch=sm_60 timer=auto
mpark-variant@1.4.0%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo patches=21a4f8de3525204ee6db2e53758a3e3fd9c13817df29d2926d24376858a369e7,4e173fe8c853eb92956a40371688b4a19498189fe65b7ceac30f6b9d6663a985,b3501f726fd40129b4aaa11453a5891c8953a34af8ac84f5ab10a22afa5e7b9b
ncurses@6.2%intel@19.1.3.304 ~symlinks+termlib abi=none
netlib-scalapack@2.1.0%intel@19.1.3.304 ~ipo~pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2
ninja@1.10.2%intel@19.1.3.304
nlohmann-json@3.9.1%intel@19.1.3.304 ~ipo+single_header build_type=RelWithDebInfo
numactl@2.0.14%intel@19.1.3.304  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006
openblas@0.3.10%intel@19.1.3.304 ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared patches=00230a6b2bc50d7722d4aad49c0c06091176910b942cd51980b345e2935300b0,2bc1db38b52435f4bbb211b9532549a8ffd0accaf3bd8f3f0d3dd39cf606151c,865703b4f405543bbd583413fdeff2226dfda908be33639276c06e5aa7ae2cae threads=openmp
openjdk@11.0.8_10%intel@19.1.3.304
openpmd-api@0.13.4%intel@19.1.3.304 ~adios1+adios2+hdf5~ipo+mpi~python+shared build_type=RelWithDebInfo
openssl@1.1.1k%intel@19.1.3.304 ~docs+systemcerts
otf2@2.3%intel@19.1.3.304
papi@6.0.0.1%intel@19.1.3.304 ~cuda+example~infiniband~lmsensors~nvml~powercap~rapl~sde+shared~static_tools
papyrus@1.0.1%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
parallel-netcdf@1.12.2%intel@19.1.3.304 ~burstbuffer+cxx+fortran+pic+shared
parmetis@4.0.3%intel@19.1.3.304 ~gdb~int64~ipo+shared build_type=RelWithDebInfo patches=4f892531eb0a807eb1b82e683a416d3e35154a455274cf9b162fb02054d11a5b,50ed2081bc939269689789942067c58b3e522c269269a430d5d34c00edbc5870,704b84f7c7444d4372cb59cca6e1209df4ef3b033bc4ee3cf50f369bce972a9d
pcre@8.44%intel@19.1.3.304 ~jit+multibyte+utf
pdsh@2.31%intel@19.1.3.304 +ssh+static_modules
pdt@3.25.1%intel@19.1.3.304 ~pic
petsc@3.15.0%intel@19.1.3.304 ~X~batch~cgns~complex~cuda~debug+double~exodusii~fftw~giflib+hdf5+hypre~int64~jpeg~knl~libpng~libyaml~memkind+metis~mkl-pardiso~moab~mpfr+mpi~mumps~p4est~ptscotch~random123~saws+shared~suite-sparse+superlu-dist~trilinos~valgrind clanguage=C
pfunit@3.3.3%intel@19.1.3.304 ~docs~ipo+mpi~openmp+shared~use_comm_world build_type=RelWithDebInfo max_array_rank=5
precice@2.2.1%intel@19.1.3.304 ~ipo+mpi+petsc~python+shared build_type=RelWithDebInfo
pumi@2.2.5%intel@19.1.3.304 ~fortran~int64~ipo~shared+simmodsuite_version_check~zoltan build_type=RelWithDebInfo simmodsuite=none
py-libensemble@0.7.2%intel@19.1.3.304 ~deap+mpi~mpmath~nlopt~petsc4py~scipy
py-mpi4py@3.0.3%intel@19.1.3.304
py-numpy@1.20.3%intel@19.1.3.304 +blas+lapack patches=873745d7b547857fcfec9cae90b09c133b42a4f0c23b6c2d84cf37e2dd816604
py-periodictable@1.5.0%intel@19.1.3.304
py-petsc4py@3.15.0%intel@19.1.3.304 +mpi patches=d344e0e8b4c5ebb96961e1905bdd749162c52b151538238609f4dec77e2a6360
py-picmistandard@develop%intel@19.1.3.304
py-psutil@5.7.2%intel@19.1.3.304
py-pyelftools@0.26%intel@19.1.3.304
py-pyparsing@2.4.7%intel@19.1.3.304
py-setuptools@50.3.2%intel@19.1.3.304
py-toml@0.10.2%intel@19.1.3.304
py-warpx@21.05%intel@19.1.3.304 +mpi
py-warpx@21.05%intel@19.1.3.304 +mpi
py-warpx@21.05%intel@19.1.3.304 +mpi
python@3.8.10%intel@19.1.3.304 +bz2+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tix~tkinter~ucs4+uuid+zlib patches=0d98e93189bc278fbc37a50ed7f183bd8aaf249a8e1670a465f0db6bb4f8cf87,ebdca648c9c1d25f586d7e2a495b62e6d91973b55264a13d89eda1beff72ef56
qthreads@1.16%intel@19.1.3.304 +hwloc~spawn_cache+static scheduler=distrib stack_size=4096
raja@0.13.0%intel@19.1.3.304 ~cuda+examples+exercises~ipo+openmp~rocm+shared~tests amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none
rankstr@0.0.3%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
readline@8.1%intel@19.1.3.304
redset@0.0.5%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
scr@3.0rc1%intel@19.1.3.304 ~bbapi_fallback+dtcmp+fortran~ipo+libyogrt async_api=NONE build_type=RelWithDebInfo cache_base=/dev/shm cntl_base=/dev/shm copy_config=none file_lock=FLOCK resource_manager=SLURM scr_config=scr.conf
shuffile@0.0.4%intel@19.1.3.304 ~ipo build_type=RelWithDebInfo
slepc@3.15.0%intel@19.1.3.304 +arpack~blopex
snappy@1.1.8%intel@19.1.3.304 ~ipo+pic+shared build_type=RelWithDebInfo patches=c9cfecb1f7a623418590cf4e00ae7d308d1c3faeb15046c2e5090e38221da7cd
spath@0.0.2%intel@19.1.3.304 ~ipo+mpi build_type=RelWithDebInfo
sqlite@3.34.0%intel@19.1.3.304 +column_metadata+fts~functions~rtree
stc@0.9.0%intel@19.1.3.304
strumpack@5.1.1%intel@19.1.3.304 ~build_dev_tests~build_tests+butterflypack+c_interface~count_flops~cuda~ipo+mpi+openmp+parmetis~rocm~scotch~shared~slate~task_timers+zfp amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none
sundials@5.7.0%intel@19.1.3.304 +ARKODE+CVODE+CVODES+IDA+IDAS+KINSOL~cuda+examples+examples-install~f2003~fcmix+generic-math~hypre~int64~ipo~klu~lapack~monitoring+mpi~openmp~petsc~pthread~raja~rocm+shared+static~superlu-dist~superlu-mt~trilinos amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none patches=8b29a47fac55346cdadfa133c51aa9314ae8c53ffdff5a8ecdc3dcea3ac26403 precision=double
superlu@5.2.1%intel@19.1.3.304 +pic
superlu-dist@6.4.0%intel@19.1.3.304 ~cuda~int64~ipo~openmp+shared build_type=RelWithDebInfo cuda_arch=none
swig@4.0.2%intel@19.1.3.304
swig@4.0.2-fortran%intel@19.1.3.304
sz@1.4.12.3%intel@19.1.3.304 ~fortran~hdf5~ipo~netcdf~pastri~python~random_access+shared~stats~time_compression build_type=RelWithDebInfo
sz@2.1.11.1%intel@19.1.3.304 ~fortran~hdf5~ipo~netcdf~pastri~python~random_access+shared~stats~time_compression build_type=RelWithDebInfo
tar@1.32%intel@19.1.3.304
tasmanian@7.5%intel@19.1.3.304 ~blas~cuda~fortran~ipo~magma+mpi+openmp~python~rocm~xsdkflags amdgpu_target=none build_type=Release cuda_arch=none
tau@2.30.1%intel@19.1.3.304 ~adios2+binutils~comm~craycnl~cuda+elf+fortran~gasnet+io~level_zero+libdwarf+libunwind~likwid+mpi~ompt~opari~opencl~openmp+otf2+papi+pdt~phase~ppc64le~profileparam+pthreads~python~rocm~rocprofiler~scorep~shmem~sqlite~x86_64
tcl@8.6.11%intel@19.1.3.304
turbine@1.3.0%intel@19.1.3.304 ~hdf5~python~r
umap@2.1.0%intel@19.1.3.304 ~ipo~logging~tests build_type=RelWithDebInfo
umpire@4.1.2%intel@19.1.3.304 +c~cuda~deviceconst+examples~fortran~ipo~numa~openmp~rocm+shared amdgpu_target=none build_type=RelWithDebInfo cuda_arch=none patches=7d912d31cd293df005ba74cb96c6f3e32dc3d84afff49b14509714283693db08 tests=none
upcxx@2021.3.0%intel@19.1.3.304 ~cuda~gasnet+mpi cross=cray-aries-slurm
util-linux-uuid@2.36.2%intel@19.1.3.304
vtk-h@0.7.1%intel@19.1.3.304 ~contourtree~cuda~logging+mpi+openmp+serial+shared cuda_arch=none
vtk-m@1.5.5%intel@19.1.3.304 ~64bitids+ascent_types~cuda+doubleprecision~hip~ipo~logging+mpi+openmp+rendering~shared~tbb~virtuals amdgpu_target=none build_type=Release cuda_arch=none
warpx@21.05%intel@19.1.3.304 +app~ascent~eb~ipo+lib+mpi+mpithreadmultiple+openpmd+psatd+qed~qedtablegen+shared+tprof build_type=RelWithDebInfo compute=omp dims=2 precision=double
warpx@21.05%intel@19.1.3.304 +app~ascent~eb~ipo+lib+mpi+mpithreadmultiple+openpmd+psatd+qed~qedtablegen+shared+tprof build_type=RelWithDebInfo compute=omp dims=3 precision=double
warpx@21.05%intel@19.1.3.304 +app~ascent~eb~ipo+lib+mpi+mpithreadmultiple+openpmd+psatd+qed~qedtablegen+shared+tprof build_type=RelWithDebInfo compute=omp dims=rz precision=double
xz@5.2.5%intel@19.1.3.304 +pic libs=shared,static
zfp@0.5.5%intel@19.1.3.304 ~aligned~c~cuda~fasthash~fortran~ipo~openmp~profile~python+shared~strided~twoway bsws=64 build_type=RelWithDebInfo cuda_arch=none
zlib@1.2.11%intel@19.1.3.304 +optimize+pic+shared
zsh@5.8%intel@19.1.3.304 +skip-tcsetpgrp-test
zstd@1.4.9%intel@19.1.3.304 ~ipo~legacy~lz4~lzma+multithread+programs+shared+static~zlib build_type=RelWithDebInfo

We have generated modules for spack packages for root specs that can be found below

---------------------------- /global/common/software/spackecp/e4s-21.05/modules/cray-cnl7-haswell/ -----------------------------
adiak/0.2.1-intel-19.1.3.304              kokkos-kernels/3.2.00-intel-19.1.3.304    py-periodictable/1.5.0-intel-19.1.3.304
adios/1.13.1-intel-19.1.3.304             kvtree/1.1.1-intel-19.1.3.304             py-petsc4py/3.15.0-intel-19.1.3.304
adios2/2.7.1-intel-19.1.3.304             lapackpp/2021.04.00-intel-19.1.3.304      py-picmistandard/develop-intel-19.1.3.304
adlbx/1.0.0-intel-19.1.3.304              legion/21.03.0-intel-19.1.3.304           py-psutil/5.7.2-intel-19.1.3.304
aml/0.1.0-intel-19.1.3.304                libbsd/0.10.0-intel-19.1.3.304            py-pyelftools/0.26-intel-19.1.3.304
amrex/21.05-intel-19.1.3.304              libdwarf/20180129-intel-19.1.3.304        py-pyparsing/2.4.7-intel-19.1.3.304
arborx/1.0-intel-19.1.3.304               libfabric/1.12.1-intel-19.1.3.304         py-setuptools/50.3.2-intel-19.1.3.304
argobots/1.1-intel-19.1.3.304             libffi/3.3-intel-19.1.3.304               py-toml/0.10.2-intel-19.1.3.304
arpack-ng/3.8.0-intel-19.1.3.304          libiconv/1.16-intel-19.1.3.304            py-warpx/21.05-dims2
ascent/0.7.1-intel-19.1.3.304             libnrm/0.1.0-intel-19.1.3.304             py-warpx/21.05-dims3
axl/0.4.0-intel-19.1.3.304                libpciaccess/0.16-intel-19.1.3.304        py-warpx/21.05-dimsRZ
binutils/2.33.1-intel-19.1.3.304          libpng/1.6.37-intel-19.1.3.304            python/3.8.10-intel-19.1.3.304
blaspp/2021.04.01-intel-19.1.3.304        libquo/1.3.1-intel-19.1.3.304             qthreads/1.16-intel-19.1.3.304
bolt/2.0-intel-19.1.3.304                 libsigsegv/2.12-intel-19.1.3.304          raja/0.13.0-intel-19.1.3.304
boost/1.75.0-cxxstd=17                    libsodium/1.0.18-intel-19.1.3.304         rankstr/0.0.3-intel-19.1.3.304
boost/1.75.0-cxxstd=98                    libunwind/1.5.0-intel-19.1.3.304          readline/8.1-intel-19.1.3.304
butterflypack/1.2.1-intel-19.1.3.304      libxml2/2.9.10-intel-19.1.3.304           redset/0.0.5-intel-19.1.3.304
bzip2/1.0.8-intel-19.1.3.304              libyogrt/1.24-intel-19.1.3.304            scr/3.0rc1-intel-19.1.3.304
c-blosc/1.21.0-intel-19.1.3.304           libzmq/4.3.3-intel-19.1.3.304             shuffile/0.0.4-intel-19.1.3.304
cabana/0.3.0-intel-19.1.3.304             loki/0.1.7-intel-19.1.3.304               slepc/3.15.0-intel-19.1.3.304
caliper/2.5.0-intel-19.1.3.304            lwgrp/1.0.3-intel-19.1.3.304              snappy/1.1.8-intel-19.1.3.304
camp/0.1.0-intel-19.1.3.304               lz4/1.9.3-intel-19.1.3.304                spath/0.0.2-intel-19.1.3.304
chai/2.3.0-intel-19.1.3.304               m4/1.4.18-intel-19.1.3.304                sqlite/3.34.0-intel-19.1.3.304
cmake/3.20.2-intel-19.1.3.304             mercury/2.0.1-intel-19.1.3.304            stc/0.9.0-intel-19.1.3.304
conduit/0.7.2-intel-19.1.3.304            metall/0.13-intel-19.1.3.304              strumpack/5.1.1-intel-19.1.3.304
darshan-runtime/3.3.0-intel-19.1.3.304    metis/5.1.0-intel-19.1.3.304              sundials/5.7.0-intel-19.1.3.304
darshan-util/3.3.0-intel-19.1.3.304       mfem/4.2.0-intel-19.1.3.304               superlu/5.2.1-intel-19.1.3.304
dtcmp/1.1.1-intel-19.1.3.304              mpark-variant/1.4.0-intel-19.1.3.304      superlu-dist/6.4.0-intel-19.1.3.304
eigen/3.3.9-intel-19.1.3.304              ncurses/6.2-intel-19.1.3.304              swig/4.0.2-fortran-intel-19.1.3.304
elfutils/0.182-intel-19.1.3.304           netlib-scalapack/2.1.0-intel-19.1.3.304   swig/4.0.2-intel-19.1.3.304
er/0.0.4-intel-19.1.3.304                 ninja/1.10.2-intel-19.1.3.304             sz/1.4.12.3-intel-19.1.3.304
exmcutils/0.6.0-intel-19.1.3.304          nlohmann-json/3.9.1-intel-19.1.3.304      sz/2.1.11.1-intel-19.1.3.304
expat/2.2.10-intel-19.1.3.304             numactl/2.0.14-intel-19.1.3.304           tar/1.32-intel-19.1.3.304
faodel/1.1906.1-intel-19.1.3.304          openblas/0.3.10-intel-19.1.3.304          tasmanian/7.5-intel-19.1.3.304
fftw/3.3.9-intel-19.1.3.304               openjdk/11.0.8_10-intel-19.1.3.304        tau/2.30.1-intel-19.1.3.304
flecsi/1.4-intel-19.1.3.304               openpmd-api/0.13.4-intel-19.1.3.304       tcl/8.6.11-intel-19.1.3.304
flit/2.1.0-intel-19.1.3.304               openssl/1.1.1k-intel-19.1.3.304           turbine/1.3.0-intel-19.1.3.304
gasnet/2021.3.0-intel-19.1.3.304          otf2/2.3-intel-19.1.3.304                 umap/2.1.0-intel-19.1.3.304
gdbm/1.18.1-intel-19.1.3.304              papi/6.0.0.1-intel-19.1.3.304             umpire/4.1.2-intel-19.1.3.304
gettext/0.21-intel-19.1.3.304             papyrus/1.0.1-intel-19.1.3.304            upcxx/2021.3.0-intel-19.1.3.304
ginkgo/1.3.0-intel-19.1.3.304             parallel-netcdf/1.12.2-intel-19.1.3.304   util-linux-uuid/2.36.2-intel-19.1.3.304
globalarrays/5.8-intel-19.1.3.304         parmetis/4.0.3-intel-19.1.3.304           vtk-h/0.7.1-intel-19.1.3.304
gmake/4.3-intel-19.1.3.304                pcre/8.44-intel-19.1.3.304                vtk-m/1.5.5-intel-19.1.3.304
gmp/6.2.1-intel-19.1.3.304                pdsh/2.31-intel-19.1.3.304                warpx/21.05-dims2
googletest/1.8.1-intel-19.1.3.304         pdt/3.25.1-intel-19.1.3.304               warpx/21.05-dims3
gotcha/1.0.3-intel-19.1.3.304             petsc/3.15.0-intel-19.1.3.304             warpx/21.05-dimsRZ
hdf5/1.8.22-intel-19.1.3.304              pfunit/3.3.3-intel-19.1.3.304             xz/5.2.5-intel-19.1.3.304
hdf5/1.10.7-intel-19.1.3.304              precice/2.2.1-intel-19.1.3.304            zfp/0.5.5-intel-19.1.3.304
hwloc/2.4.1-intel-19.1.3.304              pumi/2.2.5-intel-19.1.3.304               zlib/1.2.11-intel-19.1.3.304
hypre/2.20.0-intel-19.1.3.304             py-libensemble/0.7.2-intel-19.1.3.304     zsh/5.8-intel-19.1.3.304
kokkos/3.4.00                             py-mpi4py/3.0.3-intel-19.1.3.304          zstd/1.4.9-intel-19.1.3.304
kokkos/3.4.00-openmp                      py-numpy/1.20.3-intel-19.1.3.304

Site Configuration Scope

We have configured our spack instance with site configuration scope which provides spack settings configured for the spack instance. The site configuration can be found at $SPACK_ROOT/etc/spack which provides the following configuration: config.yaml, compilers.yaml, modules.yaml, packages.yaml and mirrors.yaml. The site configuration for this spack instance can be found in spack/spack-configs. This spack instance will define intel@19.1.3.304 in the site configuration which can be retrieved via following command:

elvis@cori> spack config --scope=site get compilers
compilers:
- compiler:
    spec: intel@19.1.3.304
    paths:
      cc: cc
      cxx: CC
      f77: ftn
      fc: ftn
    flags: {}
    operating_system: cnl7
    target: any
    modules:
    - PrgEnv-intel
    - intel/19.1.3.304
    environment: {}
    extra_rpaths: []

We have setup a mirror name cori-e4s-21.05 that can be used to configure a spack mirror in-order to install specs from buildcache.

elvis@cori> spack mirror list
cori-e4s-21.05    https://cache.e4s.io/21.05
spack-public      https://spack-llnl-mirror.s3-us-west-2.amazonaws.com/

If you want to use buildcache we recommend you setup the following

  1. Set export SPACK_GNUPGHOME=$HOME/.gnupg to store gpg key in your home directory
  2. Install key from mirror via spack buildcache keys -it

Once this is setup you should see an entry for gpg key if you run spack gpg list with the following

elvis@cori> spack gpg list
pub   rsa4096 2021-07-02 [SC]
      25645FA2B218FE55B4EF649E4345F04B40005581
uid           [ unknown] University of Oregon - E4S
sub   rsa4096 2021-07-02 [E]

The buildcache provides spack builds for intel@19.1.3.304 and gcc@9.3.0 compiler which can be retrieved by running spack buildcache list. Our deployment will only show intel built packages however you can install the gcc specs from the buildcache.

We have setup package preference for the site spack configuration which you can get by running the following

spack config --scope=site get packages

You can define packages.yaml section or set packages section in your spack.yaml to define your own build preference. We recommend you use our spack site configuration to make use of the buildcache though it may not be suitable for everyone.

Spack Workflow

If you plan to install specs, we recommend you setup a spack environment in your HOME directory or shared project space ($CFS, $SCRATCH). Please make sure you deactivate from e4s environment before you create a spack environment by running the following:

despacktivate 

Next create a spack environment in your home directory and activate the spack environment.

spack env create -d $HOME/e4s-21.05-demo
spack env activate -d $HOME/e4s-21.05-demo

Navigate to your environment directory $HOME/e4s-21.05-demo and paste the following content in your spack.yaml

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs:
  - bzip2
  view: true

Note

You may need to run spack clean -a if you run into issues with buildcache.

To install bzip2 from the spack.yaml by running spack install. One can note here, that spack will install specs from the buildcache instead of building package from source.

elvis@cori> spack install
==> Concretized bzip2
[+]  d7vdmqf  bzip2@1.0.8%intel@19.1.3.304~debug~pic+shared arch=cray-cnl7-haswell
[+]  vc6e6zh      ^diffutils@3.7%intel@19.1.3.304 arch=cray-cnl7-haswell
[+]  55pttud          ^libiconv@1.16%intel@19.1.3.304 arch=cray-cnl7-haswell

==> Installing environment /global/homes/s/elvis/e4s-21.05-demo
==> All of the packages are already installed
==> Updating view at /global/homes/s/elvis/e4s-21.05-demo/.spack-env/view

One can confirm the installation of bzip2@1.0.8@intel@19.1.3.304 spec with hash d7vdmqf is in the buildcache by running the following command:

elvis@cori> spack buildcache list -Lv bzip2%intel
==> 1 cached build.
-- cray-cnl7-haswell / intel@19.1.3.304 -------------------------
d7vdmqfnjh342v3ujqyjgq7dg3zzjsgo bzip2@1.0.8~debug~pic+shared

Note

Spack must concretize the spec to the same hash in order to install from buildcache. In this example, spack will install the following bzip2@1.0.8~debug~pic+shared spec which is provided in the buildcache. If there is a slight difference in the variant, spack will report a different hash and package will be installed from source.

If all goes well, you can run spack find to see a list of installed specs in the environment

elvis@cori> spack find
==> In environment /global/homes/s/elvis/e4s-21.05-demo
==> Root specs
bzip2

==> 3 installed packages
-- cray-cnl7-haswell / intel@19.1.3.304 -------------------------
bzip2@1.0.8  diffutils@3.7  libiconv@1.16

Spack will install the specs in your $HOME/spack-workspace, you can see the site configuration scope as shown below. The install_tree is responsible for specifying where spack will install the specs. For more details see configuring spack.

elvis@cori08> spack config --scope=site get config | head -n 10
config:
  install_tree:
    root: $HOME/spack-workspace/software
    projections:
      all: ${ARCHITECTURE}/${COMPILERNAME}-${COMPILERVER}/${PACKAGE}-${VERSION}-${HASH}
  source_cache: $HOME/.spack/cache
  module_roots:
    tcl: $HOME/spack-workspace/modules
    lmod: $HOME/spack-workspace/modules
  template_dirs:

We can see the spec is installed in our home directory via spack location command

elvis@cori> spack location -i bzip2
/global/homes/s/elvis/spack-workspace/software/cray-cnl7-haswell/intel-19.1.3.304/bzip2-1.0.8-d7vdmqfnjh342v3ujqyjgq7dg3zzjsgo

Congratulations now you have installed a spec from the buildcache!