CVMFS¶
CVMFS (CernVM-File System) is a software distribution service heavily used by High-Energy Physics (HEP) experiments to deliver software to multiple different computing systems worldwide. It presents remote software pulled over http as a POSIX read-only file system in user space (normally using FUSE). Access is performant and scalable due to multiple levels of caching such as a local file cache and squid web caches.
NERSC Setup¶
At NERSC CVMFS is mounted on Cori compute nodes as an NFS mount via DVS (Cray's I/O forwarder for remote mounted file systems that is also used for the Community File System). It is also mounted on Cori login nodes. The NFS servers themselves mount cvmfs using FUSE in the normal way and we have dedicated local squid caches. We believe this setup to now be stable and perform well enough but users should please contact us if you see any problems, or if you need a new repository that is not currently available, via the online help desk.
Using CVMFS at NERSC¶
From a normal user perspective however this custom setup mentioned above doesn't change that all our mounted repositories appear under /cvmfs
cori$ ls /cvmfs
<various repositories>
CVMFS is also available on Cori compute nodes with or without shifter
If using shifter you should specify the --modules=cvmfs flag to shifter to make that mount appear inside your container (unless you want to use your own version of cvmfs inside the container).
cori$ shifter --module=cvmfs
You can also load this module in your batch script and you should also add the cvmfs file system license -L cvmfs which would allow us to pause your jobs from running if the cvmfs mount was unavailable
1 2 3 | |
Using CVMFS on Perlmutter¶
CVMFS will be supported on Perlmutter using the more conventional FUSED-based approach. This is not currently available but an initial version is expected to be provided for further testing around the December 2021 timeframe, please contact us via the online help desk if you are interested to help test.