Skip to content

CVMFS

CVMFS (CernVM-File System) is a software distribution service heavily used by High-Energy Physics (HEP) experiments to deliver software to multiple different computing systems worldwide. It presents remote software pulled over http as a POSIX read-only file system in user space (normally using FUSE). Access is performant and scalable due to multiple levels of caching such as a local file cache and squid web caches.

NERSC Setup

At NERSC CVMFS is mounted on Cori compute nodes as an NFS mount via DVS (Cray's I/O forwarder for remote mounted file systems that is also used for the Community File System). It is also mounted on Cori login nodes. The NFS servers themselves mount cvmfs using FUSE in the normal way and we have dedicated local squid caches. We believe this setup to now be stable and perform well enough but users should please contact us if you see any problems, or if you need a new repository that is not currently available, via the online help desk.

Using CVMFS at NERSC

From a normal user perspective however this custom setup mentioned above doesn't change that all our mounted repositories appear under /cvmfs

cori$ ls /cvmfs
<various repositories>

CVMFS is also available on Cori compute nodes with or without shifter

If using shifter you should specify the --modules=cvmfs flag to shifter to make that mount appear inside your container (unless you want to use your own version of cvmfs inside the container).

cori$ shifter --module=cvmfs

You can also load this module in your batch script and you should also add the cvmfs file system license -L cvmfs which would allow us to pause your jobs from running if the cvmfs mount was unavailable

1
2
3
#!/bin/bash
#SBATCH --module=cvmfs
#SBATCH -L cvmfs

Using CVMFS on Perlmutter

CVMFS will be supported on Perlmutter using the more conventional FUSED-based approach. This is not currently available but an initial version is expected to be provided for further testing around the December 2021 timeframe, please contact us via the online help desk if you are interested to help test.