Skip to content

Backups

Danger

All NERSC users should back up important files to HPSS on a regular basis. You should also have at least one copy of critical data stored outside of NERSC. Ultimately, it is your responsibility to protect yourself from data loss.

Snapshots

Global Homes, Global Common and Community use a snapshot capability to provide users a seven-day history of their directories. Running cd .snapshots in either your $HOME directory, your project directory under /global/common/software or your project directory under global/cfs/cdirs will take you to the snapshots of that directory. From there ls will show a directory for each snapshot, named for the date of the snapshot. You can navigate to the snapshot and directory of interest, and copy files from the snapshot to a new location.

In summary:

  • .snapshots is invisible to ls, ls -a, find and similar commands
  • Contents are visible through ls -F .snapshots
  • Can be browsed normally after cd .snapshots
  • Files cannot be created, deleted or edited in snapshots
  • Files can only be copied out of a snapshot

Backup/Restore

Global homes are periodically backed up to HPSS. The timing of backups depends on the size and number of files (NERSC filesystems are large, and a single backup can take many days to complete). We typically complete a backup of all home directories 1-2 times per month.

Note

Sparse files can appear to be much larger than your space quota (as the "missing" blocks are not stored), but these are not supported by our backup system. Sparse files may be more difficult to recover, and sparse files whose extent is large may not be backed up.

These backups are retained for at least 6 months, after which they may be removed to make space for newer backups.

Extracting files from backup can take several days. We recommend first searching the snapshots, and if the files you need are no longer available there, contact NERSC Consulting with pathnames and timestamps of the missing data.

Warning

These backups make it possible to recover accidentally-deleted data, but are not architected to provide disaster protection - we do not have multiple or off-site copies of backups. For this reason you should ensure you have other copies, in other locations, of all critical data.

Purging

Note

See Filesystem Quotas and Purging for detailed information about inode, space quotas, and file system purge policies.

Warning

$SCRATCH directories are not backed up