NERSC supports the provisioning of databases to support computational work done at NERSC. These are mostly used for scientific or job workflow metadata; for derived data presented via science gateways; or for provenance information about science data stored in file formats.
Requesting a Database¶
If you would like to create your database at NERSC please fill out this form (in servicenow: requires login) and you'll be contacted by NERSC staff. Unless explicitly requested and justified, databases will only be accessible from within the NERSC network.
Connecting to a database from outside of NERSC¶
Services running on non-http ports, such as databases, cannot be directly accessed from outside of the NERSC network. However, one can connect from anywhere using an SSH tunnel authenticated with NERSC user credentials.
For connections to NERSC hosted databases from a personal laptop or workstation, using an SSH tunnel is recommended. When interactively working with a database, many graphical database clients, such as Robo 3T, have tunnel configurations which can be saved for easy reconnection. If connecting programmatically (e.g. from a Python script), an SSH alias can be created to save the SSH configuration for the tunnel. Then the SSH tunnel must be opened before launching the application which uses the connection.
Manually starting an SSH connection to connect to a database might look something like:
user@laptop> ssh -L localhost:12345:mongodb07.nersc.gov:27017 firstname.lastname@example.org
This forwards connections made to
localhost:12345 through an SSH tunnel to
dtn01.nersc.gov and on to
mongodb07.nersc.gov:27017. In the python application, one would then connect to their database as if it was located at
localhost:12345. One can pick any port number on their localhost as long as it is not being used by another service.
An SSH alias can also be created in the user’s
~/.ssh/config file which saves the command line settings, and would look like:
Host mongo-tunnel Hostname dtn01.nersc.gov User nersc-user LocalForward localhost:27017 mongodb07.nersc.gov:27017
For the application to successfully access the database, the tunnel must be opened before hand by running:
user@laptop> ssh mongo-tunnel
More information on SSH settings and using the SSH configuration file can be found with the commands:
man ssh man ssh_config
Users are responsible for their own backups. We highly recommend taking advantage of the NERSC Community Filesystem (CFS) and the archival High Performance Storage System (HPSS).
NERSC does not provide database recovery or restoration as part of our database services, and we do not have the ability to recover or restore individual databases.
We strongly recommended that users perform their own backups for any data that cannot be reproduced or replaced. Please reach out to NERSC support if you need assistance determining an appropriate backup strategy for your project.