NetCDF¶

NetCDF (Network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This includes the NetCDF library as well as the NetCDF Operators (NCO), Climate Data Operators (CDO), NCCMP, and NCVIEW packages.

Using NetCDF at NERSC¶

NetCDF libraries are provided by HPE, for each of the PrgEnvs. To check the available Cray installations and versions use the following command:

module spider cray-netcdf   (# on Perlmutter)

No compile or link options are required as long as you use the Cray compiler wrappers ftn, cc, and CC. Below is an example:

module load cray-hdf5
module load cray-netcdf
ftn my_netcdf_code.f90
cc  my_netcdf_code.c
CC  my_netcdf_code.cpp

Other NetCDF tools at NERSC¶

NERSC provides several tools, which can be useful for analyzing and manipulating NetCDF data. These tools, including NCO, NCCMP, CDO, NCView, and NCL, are provided as part of the climate-utils module.

Loading climate-utils module will cause compiler wrappers to fail, so it is not recommended to load this module in your dot files. Please unload this module before compiling your applications with compiler wrappers (ftn, cc, or CC).

The climate-utils module also provides the serial HDF5 and NetCDF package. If you load another module which also contains this package (such as cray-netcdf) as well as the climate-utils module, you will be using the utilities and libraries (such as ncdump) from the module loaded later.

NetCDF Operators (NCO)¶

The NetCDF Operators (NCO) are a suite of file operators that facilitate manipulation and analysis of self-describing data stored in the NetCDF or HDF4/5 formats.

NCO is available on Perlmutter as part of the climate-utils module.

To use NCO:

module load climate-utils

There are many NetCDF Operator tools, including:

• ncap2: netCDF Arithmetic Processor
• ncatted: netCDF ATTribute EDitor
• ncbo: netCDF Binary Operator (addition, multiplication...)
• ncclimo: netCDF CLIMatOlogy Generator
• nces: netCDF Ensemble Statistics
• ncecat: netCDF Ensemble conCATenator
• ncflint: netCDF FiLe INTerpolator
• ncks: netCDF Kitchen Sink
• ncpdq: netCDF Permute Dimensions Quickly, Pack Data Quietly
• ncra: netCDF Record Averager
• ncrcat: netCDF Record conCATenator
• ncremap: netCDF REMAPer
• ncrename: netCDF RENAMEer
• ncwa: netCDF Weighted Averager

Find usage information via <nco_tool_name> --help.

NCCMP¶

The NCCMP tool compares two NetCDF files bitwise or with a user-defined tolerance (absolute or relative percentage). NCCMP is available on Perlmutter as part of the climate-utils module.

To use NCCMP:

module load climate-utils

NCCMP usage:

nccmp [OPTION...] file1 [file2]

Find more information via nccmp --help.

CDO¶

Climate Data Operators (CDO) is a large set of tools for working on climate data. NetCDF 3/4, GRIB (including SZIP compression), EXTRA, SERVICE, and IEG are supported as I/O-formats. CDO can also be used to analyse any kind of gridded data not related to climate science. CDO has very small memory requirements and can process files larger than physical memory.

CDO is available on Perlmutter as part of the climate-utils module.

To use CDO:

module load climate-utils

CDO usage:

cdo  [Options]  Operator1  [-Operator2  [-OperatorN]]

Find more information via cdo --help.

NCView¶

NCView is a visual browser for NetCDF format files. To use NCView, first set up the necessary X11 environment (NERSC strongly recommends using NoMachine). Or use the -Y option when connecting to NERSC through ssh.

NCView is available on Perlmutter as part of the climate-utils module.

To use NCView:

module load climate-utils

NCView usage:

ncview [options] datafiles

Find more information via ncview --help.

NCL¶

NCL (NCAR Command Language) is an interpreted language designed specifically for scientific data analysis and visualization. See more information on Using NCL at NERSC.

NetCDF4-python¶

netCDF4-python is an object-oriented python interface to the NetCDF library. It is available via the python module.