Installing and Sharing Software¶

This page describes considerations for traditional software distribution methods.

NERSC File Systems¶

When installing software on NERSC's file systems it is recommended to use the /global/common/software/<your NERSC project> directories. This file system is optimized for installing and sharing software as it is mounted read-only on the compute nodes.

See our page about the global file system for more information.

Many user applications offer the option to specify the installation prefix when building a software package; that should be set to your directory in /global/common/software/<your NERSC project> as part of the configuration process.

Best Practices: Software Installation¶

Install software in a prefix such as

/global/common/software/<your NERSC project>/public/<name>-<version>-<build ID>

where <build ID> could be a git hash or the output of date +%s. Then make a link to a standard name for this installation

cd /global/common/software/<your NERSC project>/public/
ln -s <name>-<version>-<build ID> <name>-<version>

This allows for seamless deployments of new builds of packages if needed and avoids issues with the technical details of how the contents of read-only file systems are cached by compute nodes.

Best Practices: Software Build Management¶

It is common (and recommended!) practice to manage your software source code with a version control system such as git. In addition to managing your source files in this way, we also recommend keeping your build configuration (installation scripts, dependency lists, Containersfiles, etc.) under version control to streamline your software deployment accross environment updates and different compute sites.

Sharing Software and Managing Access¶

If you wish to provide your software only to the members of your project then this should be compatible with the default permissions for

/global/common/software/<your NERSC project>

However, if you wish to make an installation available generally to all NERSC users those users need:

• read permissions for the contents
• execute permissions for all binaries and parent directories

If your software is installed in /global/common/software/<your NERSC project>/public then in practice the following commands can be run to enable broad access by all NERSC users:

chmod o+x /global/common/software/<your NERSC project>
chmod -R o+rX /global/common/software/<your NERSC project>/public

See man chmod for further explanation.

Modulefiles¶

It may be desirable to provide a modulefile for your software. NERSC modulefiles are provided via Lmod but the module command provided by Lmod is also compatible with TCL environment modules with some caveats.

Modulefiles placed in

/global/common/software/<your NERSC project>/public/modulefiles/<name>/<version>.lua

will be available to other users (with appropriate Unix permissions) via

module use /global/common/software/<your NERSC project>/public/modulefiles
module load <name>/<version>

modulefile template¶

-- Required internal variables
local name               = myModuleName()
local version            = myModuleVersion()
local n9_prefix          = "/global/common/software/<your NERSC project>/public/"
local root       =  pathJoin(n9_prefix, name, version)

-- Prerequisite conditions
-- conflict("conflicting-package")

-- Product Description (optional)
local externalurl        = "https://www.example.com"
local description        = "This is a short one-line description about the software"

help (
[[========================================================================== ]] .. "\n" ..
[[Name: ]] .. name .. "\n" .. 
[[Version: ]] .. version .. "\n" ..
[[URL: ]] .. externalurl .. "\n" ..
[[Description: ]] .. description .. "\n" ..
[[========================================================================== ]] .. "\n" ..

[[
This is an extended description explaining how to use this software at NERSC.

Please provide as much description as possible to help users understand what this software
is with links to other material


References:
  - https://example.com
]]
)

whatis(description)

-- Environment Variables
setenv("DEMO_DIR", root)
setenv("DISABLE_MPI", "1")

-- User Environment
prepend_path("PATH", pathJoin(root, "bin"))
prepend_path("LD_LIBRARY_PATH", pathJoin(root, "lib64"))
prepend_path("LIBRARY_PATH", pathJoin(root, "lib64"))