Environment Modules¶
Note
Environment modules is available on the Cori system.
Environment modules provides a mechanism to dynamically modify a user's environment using modulefiles. A modulefile is a recipe required to load a particular application that may include setting environment variables; setting variables such as PATH, LD_LIBRARY_PATH, and MANPATH to include the location where an application is installed; loading dependent modules; and providing a brief description of the software.
Modulefiles are written in Tool Command Language (TCL), and are evaluated by the module function when you load/unload modulefiles via module load or module unload.
About module¶
module is a shell function that modifies the user shell upon the loading of a modulefile. The module function is defined as follows:
$ type module
module is a function
module ()
{
eval `/opt/cray/pe/modules/3.2.11.4/bin/modulecmd bash $*`
}
Note
module is not a program, so it cannot be invoked directly in a script.
Module Commands¶
Note
Cray default module is modules/3.2.11.4 and documentation is based on this version.
modules/4.1.3.1 is also available to support non-Cray modulefiles that require Modules 4.x. While most modulefiles at NERSC should work with Modules 4.x, there are some compatibility issue with PrgEnv- modules, and we don't recommend modules/4.1.3.1 for "normal" use.
General usage:
module [switches] [subcommand] [subcommand-args]
Further reading:
module --helpman moduleman modulefile- Environment Modules Documentation (note: some features may only be available in later versions than what is installed on NERSC systems)
Command Summary¶
| Command | Description |
|---|---|
module list | List active modules in the user environment |
module av[ail] [module] | List available modules in MODULEPATH |
module add|load [module] | Load a module file in the user environment |
module rm|unload [module] | Remove a loaded module from the user environment |
module purge | Remove all modules from the user environment |
module swap|switch [module1] [module2] | Replace module1 with module2 |
module show|display [module] | Show content of commands performed by loading module file |
module help [module] | Show help for a given module |
module whatis [module] | A brief description of the module, generally single line |
module use [-a] [path] | Prepend or Append path to MODULEPATH |
module unuse [path] | Remove path from MODULEPATH |
module keyword [text] | Search for keyword across all module files |
Module Usage¶
Show availability of all modules containing a substring:
module avail -S <substring>
Display what changes are made when a module is loaded:
module display <module-name>
module show <module-name>
Add a module to your current environment:
module load <module-name>
module add <module-name>
Note
Loading/adding a module is a silent operation unless problems occur with the module.
Tip
If you load the name of a module (with no version), you will get the default version.
module load gcc
To load a specific version use the full name:
module load gcc/8.1.0
Remove a module from the current environment:
module unload <module-name>
module rm <module-name>
Note
Unloading/removing will fail silently if the specified module is not loaded.
Switch currently loaded module with a new module:
module swap <old-module> <new-module>
module switch <old-module> <new-module>
To purge all modules:
Note
This will remove all your modules from active environment including startup modules loaded in your shell, which may make your environment unusable at NERSC. To restore your environment with startup modules, we suggest to logout and login again instead.
module purge
To visualize the help of a particular module:
module help <module-name>
The module whatis command displays a brief summary of the module, while module help will provide the full description of modulefile.
$ module whatis PrgEnv-cray
PrgEnv-cray: Programming environment using the Cray CCE compilers.
Note
If a module does not have a help or whatis section in its modulefile, you will see an empty line when you run module help or module whatis
To see a condensed list of modules you can use the -t option with list or avail.
$ module -t list
Currently Loaded Modulefiles:
modules/3.2.11.4
nsg/1.2.0
altd/2.0
darshan/3.1.7
intel/19.0.3.199
craype-network-aries
craype/2.6.2
cray-libsci/19.06.1
udreg/2.3.2-7.0.1.1_3.29__g8175d3d.ari
ugni/6.0.14.0-7.0.1.1_7.32__ge78e5b0.ari
pmi/5.0.14
dmapp/7.1.1-7.0.1.1_4.43__g38cf134.ari
gni-headers/5.0.12.0-7.0.1.1_6.27__g3b1768f.ari
xpmem/2.2.20-7.0.1.1_4.8__g0475745.ari
job/2.2.4-7.0.1.1_3.34__g36b56f4.ari
dvs/2.12_2.2.156-7.0.1.1_8.6__g5aab709e
alps/6.6.57-7.0.1.1_5.10__g1b735148.ari
rca/2.2.20-7.0.1.1_4.42__g8e3fb5b.ari
atp/2.1.3
PrgEnv-intel/6.0.5
craype-haswell
cray-mpich/7.7.10
craype-hugepages2M
On Cori there are three Cray-provided Programming Environment modules: PrgEnv-cray, PrgEnv-gnu and PrgEnv-intel (the default). These modules provide respectively the Cray, GNU or Intel compiler via the compiler wrappers cc (C) CC (C++) and ftn (Fortran); see also this page about using compiler wrappers.
Only one PrgEnv module can be loaded at a time: for instance if we try loading PrgEnv-gnu without first unloading PrgEnv-intel we get the following error message.
$ module load PrgEnv-gnu
PrgEnv-gnu/6.0.5(77):ERROR:150: Module 'PrgEnv-gnu/6.0.5' conflicts with the currently loaded module(s) 'PrgEnv-intel/6.0.5'
PrgEnv-gnu/6.0.5(77):ERROR:102: Tcl command execution failed: conflict PrgEnv-intel
To change programming environments, you must swap modules as follows:
module swap PrgEnv-intel PrgEnv-gnu
Or unload the old module and load the new one:
module unload PrgEnv-intel
module load PrgEnv-gnu
Creating a Custom Module Environment¶
There are ways to modify your environment so that certain modules are automatically loaded when you log in.
The first option is to use shell commands.
bash¶
In ~/.bash_profile
module swap PrgEnv-${PE_ENV,,} PrgEnv-gnu
csh¶
In ~/.login
set pe = ` echo $PE_ENV | tr "[:upper:]" "[:lower:]" `
module swap PrgEnv-${pe} PrgEnv-gnu
Snapshots¶
The second option is to use the "snapshot" feature of modules.
- Swap and load modules to your desired configuration.
- Save a "snapshot" with
module snapshot <snapshot-filename>.
Then at any time later restore the environment with module restore <snapshot-filename>.
Install Your Own Customized Modules¶
You can create and install your own modules for your convenience or for sharing software among collaborators. See man modulefile or the modulefile documentation for details of the required format and available commands. These custom modulefiles can be made visible to the module command with module use /path/to/the/custom/modulefiles.
Tip
Global Common is the recommended location to install software.
Note
Make sure the UNIX file permissions grant access to all users who want to use the software.
Warning
Do not give write permissions to your home directory to anyone else.
Note
The module use command prepends new directories before other module search paths (defined as $MODULEPATH), so modules defined in a custom directory will have precedence if there are other modules with the same name in the module search paths. If you prefer to have the new directory added at the end of $MODULEPATH, use module use -a instead of module use.
Known issues with modules¶
Zero exit code for invalid modules¶
$ module load X
ModuleCmd_Load.c(244):ERROR:105: Unable to locate a modulefile for 'X'
$ echo $?
0
This means that module commands often return a "success" code (0) even if the command failed, which can lead to surprising errors in job scripts.
Incompatibilities with modules/4.1.3.1¶
modules/4.1.3.1 is available but not recommended. If you do need Modules 4.x, you can access it with module swap modules modules/4.1.3.1 (note that module load modules/4.1.3.1 will abort with an error). If using Modules 4.x, we recommend carefully checking that your script or usage has the correct outcome.
Please refer to https://modules.readthedocs.io/en/latest/diff_v3_v4.html for a summary of differences between module v3.2 and 4.x.
Module FAQ¶
- Is there an environment variable that captures loaded modules?
Yes, active modules can be retrieved via the $LOADEDMODULES environment variable, which is automatically updated to reflect active loaded modules (similar to module list). You can access the modulefile path for loaded modules through the $_LMFILES_ environment variable.
- How can I restore MODULEPATH in my user session?
If you run into an error such as following:
$ module avail
ModuleCmd_Avail.c(217):ERROR:107: 'MODULEPATH' not set
You should try a different login shell and see if it fixes the issue (i.e., bash users try csh; csh users try bash). Then check your startup scripts to see whether the issue is caused by something there. Bash users should examine the files ~/.bashrc and ~/.bash_profile; tcsh/csh users should look at ~/.cshrc.