Julia¶
Julia is a high-level, high-performance dynamic programming language for technical computing. It has syntax that is familiar to users of many other technical computing environments. Designed at MIT to tackle large-scale partial-differential equation simulation and distributed linear algebra, Julia features a robust ecosystem of tools for optimization, statistics, parallel programming, and data visualization. Julia is actively developed by teams at MIT and in industry, along with hundreds of domain-expert scientists and programmers from around the world.
Using Julia at NERSC¶
Multiple Julia versions are available at NERSC. You can see which ones we currently support by running module avail julia which will print out a list of julia versions -- for example:
$ module avail julia
--------------------------- NERSC-provided Software ----------------------------
julia/1.8.5 julia/1.9.2 julia/1.9.3 julia/1.9.4 (D)
Where:
D: Default Module
...
You can then load a specific version of julia by specifying julia/<version> -- for example:
module load julia/1.9.4
Or you can load the default version available by omitting the version -- eg:
module load julia
Default Julia module version
The default module will almost always refer to the latest version of Julia. If a version of Julia has a known issue, then the default will skip that version.
Get access to the latest versions of Julia
We do not always include the very latest version of Julia in the system RPM. However, that shouldn't stop you form using the latest version of Julia. Just run module use /global/common/software/nersc/n9/julia/modules, now you should be able to see all of the latest versions of Julia that are installed at NERSC.
NERSC-Specific Preferences¶
Our Julia modules also configure the JULIA_LOAD_PATH to include NERSC-specific prferences. Note that the Julia modules append the Julia preferences path to the end of JULIA_LOAD_PATH. Therefore, you can always override NERSC-specific preferences using a LocalPreferences.toml in a project at an earlier location in the JULIA_LOAD_PATH.
Using an External Julia Install Risks Broken Configurations
Some packages need to interact with the system software in a very specific way. The section on MPI and CUDA is such an exmaple. We therefore strongly recommend that you use the preferences from our Julia modules.
Using NERSC-specific settings with your own version of Julia¶
Sometimes it is desirable to bring your own version of Julia. For example, you might have a well-established bootstrapping process on new systems. It is still possible to use your own version of Julia in combination with the NERSC-specific settings. It is for this reason that we do not provide a global JULIA_DEPOT as any packages therein would be tied to specific Julia versions. Instead we recommend that you use our Juliaup install that is included with every Julia module.
For example, this installs julia version 1.10.2 to $SCRATCH and makes use of NERSC-specific settings.
$ module load julia
$ export JULIAUP_DEPOT_PATH=$SCRATCH/julia
$ juliaup add 1.10.2
Installing Julia 1.10.2+0.x64.linux.gnu
$ export PATH=$JULIAUP_DEPOT_PATH/juliaup/julia-1.10.2+0.x64.linux.gnu/bin:$PATH
$ which julia
/pscratch/sd/e/elvis/julia/juliaup/julia-1.10.2+0.x64.linux.gnu/bin/julia
Upon new logins, you can use this installation with the following.
module load julia # for setting JULIA_LOAD_PATH to load NERSC-specific perferences
export JULIAUP_DEPOT_PATH=$SCRATCH/julia
export PATH=$JULIAUP_DEPOT_PATH/juliaup/julia-1.10.2+0.x64.linux.gnu/bin:$PATH
Using Julia with MPI and CUDA¶
As described in the previous section, we do not pre-install packages like MPI.jl and CUDA.jl (many users prefer specific versions, making a central installation unmaintainable). Instead MPI.jl and CUDA.jl will detect any relevant system settings when firstly installed.
However it is always good to check that these settings have been correctly applied. For this reason, many packages that rely on interactions with the system software will provide a versioninfo() function.
For example,
julia> using MPI
julia> MPI.versioninfo()
MPIPreferences:
binary: system
abi: MPICH
libmpi: libmpi_gnu_123.so
mpiexec: srun
Package versions
MPI.jl: 0.20.19
MPIPreferences.jl: 0.1.10
Library information:
libmpi: libmpi_gnu_123.so
libmpi dlpath: /opt/cray/pe/lib64/libmpi_gnu_123.so
MPI version: 3.1.0
Library version:
MPI VERSION : CRAY MPICH version 8.1.28.29 (ANL base 3.4a2)
MPI BUILD INFO : Wed Nov 15 20:57 2023 (git hash 1cde46f)
Indicates that CRAY MPICH is in use by MPI.jl -- which is exactly what we want!
Similarly for CUDA.jl:
julia> using CUDA
julia> CUDA.versioninfo()
CUDA runtime 12.2, local installation
CUDA driver 12.3
NVIDIA driver 525.105.17, originally for CUDA 12.2
CUDA libraries:
- CUBLAS: 12.2.1
- CURAND: 10.3.3
- CUFFT: 11.0.8
- CUSOLVER: 11.5.0
- CUSPARSE: 12.1.1
- CUPTI: 20.0.0
- NVML: 12.0.0+525.105.17
Julia packages:
- CUDA: 5.1.2
- CUDA_Driver_jll: 0.7.0+1
- CUDA_Runtime_jll: 0.10.1+0
- CUDA_Runtime_Discovery: 0.2.3
Toolchain:
- Julia: 1.9.4
- LLVM: 14.0.6
Preferences:
- CUDA_Runtime_jll.version: 12.2
- CUDA_Runtime_jll.local: true
1 device:
0: NVIDIA A100-PCIE-40GB (sm_80, 17.308 GiB / 40.000 GiB available)
Indicates that a the locally-installed CUDA runtime is in use -- also exactly what we want.
Share your versioninfo in support tickets
If you're encountering problems with either MPI.jl, or CUDA.jl, and you've opened a support-ticket, please include the output from MPI.versioninfo() and CUDA.versioninfo().
Running MPI.jl Programs¶
You can use Slurm's srun as usual:
srun -n N julia prog.jl
or you can use the mpiexecjl wrapper provided by Julia:
mpiexecjl -n N julia prog.jl
Note that, in order to use Julia's mpiexecjl, you need to install it first:
julia> using MPI
julia> MPI.install_mpiexecjl()
[ Info: Installing `mpiexecjl` to `/global/homes/e/elvis/.julia/bin`...
[ Info: Done!
Then add ~/.julia/bin to your PATH with export PATH=~/.julia/bin:$PATH.
Developing Julia Code for GPUs¶
Please take a look at our Perlmutter transition guide's section on Julia.
