Portability Between Containers¶

For Users Who Build and Run Containers using podman-hpc¶

You may be able to continue to build containers on Perlmutter as you have before, or you may need to build using a separate resource, such as building locally using Docker.

You will need to push your container to a registry.

• Log into a registry. If using podman-hpc on Perlmutter, this command will be similar to podman-hpc login docker.io
• Push your container to that registry. This will look something like podman-hpc push mycontainer:24.03 docker.io/user/mycontainer:24.03
• Information can be found in the NERSC documentation about registries for DockerHub for DockerHub and registries for registry.nersc.gov.

You will then pull your container onto Perlmutter using Shifter

• You will need to log in to the registry with a command similar to shifterimg login docker.io
• Pull the image with a command similar to shifterimg pull docker:godlovedc/lolcow:latest
• For more information on logging in to a registry and pulling an image, please see our documentation
• Note that you do not need to migrate a container in shifter as you would with podman-hpc

You will run your container using Shifter

• You will use a command like shifter --image=godlovedc/lolcow:latest --entrypoint
• See the Shifter documentation for how to run interactively on a login node, using an interactive job, and as a batch job.
• Several changes will need to be made to your podman-hpc run commands ** Modules like –gpu are put in with –module=gpu ** Shifter mounts must be made to locations that already exist and are available within the container
• Note that you may need to provide access to the nobody user via ACLs for the particular directories that you wish to mount.

For Users Who Pull and Run Containers using podman-hpc¶

You will then pull your container onto Perlmutter using Shifter

• You will need to log in to the registry with a command similar to shifterimg login docker.io
• Pull the image with a command similar to shifterimg pull docker:godlovedc/lolcow:latest
• For more information on logging in to a registry and pulling an image, please see our documentation
• Note that you do not need to migrate a container in shifter as you would with podman-hpc

You will run your container using Shifter

• You will use a command like shifter --image=godlovedc/lolcow:latest --entrypoint
• See the Shifter documentation for how to run interactively on a login node, using an interactive job, and as a batch job.
• Several changes will need to be made to your podman-hpc run commands ** Modules like –gpu are put in with –module=gpu ** Shifter mounts must be made to locations that already exist and are available within the container
• Note that you may need to provide access to the nobody user via ACLs for the particular directories that you wish to mount.

Rosetta Stone¶

Common Issues¶

There are several differences between Shifter and podman-hpc that may cause users issues.

Shifter uses the nobody user to provide access. The nobody user must be granted access through the use of ACLs to any locations being used.

Shifter overwrites certain directories when the container is run. This includes /etc, /var/ and /tmp.

Shifter containers do not need to be migrated like podman-hpc containers.

Shifter requires you to run as a non-root user and so your container may have to be rearchitected and rebuilt. Often, this is as easy as including an additional RUN chmod -R o+rx /installed/software/path in your Containerfile.

Shifter requires that the location for the mount exist within the container and be accessible to a non-root user. For example, mounting to /mountHere will create a directory called mountHere within /, but mounting to /middleFolder/mountHere will only work if /middleFolder already exists. Additionally, a trailing / causes issues. You are unable to mount to /mountHere/ unless the directory already exists. See the Volume Mapping section of shifter -h for more information about this.

Further troubleshooting¶

If you have a Shifter question or problem, please open a ticket at help.nersc.gov.