Shifter for Beginners Tutorial¶
This tutorial is meant to demonstrate all the steps you need to build a Docker container, upload it, and run it as a Shifter image on Cori at NERSC.
- Install Docker on your local system
- Set up your free account on Docker Hub
- Build your hello-world Docker image on your local machine
- Run your hello-world Docker container on your local machine
- Push your Docker image to the Docker Hub registry
- Pull your Docker image onto Cori via Shifter
- Inspect your Shifter image on Cori
- Submit an interactive job using your Shifter image on Cori
- Submit a batch job using your Shifter image on Cori
- Go forth and be productive with Shifter!
This tutorial works on both Cori and Perlmutter
We'll demonstrate all of the Shifter functionality in this tutorial on Cori, but everything we show works on Perlmutter, too.
Install Docker on your local system¶
For now, NERSC does not offer users any place where they can build their own Docker containers. (We are working on this!) This means that users will need to build their images on their own systems before pulling them to NERSC.
Please download and install the Docker client for your appropriate operating system. Once installed, make sure the Docker daemon (or client) is running. Note that this will likely require you to have admin permissions on your system.
Set up your free account on Docker Hub¶
Since you'll need to push your image to Docker Hub, it's easiest to set up your account and choose your username before you build your image. You may find it easier to use your NERSC username if possible. When you have your Docker Hub username, you can use it in the next few steps. Note that there are many other registries out there, but we're using Docker Hub here since it's currently a very commonly used option.
Build your hello-world Docker image on your local machine¶
Let's create our first image that will print
hello world from inside a container! when we run it. Please make yourself a directory called
docker and inside of it, paste the following contents into a file called
FROM ubuntu:latest ENTRYPOINT echo "hello from inside a container!"
What do these commands do?
FROM ubutu:latest downloads an existing Docker image that already contains the Ubuntu operating system. In general it's a good idea to use some of the many existing building blocks within the Docker (and wider container) ecosystem.
ENTRYPOINT echo "hello from inside a container!" will execute your echo statement to print hello when the container runs. Note that Docker will always execute the
ENTRYPOINT, which we'll see is different than in Shifter.
Now let's build your container. To make things easier later, let's prefix it with your Docker Hub username. We'll assume that your Docker Hub and NERSC username is
elvis, so please substitute your own username where it appears. If your Docker Hub username is different than your NERSC username, make sure you name your image with your Docker Hub username. This will make it easy to push to your Docker Hub account when you are ready.
docker build -t elvis/hello-world:1.0 .
What does this do? It simultaneously builds your Docker image based on what you specified in your Dockerfile and also names and tags your image with your Docker Hub username
elvis, the name you specified for your image
hello-world, and the tag you gave your image
1.0. Now that it's appropriately named and tagged, it will be easier to push to Docker Hub when you're ready.
Other useful Dockerfile commands:
RUN will run a command inside the container environment to modify its contents. This is usually used for installing and configuring software.
COPY will copy files into your Docker image.
WORKDIR will establish or change the current working directory during the Docker build. It will also affect the default directory that used when you run a container with this image.
ENV can be used for setting or appending to existing environment variables.
Run your hello-world Docker container on your local machine¶
Your Docker image has been built and is now ready to run as a container. An instantiation of an image is a container, so that's why we call it a container in the context of running.
elvis$ docker run --rm elvis/hello-world:1.0 hello from inside a container!
Here you are running your Docker image. The
--rm flag instructs the Docker daemon to clean up the Docker container after you are done running it. You don't need this, but if you don't use it, you'll have a lot of unused Docker containers accumulating on your system that you will eventually need to clean up.
Congratulations, you've now built and run your first Docker container.
Push your Docker image to the Docker Hub registry¶
In order to get your Docker image onto NERSC systems, you will need to push your image to public registry such as Docker Hub. Here we are assuming you have already set up your Docker Hub account. You will need to log in to the Docker public registry via
docker login and then you will be prompted for your username and password. You will typically only need to log in once and Docker will store your login credentials for later.
Once you've done that, you can push your image via
docker push elvis/hello-world:1.0
This will push your image to your Docker Hub account. Please note that if you are using the free account, all Dockerfiles and images are public.
Pull your Docker image onto Cori via Shifter¶
Once your image is pushed to Docker Hub, we can pull the image onto Cori using Shifter. Shifter is configured to pull images directly from Docker Hub via the
shifterimg pull command, which will automatically convert your Docker image into Shifter format.
shifterimg pull elvis/hello-world:1.0
Inspect your Shifter image on Cori¶
You can run your Shifter container interactively on a Cori login node. The Shifter
--image flag is used to select your container and is followed by arguments that will be run inside the container. If we specify
/bin/bash, we will start a bash shell inside our container.
elvis@cori07:~> shifter --image=elvis/hello-world:1.0 /bin/bash elvis@cori07:~$ cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS" elvis@cori07:~$ exit exit
For comparison, note that Cori's operating system is SUSE Linux, so this is additional verification that we are in fact executing this command inside our container.
Once inside your image, you can verify the location of your software, the environment variables you have set, etc. You can even run it on our login nodes, although we kindly request that you adhere to our general login node use policies. When you are done inspecting your image, type
exit to leave your container.
Shifter users should note that their prompt will not change once inside a Shifter container. If you can't remember whether you have typed
exit and are still inside your Shifter container, there are several ways to check:
env | grep "SHIFTER_RUNTIME"- this will return 1 if you are inside a Shifter container
cat /etc/os-release | grep "SUSE"- this will check to see if the OS is SUSE, which is Cori's OS. Note that this will not return anything if you are inside a Shifter container.
echo $LD_LIBRARY_PATH | grep "udiImage"- this will look to see if there are any Shifter modules in your
LD_LIBRARY_PATH. Note this will not return anything if you are outside a Shifter container.
Submit an interactive job using your Shifter image on Cori¶
To use your Shifter image inside an interactive job, you can submit as you normally would with the addition of the image you would like to use
salloc -N 1 -t 60 -C haswell -q interactive --image=elvis/hello-world:1.0
This will make your image available to your job. However, to actually use it, you'll need to issue the
shifter command. Anything that follows this command will be executed inside your container. One difference from our Docker example is that you must ask Shifter to use your
ENTRYPOINT with the
--entrypoint flag. Let's try it:
elvis@nid00283:~> srun shifter --entrypoint hello from inside a container!
Submit a batch job using your Shifter image on Cori¶
Submitting a batch job with a Shifter image is very similar. You'll need to request the image you need in the
SBATCH directives. Here is an example jobscript we'll call
#!/bin/bash #SBATCH --image=elvis/hello-world:1.0 #SBATCH --qos=debug #SBATCH --constraint=haswell #SBATCH -t 00:02:00 #SBATCH -N 1 #SBATCH -o output.o%j srun shifter --entrypoint
We can submit this job with
If we check the output, we'll see
elvis@cori07:> cat output.o50623860 hello from inside a container!
Let's say we don't want to use the
ENTRYPOINT we have specified in our image. Instead let's go ahead and print our Ubuntu information like we did above.
#!/bin/bash #SBATCH --image=elvis/hello-world:1.0 #SBATCH --qos=debug #SBATCH --constraint=haswell #SBATCH -t 00:02:00 #SBATCH -N 1 #SBATCH -o output.o%j srun shifter cat /etc/lsb-release
The output of this job looks like
elvis@cori07:> cat output.o50626055 DISTRIB_ID=Ubuntu DISTRIB_RELEASE=20.04 DISTRIB_CODENAME=focal DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
This should look familiar-- it should be same output you saw in our Inspect your Shifter image on Cori. You can substitute in any command you like after the
shifter command. This command will run inside your container. Keep in mind that you may need to
COPY or bind-mount a directory if you would like a script to be accessible inside your container.
Go forth and be productive with Shifter¶
Now that you have finished this tutorial, you might like some more detailed information about the different ways to use Shifter at NERSC. Here are several resources:
- How to use Shifter, a more detailed guide to Shifter at NERSC
- Using GPUs in Shifter, more information about running GPU code in Shifter with a focus on Perlmutter
- Using Shifter in Jupyter, explains how you can use a Shifter image as a Jupyter kernel
- Example Dockerfiles, provides some examples from our staff and users
- NERSC examples, a new effort to try to provide some official NERSC example images for users. Suggestions for useful images are welcome at our helpdesk.