{"id":75,"date":"2018-01-16T20:11:57","date_gmt":"2018-01-16T20:11:57","guid":{"rendered":"https:\/\/in.nau.edu\/hpc\/?page_id=75"},"modified":"2024-08-14T08:23:57","modified_gmt":"2024-08-14T15:23:57","slug":"containers","status":"publish","type":"page","link":"https:\/\/in.nau.edu\/arc\/overview\/containers\/","title":{"rendered":"Containers"},"content":{"rendered":"\n
\n
<\/p>\n

\n

\n
\n

Contact Advanced Research Computing<\/h3>\n <\/div>\n
\n \n
\n <\/i>\n Email:<\/span>\n <\/div>\n
ask-arc​@nau.edu<\/div>\n <\/a>\n <\/div>\n<\/div>\n\n
\n<\/div>\n<\/div>\n\n

Containers<\/h1>\n

Using containers in Linux has become a common practice in many business environments. Using technologies like Docker or Kubernetes, it is possible to create self-contained environments or “containers” that are designed for a specific purpose. Unfortunately, the use of these technologies in HPC has been problematic due to security concerns. The open source project Singularity addresses this problem, allowing for the creation of containers that maintain the security infrastructure of the machine they are running on.<\/p>\n

Monsoon users now have access to Singularity, allowing them to run specific codes that were built in a container. Although Docker is not supported on Monsoon, Singularity can be used to import and convert Docker containers, allowing them to be run on Monsoon.<\/p>\n

Basic Usage<\/h2>\n

You can download pre-made singularity images from the singularity hub website. In the following example, we will download and run a simple singularity image called singularity-hello-world<\/span>.<\/p>\n

[abc123@wind ~]$ singularity pull shub:\/\/346\r\nFound image vsoch\/singularity-hello-world:master\r\nDownloading image... 928b4fc5dbe536cc8ba678eb5e37aca814d4d413.img.gz\r\nDecompressing \/home\/mkg52\/928b4fc5dbe536cc8ba678eb5e37aca814d4d413.img.gz\r\n[abc123@wind ~]$ mv 928b4fc5dbe536cc8ba678eb5e37aca814d4d413.img hello.img\r\n[abc123@wind ~]$ singularity run hello.img\r\nHello\r\n[abc123@wind ~]$ singularity run hello.img from inside docker\r\nHello from inside docker<\/code><\/pre>\n
More examples can be found on the web.<\/p>\n
Running Singularity with Slurm<\/h2>\n
Any Singularity image can also be used with Slurm to run jobs. Using the image downloaded in the previous example, we could submit a job array and pass the current array id number as a parameter to the Singularity container (see Job Arrays<\/a> for more information). This is shown in the following code snippet:<\/p>\n
#!\/bin\/bash\r\n#SBATCH --job-name=singularity-test\r\n#SBATCH --array=1-5\r\n\r\nmodule load singularity\r\nsrun singularity run hello.img \"$SLURM_ARRAY_TASK_ID\"<\/code><\/pre>\n
Submitting the job and checking the output files shows that the “Hello” command was run with the inputs 1 through 5:<\/p>\n
[abc123@wind ~]$ sbatch singularity_test.sh\r\nSubmitted batch job 320\r\n[abc123@wind ~]$ ls slurm-*\r\nslurm-320_1.out slurm-320_2.out slurm-320_3.out slurm-320_4.out slurm-320_5.out\r\n[abc123@wind ~]$ cat slurm-*\r\nHello 1\r\nHello 2\r\nHello 3\r\nHello 4\r\nHello 5<\/code><\/pre>\n
If you require mpi support in your Singularity image, please note that the image must be specifically built with this functionality.<\/p>\n
Converting Docker Images to Singularity Images<\/h2>\n
Singularity images cannot be created from scratch on Monsoon without root user permissions.<\/p>\n