Singularity

Singularity

Singularity is a container platform. A container is a single file and it is image based.

  • Singularity is an opensource project that was created to run complex applications on HPC clusters.
  • It enables users to have full control of their environment. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible.
  • You can build a container using Singularity on your laptop, and then run it on HPC clusters.
  • Singularity also allows you to leverage the resources of whatever host you are on. This includes HPC interconnects, resource managers, file systems, GPUs and/or accelerators, etc.

A non-privileged user can "swap out" the operating system on the host for one they control. So if the host system is running RHEL6 but your application runs in Ubuntu/RHEL7, you can create an Ubuntu/RHEL7 image, install your applications into that image, copy the image to another host, and run your application on that host in its native Ubuntu/RHEL7 environment.

Singularity containers can be in three different formats:

  • read-only squashfs (default) – vhodný pre produkčný image;
  • writable ext3 (–writable option);
  • writable (ch)root directory (--sandbox option) - best for development.

Squashfs and (ch)root directory images can be built from Docker source directly on the cluster, no root privileges are needed. It is strongly recommended to create a native Singularity image to speed up the launch of the container. Singularity is available in Kosice cluster.

GPU Example

Tensorflow is commonly used for machine learning projects. The official tensorflow repository on Docker Hub contains NVIDA GPU supporting containers, that will use CUDA for processing.

singularity pull docker://tensorflow/tensorflow:latest-gpu

An example is located in /lustre/home/freeware/EXAMPLE_JOBS/gpu.

Commands that run, or otherwise execute containers (shell, exec) can take an --nv option, which will setup the container’s environment to use an NVIDIA GPU and the basic CUDA libraries to run a CUDA enabled application.

You can run CUDA application on compute nodes which have GPUs: comp[17-24,39-56]. There is run script test.sh, which is simple tensorflow example. You can run it with command sbatch test.sh.

OpenMPI Example

An openmpi example is located in /lustre/home/freeware/EXAMPLE_JOBS/singularity/openmpi/.
In this example mpiexec command is executed on singularity container. Inside container is installed the same version of openmpi. Image was built on local linux machine using recipe file openmpi-test.recipe with root privileges: sudo singularity build openmpi.simg openmpi-test.recipe

During the process of creating the image openmpi with infiniband support is built from source code and simple test is compiled. You can run the example with command sbatch test.sh.

The interactive shell can be invoked by the singularity shell command. This is useful for development purposes. Use the -w | --writable option to make changes inside the container permanent.

A user home directory is mounted inside the container automatically. If you need access to the /scratch local disk storage for your computation, this must be mounted by the -B | --bind option. /scratch na lokálnom disku, toto spravíte pomocou prepínača -B | –bind.

A complete documentation can be found at the Singularity webpage. Singularity.