{"id":504,"date":"2020-11-08T20:43:49","date_gmt":"2020-11-08T19:43:49","guid":{"rendered":"https:\/\/eurocc.nscc.sk\/?page_id=504"},"modified":"2021-02-01T14:44:47","modified_gmt":"2021-02-01T13:44:47","slug":"singularity","status":"publish","type":"page","link":"https:\/\/eurocc.nscc.sk\/en\/services\/manualy\/singularity\/","title":{"rendered":"Singularity"},"content":{"rendered":"<div class=\"is-layout-flow wp-block-group manuals\"><div class=\"wp-block-group__inner-container\">\n<h4>Singularity<hr style=\"border: 1px solid #b8870b; width: 100px;\"><\/h4>\n\n\n\n<p class=\" translation-block\"><a href=\"https:\/\/sylabs.io\/\" target=\"_self\">Singularity<\/a> is a container platform. A container is a single file and it is image based.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-medium\"><a href=\"https:\/\/sylabs.io\/\"><img decoding=\"async\" loading=\"lazy\" width=\"297\" height=\"300\" src=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo-297x300.png\" alt=\"\" class=\"wp-image-527\" srcset=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo-297x300.png 297w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo-12x12.png 12w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo.png 495w\" sizes=\"(max-width: 297px) 100vw, 297px\" \/><\/a><\/figure><\/div>\n\n\n\n<ul><li>Singularity is an opensource project that was created to run complex applications on HPC clusters.<\/li><li>It enables users to have full control of their environment. It allows you to create and run containers that package up pieces of software in a way that is portable and reproducible.<\/li><li>You can build a container using Singularity on your laptop, and then run it on HPC clusters.<\/li><li>Singularity also allows you to leverage the resources of whatever host you are on. This includes HPC interconnects, resource managers, file systems, GPUs and\/or accelerators, etc.<\/li><\/ul>\n\n\n\n<p>A non-privileged user can \"swap out\" the operating system on the host for one they control. So if the host system is running RHEL6 but your application runs in Ubuntu\/RHEL7, you can create an Ubuntu\/RHEL7 image, install your applications into that image, copy the image to another host, and run your application on that host in its native Ubuntu\/RHEL7 environment.<\/p>\n\n\n\n<p>Singularity containers can be in three different formats:<\/p>\n\n\n\n<ul><li>read-only <strong>squashfs<\/strong> (default) \u2013 vhodn\u00fd pre produk\u010dn\u00fd image;<\/li><li>writable <strong>ext3<\/strong> (&#8211;writable option);<\/li><li>writable <strong>(ch)root directory<\/strong> (--sandbox option) - best for development.<br><\/li><\/ul>\n\n\n\n<p>Squashfs and (ch)root directory images can be built from Docker source directly on the cluster, no root privileges are needed. It is strongly recommended to create a native Singularity image to speed up the launch of the container. Singularity is available in Kosice cluster.<\/p>\n\n\n\n<p class=\"has-large-font-size\"><strong>GPU Example<\/strong><\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-medium\"><a href=\"https:\/\/medium.com\/singularityapp\/singularity-containers-tensorflow-and-the-nvidia-jetson-nano-an-experiment-5be66ebbd4ac\"><img decoding=\"async\" loading=\"lazy\" width=\"300\" height=\"293\" src=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-300x293.png\" alt=\"\" class=\"wp-image-522\" srcset=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-300x293.png 300w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-1024x999.png 1024w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-768x749.png 768w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-12x12.png 12w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu.png 1050w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/figure><\/div>\n\n\n\n<p>Tensorflow is commonly used for machine learning projects. The official tensorflow repository on Docker Hub contains NVIDA GPU supporting containers, that will use CUDA for processing.<\/p>\n\n\n\n<p>singularity pull docker:\/\/tensorflow\/tensorflow:latest-gpu<\/p>\n\n\n\n<p>An example is located in \/lustre\/home\/freeware\/EXAMPLE_JOBS\/gpu.<\/p>\n\n\n\n<p>Commands that run, or otherwise execute containers (shell, exec) can take an --nv option, which will setup the container\u2019s environment to use an NVIDIA GPU and the basic CUDA libraries to run a CUDA enabled application.<\/p>\n\n\n\n<p>You can run CUDA application on compute nodes which have GPUs: comp[17-24,39-56]. There is run script test.sh, which is simple tensorflow example. You can run it with command sbatch test.sh. <\/p>\n\n\n\n<p class=\"has-large-font-size\"><strong>OpenMPI Example<\/strong><\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-medium\"><a href=\"https:\/\/kks32-courses.gitbook.io\/hpc-containers\/singularity\/building-a-container-configuration-file\/hands-on-3\"><img decoding=\"async\" loading=\"lazy\" width=\"300\" height=\"274\" src=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi-300x274.png\" alt=\"\" class=\"wp-image-525\" srcset=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi-300x274.png 300w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi-13x12.png 13w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi.png 600w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/figure><\/div>\n\n\n\n<p>An openmpi example is located in \/lustre\/home\/freeware\/EXAMPLE_JOBS\/singularity\/openmpi\/. <br>In this example mpiexec command is executed on singularity container. Inside container is installed the same version of openmpi. Image was built on local linux machine using recipe file openmpi-test.recipe with root privileges: sudo singularity build openmpi.simg openmpi-test.recipe<\/p>\n\n\n\n<p>During the process of creating the image openmpi with infiniband support is built from source code and simple test is compiled. You can run the example with command sbatch test.sh.<\/p>\n\n\n\n<p>The interactive shell can be invoked by the singularity shell command. This is useful for development purposes. Use the -w | --writable option to make changes inside the container permanent.<\/p>\n\n\n\n<p>A user home directory is mounted inside the container automatically. If you need access to the \/scratch local disk storage for your computation, this must be mounted by the -B | --bind option. <strong>\/scratch<\/strong> na lok\u00e1lnom disku, toto sprav\u00edte pomocou prep\u00edna\u010da -B | &#8211;bind.<\/p>\n\n\n\n<p>A complete documentation can be found at the Singularity webpage. <a href=\"https:\/\/sylabs.io\/docs\/\">Singularity<\/a>.<\/p>\n<\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Singularity Singularity je kontajnerov\u00e1 platforma. Kontajner je jeden s\u00fabor, ktor\u00fd obsahuje nastavenie prostredia a v\u0161etok potrebn\u00fd softv\u00e9r pre spustenie va\u0161ej aplik\u00e1cie. Singularity je opensource projekt, ktor\u00fd bol vytvoren\u00fd pre sp\u00fa\u0161\u0165anie komplexn\u00fdch aplik\u00e1ci\u00ed v HPC prostred\u00ed. Umo\u017e\u0148uje pou\u017e\u00edvate\u013eovi ma\u0165 pln\u00fa kontrolu nad prostred\u00edm a vytv\u00e1ra\u0165 a sp\u00fa\u0161\u0165a\u0165 kontajnery, ktor\u00e9 obsahuj\u00fa v\u0161etok softv\u00e9r v prenosnom a reprodukovate\u013enom [&hellip;]<\/p>","protected":false},"author":2,"featured_media":0,"parent":1524,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"templates\/template-full-width.php","meta":[],"_links":{"self":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/504"}],"collection":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/comments?post=504"}],"version-history":[{"count":17,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/504\/revisions"}],"predecessor-version":[{"id":1932,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/504\/revisions\/1932"}],"up":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/1524"}],"wp:attachment":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/media?parent=504"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}