{"id":504,"date":"2020-11-08T20:43:49","date_gmt":"2020-11-08T19:43:49","guid":{"rendered":"https:\/\/eurocc.nscc.sk\/?page_id=504"},"modified":"2021-02-01T14:44:47","modified_gmt":"2021-02-01T13:44:47","slug":"singularity","status":"publish","type":"page","link":"https:\/\/eurocc.nscc.sk\/en\/services\/manualy\/singularity\/","title":{"rendered":"Singularity"},"content":{"rendered":"\n<div class=\"is-layout-flow wp-block-group manuals\"><div class=\"wp-block-group__inner-container\">\n<h4>Singularity<hr style=\"border: 1px solid #b8870b; width: 100px;\"><\/h4>\n\n\n\n<p><a href=\"https:\/\/sylabs.io\/\">Singularity<\/a> je kontajnerov\u00e1 platforma. Kontajner je jeden s\u00fabor, ktor\u00fd obsahuje nastavenie prostredia a v\u0161etok potrebn\u00fd softv\u00e9r pre spustenie va\u0161ej aplik\u00e1cie.<\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-medium\"><a href=\"https:\/\/sylabs.io\/\"><img decoding=\"async\" loading=\"lazy\" width=\"297\" height=\"300\" src=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo-297x300.png\" alt=\"\" class=\"wp-image-527\" srcset=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo-297x300.png 297w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo-12x12.png 12w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/logo.png 495w\" sizes=\"(max-width: 297px) 100vw, 297px\" \/><\/a><\/figure><\/div>\n\n\n\n<ul><li>Singularity je opensource projekt, ktor\u00fd bol vytvoren\u00fd pre sp\u00fa\u0161\u0165anie komplexn\u00fdch aplik\u00e1ci\u00ed v HPC prostred\u00ed.<\/li><li>Umo\u017e\u0148uje pou\u017e\u00edvate\u013eovi ma\u0165 pln\u00fa kontrolu nad prostred\u00edm a vytv\u00e1ra\u0165 a sp\u00fa\u0161\u0165a\u0165 kontajnery, ktor\u00e9 obsahuj\u00fa v\u0161etok softv\u00e9r v prenosnom a reprodukovate\u013enom bal\u00edku.<\/li><li>M\u00f4\u017eete si vytvori\u0165 kontajner na svojom laptope a sp\u00fa\u0161\u0165a\u0165 ho na HPC klastroch.<\/li><li>Singularity v\u00e1m taktie\u017e dovol\u00ed vyu\u017e\u00edva\u0165 zdroje na akomko\u013evek stroji sa nach\u00e1dzate, ktor\u00fd zah\u0155\u0148a aj HPC sie\u0165, mana\u017e\u00e9r zdrojov, s\u00faborov\u00e9 syst\u00e9my, GPU a akceler\u00e1tory at\u010f.<\/li><\/ul>\n\n\n\n<p>Neprivilegovan\u00fd pou\u017e\u00edvate\u013e si takto m\u00f4\u017ee zmeni\u0165 opera\u010dn\u00fd syst\u00e9m na server za ten, ktor\u00fd m\u00e1 pod kontrolou. Tak\u017ee ke\u010f na serveri be\u017e\u00ed RHEL6, ale va\u0161a aplik\u00e1cia be\u017e\u00ed na Ubuntu\/RHEL7, m\u00f4\u017eete si vytvori\u0165 Ubuntu\/RHEL7 image, nain\u0161talova\u0165 si svoje aplik\u00e1cie v tomto image, nakop\u00edrova\u0165 ho na \u010fal\u0161ie server a sp\u00fa\u0161\u0165a\u0165 tieto aplik\u00e1cie v ich nat\u00edvnom Ubuntu\/RHEL7 prostred\u00ed.<\/p>\n\n\n\n<p>Singularity kontajnery m\u00f4\u017eu ma\u0165 tri form\u00e1ty:<\/p>\n\n\n\n<ul><li>read-only <strong>squashfs<\/strong> (default) \u2013 vhodn\u00fd pre produk\u010dn\u00fd image;<\/li><li>writable <strong>ext3<\/strong> (&#8211;writable option);<\/li><li>writable <strong>(ch)root directory<\/strong> (&#8211;sandbox option) \u2013 vhodn\u00fd pre v\u00fdvoj.<br><\/li><\/ul>\n\n\n\n<p>Squashfs a (ch)root directory image m\u00f4\u017eu by\u0165 vytvoren\u00e9 z Docker source priamo na klastri, nie s\u00fa potrebn\u00e9 root pr\u00e1va. Odpor\u00fa\u010da sa vytvori\u0165 nat\u00edvny Singularity image pre r\u00fdchly beh kontajneru. Singularity je dostupn\u00e1 na na\u0161om klastri v Ko\u0161iciach.<\/p>\n\n\n\n<p class=\"has-large-font-size\"><strong>GPU pr\u00edklad<\/strong><\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-medium\"><a href=\"https:\/\/medium.com\/singularityapp\/singularity-containers-tensorflow-and-the-nvidia-jetson-nano-an-experiment-5be66ebbd4ac\"><img decoding=\"async\" loading=\"lazy\" width=\"300\" height=\"293\" src=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-300x293.png\" alt=\"\" class=\"wp-image-522\" srcset=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-300x293.png 300w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-1024x999.png 1024w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-768x749.png 768w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu-12x12.png 12w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_gpu.png 1050w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/figure><\/div>\n\n\n\n<p>Tensorflow be\u017ene pou\u017e\u00edvan\u00fd softv\u00e9r pre machine learning projekty. Ofici\u00e1lny tensorflow repozit\u00e1r na Docker Hub obsahuje kontajnery s podporou NVIDA GPU, ktor\u00e9 vyu\u017eij\u00fa CUDA na v\u00fdpo\u010dty.<\/p>\n\n\n\n<p>singularity pull docker:\/\/tensorflow\/tensorflow:latest-gpu<\/p>\n\n\n\n<p>Pr\u00edklad je umiestnen\u00fd v \/lustre\/home\/freeware\/EXAMPLE_JOBS\/gpu.<\/p>\n\n\n\n<p>Pr\u00edkazy, ktor\u00e9 obsahuj\u00fa run, alebo inak sp\u00fa\u0161\u0165aj\u00fa kontajner (shell, exec) m\u00f4\u017eu pou\u017ei\u0165 prep\u00edna\u010d &#8211;nv, ktor\u00fd nastav\u00ed v kontajneri prostredie pre vyu\u017eitie NVIDIA GPU a z\u00e1kladn\u00e9 kni\u017enice CUDA a umo\u017en\u00ed tak spustenie CUDA aplik\u00e1ci\u00ed.<\/p>\n\n\n\n<p>CUDA aplik\u00e1cie m\u00f4\u017eete be\u017ea\u0165 na n\u00f3doch s GPU: comp[17-24,39-56]. Sp\u00fa\u0161\u0165ac\u00ed skript je test.sh, ktor\u00fd spust\u00ed jednoduch\u00fd tensorflow pr\u00edklad. M\u00f4\u017eete ho spusti\u0165 pr\u00edkazom sbatch test.sh. <\/p>\n\n\n\n<p class=\"has-large-font-size\"><strong>OpenMPI pr\u00edklad<\/strong><\/p>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-medium\"><a href=\"https:\/\/kks32-courses.gitbook.io\/hpc-containers\/singularity\/building-a-container-configuration-file\/hands-on-3\"><img decoding=\"async\" loading=\"lazy\" width=\"300\" height=\"274\" src=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi-300x274.png\" alt=\"\" class=\"wp-image-525\" srcset=\"https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi-300x274.png 300w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi-13x12.png 13w, https:\/\/eurocc.nscc.sk\/wp-content\/uploads\/2020\/11\/singularity_mpi.png 600w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/figure><\/div>\n\n\n\n<p>Openmpi pr\u00edklad sa nach\u00e1dza v \/lustre\/home\/freeware\/EXAMPLE_JOBS\/singularity\/openmpi\/. <br>V tomto pr\u00edpade je pr\u00edkazom mpiexec spusten\u00fd singularity kontajner. Vn\u00fatri kontajneru je nain\u0161talovan\u00e1 rovnak\u00e1 verzia openmpi ako na server. Image bol vytvoren\u00fd na lok\u00e1lnom linuxovom po\u010d\u00edta\u010di&nbsp; pou\u017eit\u00edm \u201creceptu\u201d openmpi-test.recipe s pr\u00e1vami roota: sudo singularity build openmpi.simg openmpi-test.recipe<\/p>\n\n\n\n<p>Po\u010das vytvorenia kontajneru je skompilovan\u00e1 verzia openmpi s podporou infiniband a takisto je skompilovan\u00fd mali testovac\u00ed program. Tento pr\u00edklad m\u00f4\u017eete spusti\u0165 pr\u00edkazom sbatch test.sh.<\/p>\n\n\n\n<p>Interakt\u00edvny shell v kontajneri m\u00f4\u017eete spusti\u0165 pr\u00edkazom singularity shell. Je vhodn\u00fd pre v\u00fdvoj\u00e1rske \u00fa\u010dely. Pou\u017eite prep\u00edna\u010d -w | &#8211;writable aby sa zmeny v kontajneri ulo\u017eili natrvalo.<\/p>\n\n\n\n<p>Domovsk\u00fd adres\u00e1r pou\u017e\u00edvate\u013ea je v kontajneri mountovan\u00fd automaticky. Pokia\u013e chcete mountova\u0165 <strong>\/scratch<\/strong> na lok\u00e1lnom disku, toto sprav\u00edte pomocou prep\u00edna\u010da -B | &#8211;bind.<\/p>\n\n\n\n<p>\u00dapln\u00e1 dokument\u00e1cia sa nach\u00e1dza na str\u00e1nke <a href=\"https:\/\/sylabs.io\/docs\/\">Singularity<\/a>.<\/p>\n<\/div><\/div>\n","protected":false},"excerpt":{"rendered":"<p>Singularity Singularity je kontajnerov\u00e1 platforma. Kontajner je jeden s\u00fabor, ktor\u00fd obsahuje nastavenie prostredia a v\u0161etok potrebn\u00fd softv\u00e9r pre spustenie va\u0161ej aplik\u00e1cie. Singularity je opensource projekt, ktor\u00fd bol vytvoren\u00fd pre sp\u00fa\u0161\u0165anie komplexn\u00fdch aplik\u00e1ci\u00ed v HPC prostred\u00ed. Umo\u017e\u0148uje pou\u017e\u00edvate\u013eovi ma\u0165 pln\u00fa kontrolu nad prostred\u00edm a vytv\u00e1ra\u0165 a sp\u00fa\u0161\u0165a\u0165 kontajnery, ktor\u00e9 obsahuj\u00fa v\u0161etok softv\u00e9r v prenosnom a reprodukovate\u013enom [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":1524,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"templates\/template-full-width.php","meta":[],"_links":{"self":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/504"}],"collection":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/comments?post=504"}],"version-history":[{"count":17,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/504\/revisions"}],"predecessor-version":[{"id":1932,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/504\/revisions\/1932"}],"up":[{"embeddable":true,"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/pages\/1524"}],"wp:attachment":[{"href":"https:\/\/eurocc.nscc.sk\/en\/wp-json\/wp\/v2\/media?parent=504"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}