{"id":1041,"date":"2021-08-19T14:32:34","date_gmt":"2021-08-19T12:32:34","guid":{"rendered":"http:\/\/cdc.cyu.fr\/?page_id=1041"},"modified":"2021-08-31T21:55:35","modified_gmt":"2021-08-31T19:55:35","slug":"job-script-array","status":"publish","type":"page","link":"https:\/\/cdc.cyu.fr\/index.php\/documentations-main\/job-script-array\/","title":{"rendered":"Script de job array"},"content":{"rendered":"\n<p>Les <em>job array<\/em> permettent de lancer plusieurs instances d&rsquo;un m\u00eame job (s\u00e9quentiel ou parall\u00e8le) en une seule commande. Chaque instance sera identique aux autres, mis \u00e0 part le num\u00e9ro de task qui est accessible dans la pseudo variable d&rsquo;environnement SGE_TASK_ID. Il est ainsi possible de fournirs des param\u00e8tres diff\u00e9rents \u00e0 chaque task en utilisant le num\u00e9ro de task. Plusieurs solutions sont possibles, 2 sont propos\u00e9es ici.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Utiliser autant de fichier input que de tasks<\/h2>\n\n\n\n<p>Soit les fichiers <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">input1.dat<\/code>, <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">input2.dat<\/code>, &#8230; <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">inputN.dat<\/code>, contenant chacun les param\u00e8tres \u00e0 transmettre aux diff\u00e9rentes instances du programme \u00e0 ex\u00e9cuter.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">>> more input*.dat\n::::::::::::::\ninput1.dat\n::::::::::::::\n0.5  1.0  0.1\n\n::::::::::::::\ninput2.dat\n::::::::::::::\n0.6  1.0  0.1\n\n.\n.\n.\n\n::::::::::::::\ninputN.dat\n::::::::::::::\n0.6  1.0  0.3<\/pre>\n\n\n\n<p>Le script de jobs suivant r\u00e9serve 6 jobs s\u00e9quentiels dont les param\u00e8tres sont lus dans les fichier <code data-enlighter-language=\"apache\" class=\"EnlighterJSRAW\">inputX.dat<\/code> avec X correspondant au num\u00e9ro de \u00ab\u00a0task\u00a0\u00bb. Ce num\u00e9ro est contenu dans la variable <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">SGE_TASK_ID<\/code>.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">#$ -N seq-array\n## Job s\u00e9quentiel court\n#$ -q seq_short\n## TASK ARRAY : matrice de jobs (num\u00e9rot\u00e9s de  1 \u00e0 6)\n#$ -t 6\n## Quantit\u00e9 de m\u00e9moire r\u00e9serv\u00e9e\n#$ -l m_mem_free=500M\n## Messages\n#$ -M prenom.nom@cyu.fr\n#$ -m esa\n## Environnement\n#$ -cwd\n#$ -j y\nmodule load module1 module2 module3\n.\/programme &lt;&lt; input${SGE_TASK_ID}.dat<\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Utiliser un unique fichier input<\/h2>\n\n\n\n<p>Soit le fichier <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">inputs.dat<\/code> contenant N lignes de param\u00e8tres \u00e0 transmettre au programme \u00e0 ex\u00e9cuter.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"generic\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">0.5  1.0  0.1\n0.6  1.0  0.1\n0.5  1.0  0.2\n0.6  1.0  0.2\n0.5  1.0  0.3\n0.6  1.0  0.3\n...<\/pre>\n\n\n\n<p>Le script de jobs suivant r\u00e9serve 4 jobs parall\u00e8les openMP dont les param\u00e8tres sont lus \u00e0 la ligne du fichier <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">inputs.dat<\/code> correspondant au num\u00e9ro de \u00ab\u00a0task\u00a0\u00bb. Ce num\u00e9ro est contenu dans la variable <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">SGE_TASK_ID<\/code>. Pour illustrer les possibilit\u00e9s de l&rsquo;option <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">-t<\/code>, seules les lignes 2, 4, 6 et 8 du fichier <code data-enlighter-language=\"generic\" class=\"EnlighterJSRAW\">inputs.dat<\/code> seront utilis\u00e9es.<\/p>\n\n\n\n<pre class=\"EnlighterJSRAW\" data-enlighter-language=\"shell\" data-enlighter-theme=\"\" data-enlighter-highlight=\"\" data-enlighter-linenumbers=\"\" data-enlighter-lineoffset=\"\" data-enlighter-title=\"\" data-enlighter-group=\"\">#$ -N omp-array\n## Nombre de coeurs CPU r\u00e9serv\u00e9s ($NSLOTS)\n#$ -pe openmp 4\n## TASK ARRAY : matrice de jobs (num\u00e9rot\u00e9s de  2 \u00e0 8 par pas de 2)\n#$ -t 2-8:2\n## Quantit\u00e9 de m\u00e9moire r\u00e9serv\u00e9e par process openMP\n#$ -l m_mem_free=250M\n## Messages\n#$ -M prenom.nom@cyu.fr\n#$ -m esa\n## Environnement\n#$ -cwd\n#$ -j y\nmodule load module1 module2 module3\nexport OMP_NUM_THREADS=$NSLOTS\n## Lecture de la i-\u00e8me ligne des param\u00e8tres stock\u00e9s dans le fichier 'inputs.dat'\nparams=$(sed $SGE_TASK_ID'q;d' inputs.dat)\n.\/programme-openmp $params<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Les job array permettent de lancer plusieurs instances d&rsquo;un m\u00eame job (s\u00e9quentiel ou parall\u00e8le) en une seule commande. Chaque instance sera identique aux autres, mis \u00e0 part le num\u00e9ro de task qui est accessible dans la pseudo variable d&rsquo;environnement SGE_TASK_ID. &hellip; <a href=\"https:\/\/cdc.cyu.fr\/index.php\/documentations-main\/job-script-array\/\">Continuer la lecture <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":124,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1041","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/pages\/1041","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/comments?post=1041"}],"version-history":[{"count":7,"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/pages\/1041\/revisions"}],"predecessor-version":[{"id":1250,"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/pages\/1041\/revisions\/1250"}],"up":[{"embeddable":true,"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/pages\/124"}],"wp:attachment":[{"href":"https:\/\/cdc.cyu.fr\/index.php\/wp-json\/wp\/v2\/media?parent=1041"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}