This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
institute_lorentz:institutelorentz_maris_slurm [2018/03/29 10:18] – [Maris QOS] lenocil | institute_lorentz:institutelorentz_maris_slurm [2018/10/08 07:49] – [Maris partitions] lenocil | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== | + | ====== |
All maris nodes have been configured to use [[http:// | All maris nodes have been configured to use [[http:// | ||
Line 19: | Line 19: | ||
Account information for a given < | Account information for a given < | ||
- | < | + | < |
sacctmgr list associations cluster=maris user=< | sacctmgr list associations cluster=maris user=< | ||
</ | </ | ||
Line 42: | Line 42: | ||
^ Name ^ CPUs ^ Memory ^ Deafult Job Memory, | ^ Name ^ CPUs ^ Memory ^ Deafult Job Memory, | ||
- | |playground|288|851832M|all| |maris0[04-22, | + | |playground|288|851832M|all| |maris0[04-22, |
- | |notebook| 48|193044M|all| |maris0[23-28]|6|4| 1| inf | notebook | all | | + | |notebook| 48|193044M|all| |maris0[23-28]|6|4| 1| | notebook | all | |
- | |computation| 1552|6578050M |400M, 3Days | inf |maris0[47-74] | 28 | inf | inf | inf | normal | all | | + | |computation| 1552|6578050M |400M, 3Days | |maris0[47-74] | 28 | | | | normal | all | |
- | |compintel| 192 |1030000M|400M, | + | |compintel| 192 |1030000M|400M, |
- | |emergency| 384 |2773706M|all| |maris0[69-74] |6| inf | inf | inf | normal | NOBODY | | + | |ibintel| 96 |512000M|400M, |
- | |gpu| 56 |256000M|400M, | + | |emergency| 384 |2773706M|all| |maris0[69-74] |6| | | | normal | NOBODY | |
+ | |gpu| 56 |256000M|400M, | ||
The `playground' | The `playground' | ||
Line 72: | Line 73: | ||
< | < | ||
+ | #sacctmgr show qos format=Name, | ||
Name MaxCPUsPU MaxJobsPU GrpNodes | Name MaxCPUsPU MaxJobsPU GrpNodes | ||
---------- --------- --------- -------- -------- ----------- -------------------- | ---------- --------- --------- -------- -------- ----------- -------------------- | ||
normal | normal | ||
- | playground | + | playground |
notebook | notebook | ||
- | guests | + | guests |
Line 107: | Line 109: | ||
Maris075 is the only GPU node in the maris cluster. GPUs are configured as generic resources or GRES. In order to use a GPU in your calculations, | Maris075 is the only GPU node in the maris cluster. GPUs are configured as generic resources or GRES. In order to use a GPU in your calculations, | ||
- | :!: Please note that on maris GPUs are configured as __consumable__ | + | :!: Please note that on maris GPUs are configured as __not-consumable__ |
+ | To compile your cuda application on maris using slurm, note that in your submission script you might have to export the libdevice library path and include the path in which the cuda headers can be found, for instance | ||
+ | <code bash> | ||
+ | #!/bin/env bash | ||
+ | .... | ||
+ | NVVMIR_LIBRARY_DIR=/ | ||
+ | </ | ||
Line 122: | Line 130: | ||
In principle to run an MPI application you could just execute it using mpirun as shown in the session below | In principle to run an MPI application you could just execute it using mpirun as shown in the session below | ||
- | < | + | < |
novamaris$ cat slurm_script.sh | novamaris$ cat slurm_script.sh | ||
#!/bin/env bash | #!/bin/env bash | ||
Line 131: | Line 139: | ||
</ | </ | ||
However, __**it is highly advised you use slurm' | However, __**it is highly advised you use slurm' | ||
- | < | + | < |
novamaris$ cat slurm_script.sh | novamaris$ cat slurm_script.sh | ||
#!/bin/env bash | #!/bin/env bash | ||
Line 141: | Line 149: | ||
At the moment maris supports only OpenMPI with slurm so you are required to load a particular openmpi/ | At the moment maris supports only OpenMPI with slurm so you are required to load a particular openmpi/ | ||
- | < | + | < |
# load openMPI | # load openMPI | ||
module load openmpi-slurm/ | module load openmpi-slurm/ | ||
Line 170: | Line 178: | ||
An example of batch script is given below: | An example of batch script is given below: | ||
- | < | + | < |
#!/bin/env bash | #!/bin/env bash | ||
##comment out lines by adding at least two `#' at the beginning | ##comment out lines by adding at least two `#' at the beginning | ||
Line 196: | Line 204: | ||
Consider the batch script below | Consider the batch script below | ||
- | < | + | < |
$ cat slurmcp.sh | $ cat slurmcp.sh | ||
#!/bin/env bash | #!/bin/env bash | ||
Line 223: | Line 231: | ||
Slurm can be instructed to email any job state changes to a chosen email address. This is accomplished by using the '' | Slurm can be instructed to email any job state changes to a chosen email address. This is accomplished by using the '' | ||
- | < | + | < |
... | ... | ||
#SBATCH --mail-user=myemail@address.org | #SBATCH --mail-user=myemail@address.org |