User Tools

Site Tools


institute_lorentz:xmaris

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
institute_lorentz:xmaris [2020/07/13 07:32]
lenocil [Installing extra software]
institute_lorentz:xmaris [2020/09/25 15:02] (current)
lenocil [Xmaris]
Line 1: Line 1:
-====== ​xmaris ​======+====== ​Xmaris ​======
 xmaris is a small computational cluster at the [[https://​www.lorentz.leidenuniv.nl|Lorentz Institute]] xmaris is a small computational cluster at the [[https://​www.lorentz.leidenuniv.nl|Lorentz Institute]]
 financed by external research grants. As such, its access is granted **primarily** to the financed by external research grants. As such, its access is granted **primarily** to the
-{{ :​institute_lorentz:​xmaris.jpg?​nolink&​300|}} 
 research groups who have been awarded the grants. ​ Other research groups wishing to use xmaris can enquire whether there is any left-over computing time by getting in touch with either ​ research groups who have been awarded the grants. ​ Other research groups wishing to use xmaris can enquire whether there is any left-over computing time by getting in touch with either ​
  
-  * Xavier Bonet Monroig ​(Oort 260) +|Xavier Bonet MonroigOort 260| 
-  ​* ​Carlo Beenakker ​(Oort 261)+|Carlo Beenakker|Oort 261|
  
-to discuss what resources can be made available to their needs. After a preliminary assessment and approval, access to xmaris will be granted by the IT staff. Any technical questions should be addressed via https://​helpdesk.lorentz.leidenuniv.nl ​or in person ​to+to discuss what resources can be made available to their needs. After a preliminary assessment and approval, access to xmaris will be granted by the IT staff. Any technical questions should be addressed via https://​helpdesk.lorentz.leidenuniv.nl to
  
-  * Leonardo Lenoci ​-- System Administrator (HL409b)+Leonardo Lenoci ​HL409b|
  
 :!: External research groups to the Lorentz Institute are strongly encouraged to explore other HPC possibilities,​ such as the [[https://​wiki.alice.universiteitleiden.nl/​index.php?​title=ALICE_User_Documentation_Wiki|ALICE HPC cluster]] of the University of Leiden. :!: External research groups to the Lorentz Institute are strongly encouraged to explore other HPC possibilities,​ such as the [[https://​wiki.alice.universiteitleiden.nl/​index.php?​title=ALICE_User_Documentation_Wiki|ALICE HPC cluster]] of the University of Leiden.
  
-Xmaris is optimised for [[https://​en.wikipedia.org/​wiki/​Thread_(computing)#​Multithreading|multithreading applications]] and  [[https://​en.wikipedia.org/​wiki/​Embarrassingly_parallel|embarrassingly parallel problems]] but there have been some recent investments to improve nodes interconnection communications to enable [[institute_lorentz:​xmaris#​parallelism_101|multiprocessing]]. Currently, multiprocessing is possible on maris0[78-81] which are interconnected via an InfiniBand EDR switch. Each one of these nodes  is capable of a practical __9.6 TFLOPS*__.+Xmaris is optimised for [[https://​en.wikipedia.org/​wiki/​Thread_(computing)#​Multithreading|multithreading applications]] and  [[https://​en.wikipedia.org/​wiki/​Embarrassingly_parallel|embarrassingly parallel problems]]but there have been some recent investments to improve nodes interconnection communications to enable [[institute_lorentz:​xmaris#​parallelism_101|multiprocessing]]. Currently, multiprocessing is possible on maris0[78-81] which are interconnected via an InfiniBand EDR switch. Each one of these nodes  is capable of a practical __9.6 TFLOPS__.
  
-* Curious? Ask `//​support//​` how this number was estimated. 
  
 Xmaris is the successor of the maris cluster, renamed with a prefix ''​x''​ because its nodes deployment is automated using the [[https://​www.xcat.org/​|xCAT]] software. Less formally, the presence of the ''​x''​ prefix also suggests the time of the year when xmaris was first made available to IL users, that is Christmas (Xmas). Xmaris is the successor of the maris cluster, renamed with a prefix ''​x''​ because its nodes deployment is automated using the [[https://​www.xcat.org/​|xCAT]] software. Less formally, the presence of the ''​x''​ prefix also suggests the time of the year when xmaris was first made available to IL users, that is Christmas (Xmas).
-===== xmaris features and expected lifetime ===== 
  
-xmaris runs CentOS v7.6 and consists for historical reasons of heterogeneous computation nodesA list of configured nodes and partitions on the cluster can be obtained using slurm ''​sinfo''​.+[[https://​www.gnu.org/​|{{https://​www.gnu.org/​graphics/​heckert_gnu.transp.small.png?​50 }}]][[https://​wiki.centos.org/​|{{https://​wiki.centos.org/​ArtWork/​Brand/​Logo?​action=AttachFile&​do=get&​target=centos-logo-light.png?​200 }}]] [[https://​openondemand.org/​|{{https://​openondemand.org/​assets/​images/​ood_logo_stack_rgb.png?​200 ​ }}]] [[https://slurm.schedmd.com|{{https://​slurm.schedmd.com/​slurm_logo.png?​60 ​ }}]] [[https://​easybuild.readthedocs.io/​en/​latest/​|{{https://​easybuild.readthedocs.io/​en/​latest/​_static/​easybuild_logo_alpha.png?​200 ​ }}]] 
 +===== Xmaris features and expected cluster lifetime =====
  
-:!: Because xmaris features different CPU types that understand different types of instructions (see [[https://​gcc.gnu.org/​onlinedocs/​gcc/​x86-Options.html|here]]),​ each slurm node is characterised by a list of ''​Features'' ​that, among other things, describe the type of CPUs mounted in that node.+Xmaris runs CentOS v7.6 and consists for historical reasons ​of heterogeneous computation nodeslist of configured nodes and partitions on the cluster can be obtained on the command line using slurm'''sinfo''​.
  
-In the example below you can see that  ​maris077'​s CPUs belong ​to the ''​broadwell'' ​family. To request allocation of specific features, ​please ​see [[institute_lorentz:​xmaris#​request_nodes_with_particular_features|below]].+:!: Because Xmaris features different CPU types that understand different types of instructions (see [[https://​gcc.gnu.org/​onlinedocs/​gcc/​x86-Options.html|here]]),​ we have associated ​to each computation node a list of slurm ''​Features'' ​that also describe the type of CPUs mounted in that node. To request allocation of specific features ​to the resource manager, see [[institute_lorentz:​xmaris#​request_nodes_with_particular_features|this example]]. 
 + 
 +You can display nodes features with ''​sinfo''​ as below
  
 <​code>​ <​code>​
Line 33: Line 33:
 </​code>​ </​code>​
  
-xmaris ​aims to offer a stable computational environment to its users in the period **Dec 2019 -- Jan 2024**. Within this period, the OS might be patched only with important security updates. Past January 2024 all working xmaris nodes will be re-provisioned from scratch with newer version ​of CentOS. At this time all scratch ​data disks will be reformatted.+Xmaris ​aims to offer a //stable// computational environment to its users in the period **Dec 2019 -- Jan 2024**. Within this period, the OS might be patched only with important security updates. Past January 2024all working xmaris nodes will be re-provisioned from scratch with  newer versions ​of the operating system and software infrastructure. At this time all data stored in the [[institute_lorentz:​xmaris#​compute_nodes_data_disks|temporary scratch ​disks]] will be destroyed and the disks reformatted.
  
 ==== Compute nodes data disks ==== ==== Compute nodes data disks ====
Line 39: Line 39:
 All compute nodes have **//at least//** access to the following data partitions ​ All compute nodes have **//at least//** access to the following data partitions ​
  
-  * /scratch (temporary, local to each node, therefore faster I/O)+  * /scratch ( **temporary**, local to each node, therefore faster I/O)
   * /marisdata (NetApp, 2TB/user quota)   * /marisdata (NetApp, 2TB/user quota)
   * /home (NetApp, 10GB/user quota)   * /home (NetApp, 10GB/user quota)
  
-An extra scratch ​space is available to all nodes which features ​infiniband ​cards+Extra efficient ​scratch ​spaces are available to all nodes on the infiniband ​network
  
-  * /IBSSD (maris0[78,​79,​80,​81] -- temporary, InfiniBand/​iSER high-rate I/O)   +  * /IBSSD (maris0[78,​79,​80,​81] -- **temporary**, InfiniBand/​iSERhigh-rate I/O)   
-  * /PIBSSD (maris0[78,​79,​80,​81] -- temporary, InfiniBand/Lustre ​Parallel FS), test from end 2020 +  * /PIBSSD (maris0[78,​79,​80,​81] -- **temporary**, InfiniBand/BeeGFS ​Parallel FS -- testing until mid 2021
  
-:!: iSER stands for “iSCSI Extensions for RDMA”. It is an extension of the iSCSI protocol that includes RDMA (Remote Dynamic Memory Access) support.+iSER stands for “iSCSI Extensions for RDMA”. It is an extension of the iSCSI protocol that includes RDMA (Remote Dynamic Memory Access) support.
  
  
Line 57: Line 57:
 The home disk ''/​home''​ has a 10GB/user quota where as ''/​marisdata''​ has a 2TB/user quota. Note that **these policies might change at any time at the discretion of the cluster owners**. The home disk ''/​home''​ has a 10GB/user quota where as ''/​marisdata''​ has a 2TB/user quota. Note that **these policies might change at any time at the discretion of the cluster owners**.
  
-:!: The **OLD** ''/​clusterdata''​ is deliberately made unavailable on xmaris, because it is **no longer** maintained. ​ If you have any data on it, **it is your responsibility** to create backups. All data on ''/​clusterdata''​ will get permanently lost in case of hardware failure. ​ 
  
 +----
  
-**All data** on the scratch partitions is assumed to be temporary and will be **deleted** upon a node re-installation.+  
 +:!: **All data** on the scratch partitions is assumed to be temporary and will be **deleted** upon a node re-installation.
  
-:!: maris' homes are different than IL workstations' homes.+:!: When usisng xmaris please note that the home partition ​ is different than the IL workstations ​home.
  
-==== xmaris ​usage policies ====+:!: The **OLD** (as in the old maris) ''/​clusterdata''​ is deliberately made unavailable on xmaris, because it is **no longer** maintained. ​ If you have any data on it, **it is your responsibility** to create backups. All data on ''/​clusterdata''​ will get permanently lost in case of hardware failure. 
 +==== Xmaris ​usage policies ====
  
-Usage policies are updated regularly in accordance with the needs of the cluster owners and **may change at any time without notice**. At the moment there is an enforced ​usage limit of 128 CPUs per user that does not apply to the owners. Job execution priorities are defined via a complex [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​priority_multifactor.html|multi-factor algorithm]] whose parameters can be queried ​via+Usage policies are updated regularly in accordance with the needs of the cluster owners and **may change at any time without notice**. At the moment there is an __enforced ​usage limit of 128 CPUs per user__ ​that does not apply to the owners. Job execution priorities are defined via a complex [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​priority_multifactor.html|multi-factor algorithm]] whose parameters can be displayed on the command line via
 <​code>​ <​code>​
 scontrol show config | grep -i priority scontrol show config | grep -i priority
 </​code>​ </​code>​
-Xmaris usage is regularly monitored to prevent resource abuse. 
-===== xmaris live state ===== 
  
-To monitor live usage of xmaris ​you can either+===== Xmaris live state =====  
 +To monitor live usage of Xmaris ​you can either
  
-  * ssh into xmaris (see below) and use ''​sinfo''​ +  * execute slurm'​s ​''​sinfo'' ​(this requires shell access to the cluster, see below) 
-  * use your web browser to visit https://marishub.lorentz.leidenuniv.nl:​4433/​pun/​sys/​ganglia-status+  * visit https://xmaris.lorentz.leidenuniv.nl:​4433/​pun/​sys/​ganglia-status ​(via a web browser)
  
 :!: The link above is accessible only within the IL workstations network. :!: The link above is accessible only within the IL workstations network.
-===== How to access ​xmaris ​===== +===== How to access ​Xmaris ​===== 
-:!: Only from within ​the IL workstations subnet+Access to Xmaris is not granted automatically to all Lorentz Institute members. Instead, a preliminary approval must be granted to you by the cluster owners (read [[|here]]).
  
-Once you have been authorised to use xmarisyou have two ways to access its services:+Once you have been authorised to use Xmaristhere are two ways to access its services:
  
-  - terminal access+  - using an ssh client (traditional method)
-  - web access via OpenOnDemand+  - using a web browser (modern method)
  
 +Both methods can provide terminal access, but connections via web browsers offer you extra services such as sftp (drag-and-drop file transfers) and jupyter interactive notebooks. We advise all users unfamilair with  GNU/Linux to use a web browser to interact with Xmaris.
  
-==== Terminal access ​====+==== Access via an ssh client ​====
  
 +The procedure differs on whether you try to connect with a client connected to the IL network or not.
  
-Terminal access is provided via login to xmaris'​ headnode reachable at ''​marishead.lorentz.leidenuniv.nl''​. For connections from outside ​the IL network, ​an [[institute_lorentz:​institutelorentz_remoteaccess|ssh tunnel]] into the IL ssh server is needed. ​+  * When within ​the IL network, ​for instance if you are using a Lorentz Institute workstation,​ you have direct access to Xmaris. Open a terminal and type the command below
  
 +<​code>​
 +ssh xmaris.lorentz.leidenuniv.nl -l <​your-IL-username>​
 +</​code>​
 + 
 +  * When outside the IL network, for instance from home or using a wireless connection, you must first initiate an ssh tunnel to our SSH server and then connect to Xmaris. You will need to open two terminal windows
  
-{{ :institute_lorentz:terminal.png?​nolink&​600 |}}+<​code>​ 
 +# First set up the tunnel in terminal window 1 
 +ssh -f <​your-IL-username>​@ssh.lorentz.leidenuniv.nl -L 2222:xmaris.lorentz.leidenuniv.nl:22 -N 
 +</​code>​ 
 + 
 +then 
 +<​code>​ 
 +# Connect to Xmaris in terminal ​window 2 
 +ssh -p 2222 localhost -l  <​your-IL-username>​ 
 +</​code>​
  
-:!: If you were a maris user prior to the configuration switch to xmaris, you will find out that many terminal functions and programs ​are not working as expected. This is due to the presence in your maris home directory of old shell initialisation scripts still tied to the STRW sfinx environment. You can override them (after making a backup copy) by replacing their contents with the default ​ CentOS shell initialisation scripts, for instance for bash these are located in ''/​etc/​skel/​.bashrc''​ and ''/​etc/​skel/​.bash_profile''​.+:!: If you were a maris user prior to the configuration switch to xmaris, you might find out that many terminal functions and programs ​could not be working as expected. This is due to the presence in your xmaris ​home directory of old shell initialisation scripts still tied to the STRW sfinx environment. You can override them (preferably ​after making a backup copy) by replacing their contents with the default CentOS shell initialisation scripts, for instance for bash these are located in ''/​etc/​skel/​.bashrc''​ and ''/​etc/​skel/​.bash_profile''​.
 ==== Web access ==== ==== Web access ====
  
-xmaris ​services, that is terminal, scheduler/​resource manager, jupyter notebooks and monitoring facilities, can be accessed easily via a browser without the need of additional plugins navigating to [[https://marishead.lorentz.leidenuniv.nl:​4433|xmaris OpenOnDemand]] +Xmaris ​services, that is terminal, scheduler/​resource manager, jupyter notebooks and monitoring facilities, can be accessed easily via a browser without the need of additional plugins navigating to [[https://xmaris.lorentz.leidenuniv.nl:​4433|xmaris OpenOnDemand]]. 
-{{ :​institute_lorentz:​ondemand.png?​direct&​800 }} +{{ :​institute_lorentz:​ondemand.png?​direct&​500 }}  
-Similarly to a standard terminal ​access, ​xmaris ​OpenOnDemand is available only for connections within the IL subnetwork. IL users  who wish to access OpenOnDemand from their home could instruct their browser ​to connect via a SOCKS proxy, for instance open a local terminal and type +Similarly to a traditional shell access, ​Xmaris ​OpenOnDemand is available only for connections within the IL subnetwork. IL users  who wish to access OpenOnDemand from their homes could instruct their browsers ​to SOCKS-proxy their connections via our SSH server. 
 +Open a local terminal and type 
  
 <​code>​ <​code>​
Line 107: Line 125:
 </​code>  ​ </​code>  ​
  
-then in your browser settings find the tab relative to the connection type and instruct the browser to use the SOCKS proxy at ''​localhost:​7777''​ to connect to the internet.+then in your browser settings find the tab relative to the connection type and instruct the browser to use the SOCKS proxy located ​at ''​localhost:​7777''​ to connect to the internet.
  
-xmaris ​OnDemand allows you to +Xmaris ​OnDemand allows you to 
  
   * Create/edit files and directories.   * Create/edit files and directories.
Line 119: Line 137:
  
  
-===== xmaris ​GPUs =====+===== Xmaris ​GPUs =====
  
-Currently only maris075 ​features GPUs. They are two ''​nvidia ​Tesla P100 16GB''​ (cuda-compute-capability = 6.0) . In order to request them you must use the ''​--gres''​ option, for instance ''​srun -p gpuIntel --gres=gpu:​1 --pty bash -i''​. +^Node^Partition^GPUs^CUDA compatibility^ 
-===== xmaris scientific software =====+|maris075 ​|gpuIntel|2 x Nvidia ​Tesla P100 16GB 6.0|
  
-xmaris ​uses [[https://​easybuild.readthedocs.io/​en/​latest/​|EasyBuild]] to provide a build environment for its (scientific) software. Pre-installed software can be explored by means of the ''​module spider''​ command. For instance you can query the system for all modules whose name starts with  `mpi' by executing ''​module -r spider '​^mpi'''​. Installed softwares include+Use the ''​--gres''​ slurm option to allocate them for your job,  for instance via ''​srun -p gpuIntel --gres=gpu:​1 --pty bash -i''​. 
 +===== Xmaris scientific software ===== 
 + 
 +Xmaris ​uses [[https://​easybuild.readthedocs.io/​en/​latest/​|EasyBuild]] to provide a build environment for its (scientific) software. Pre-installed software can be explored by means of the ''​module spider''​ command. For instanceyou can query the system for all modules whose name starts with  `mpi' by executing ''​module -r spider '​^mpi'''​. Installed softwares include
  
  
Line 154: Line 175:
 * Usage of proprietary software is discouraged. * Usage of proprietary software is discouraged.
  
 +For an up-to-date list of installed software use the ''​module avail''​ command.
 +Any pre-installed software can be made available in your environment via the ''​module load <​module_name>''​ command.
  
-For an up-to-date list of installed software use the ''​module''​ command. +It is possible ​to save a list of modules you use often in a //​module ​collection// to load them in one command
-Any pre-installed software can be sourced by means of the ''​module load''​ command. +
- +
-Sometimes it is useful ​to save a list of modules you use often in a `collection'. Consider the following example+
 <​code>​ <​code>​
 module load mod1 mod2 mod3 mod4 mod5 mod6 module load mod1 mod2 mod3 mod4 mod5 mod6
Line 168: Line 188:
 ==== Installing extra software ==== ==== Installing extra software ====
  
-  ​- Request ​it via https://​helpdesk.lorentz.leidenuniv.nl+If you need to run a software that is not present on Xmaris, you might: 
 + 
 +  ​- Request ​its installation ​via https://​helpdesk.lorentz.leidenuniv.nl
   - Install it yourself   - Install it yourself
-    * via EasyBuild (see instructions below on how to setup your EasyBuild environment) +    * via EasyBuild (see instructions below on how to setup your personal ​EasyBuild environment) 
-    * using your own method ​ +    * via a traditional //​configure/​make//​ procedure 
 + 
 +Whatever installation ​method ​you might choose, please note that you do not have administrative rights to the cluster. ​
  
-:!: You do not have administrative rights to the cluster. 
  
 === Installing software via EasyBuild === === Installing software via EasyBuild ===
  
-Load the EasyBuild module ​and define ​a directory in which to store your EasyBuild-built softwares+In order to use EasyBuild ​to build a software, you must first set up your development environment. This is usually done by 
 + 
 +  * Loading the EasyBUild ​module 
 +  * Indicating ​a directory in which to store your EasyBuild-built softwares 
 +  * Specifying EasyBuild'​s behaviour via EASYBUILD_* environment variables 
 +  * Build a software 
 + 
 +In their simplest form, the steps outlined above can be translated into the following shell commands
  
 <​code>​ <​code>​
-module avail EasyBuild+
 module load EasyBuild module load EasyBuild
 mkdir /​marisdata/<​uname>/​easybuild mkdir /​marisdata/<​uname>/​easybuild
 export EASYBUILD_PREFIX=/​marisdata/<​uname>/​easybuild export EASYBUILD_PREFIX=/​marisdata/<​uname>/​easybuild
 export EASYBUILD_OPTARCH=GENERIC export EASYBUILD_OPTARCH=GENERIC
 +
 +eb -S ^Miniconda
 +eb Miniconda2-4.3.21.eb -r
 +
 </​code>​ </​code>​
  
-:!: NOTE 1: The environment variable ''​EASYBUILD_OPTARCH''​ instructs EasyBuild to compile software in a generic way so that it can be used on different CPUs. This is rather convenient in heterogeneous clusters such as xmaris to avoid recompilations of the same softwares on different compute nodes. This convenience comes of course at a cost; the executables so produced will not be as efficient as they would be on a given CPU. For more info read [[https://​easybuild.readthedocs.io/​en/​latest/​Controlling_compiler_optimization_flags.ht+:!: The environment variable ''​EASYBUILD_OPTARCH''​ instructs EasyBuild to compile software in a generic way so that it can be used on different CPUs. This is rather convenient in heterogeneous clusters such as xmaris to avoid recompilations of the same softwares on different compute nodes. This convenience comes of course at a cost; the executables so produced will not be as efficient as they would be on a given CPU. For more info read [[https://​easybuild.readthedocs.io/​en/​latest/​Controlling_compiler_optimization_flags.ht
 ml|here]]. ml|here]].
  
-:!: NOTE 2: When compiling OpenBLAS it is not sufficient to define ''​EASYBUILD_OPTARCH''​ to ''​GENERIC''​ to achieve portability of the executables. Some extra steps must be taken as described in https://​github.com/​easybuilders/​easybuild/​blob/​master/​docs/​Controlling_compiler_optimization_flags.rst. A list of targets supported by OpenBLAS can be found [[https://​github.com/​xianyi/​OpenBLAS/​blob/​develop/​TargetList.txt|here]].+:!: When compiling OpenBLAS it is not sufficient to define ''​EASYBUILD_OPTARCH''​ to ''​GENERIC''​ to achieve portability of the executables. Some extra steps must be taken as described in https://​github.com/​easybuilders/​easybuild/​blob/​master/​docs/​Controlling_compiler_optimization_flags.rst. A list of targets supported by OpenBLAS can be found [[https://​github.com/​xianyi/​OpenBLAS/​blob/​develop/​TargetList.txt|here]].
  
-Search a software to build, build it and make it available to your environment+Then execute ​
  
 <​code>​ <​code>​
-eb -S ^Miniconda 
-eb Miniconda2-4.3.21.eb -r 
 module use /​marisdata/<​uname>/​easybuild/​modules/​all module use /​marisdata/<​uname>/​easybuild/​modules/​all
 </​code>​ </​code>​
 +
 +to make available to the ''​module''​ comamnd any of the softwares built in your EasyBuild userspace.
  
 :!: ''​module use <​path>''​ will prepend <​path>​ to your  ''​MODULEPATH''​. Should you want to append it instead, then add the option ''​-a''​. To remove <​path>​ from ''​MODULEPATH''​ execute ''​module unuse <​path>''​. :!: ''​module use <​path>''​ will prepend <​path>​ to your  ''​MODULEPATH''​. Should you want to append it instead, then add the option ''​-a''​. To remove <​path>​ from ''​MODULEPATH''​ execute ''​module unuse <​path>''​.
  
 Should you want to customise the building process of a given software please read how to implement [[https://​easybuild.readthedocs.io/​en/​latest/​Implementing-easyblocks.html|EasyBlocks]] and write [[https://​easybuild.readthedocs.io/​en/​latest/​Writing_easyconfig_files.html|EasyConfig]] files  or  Should you want to customise the building process of a given software please read how to implement [[https://​easybuild.readthedocs.io/​en/​latest/​Implementing-easyblocks.html|EasyBlocks]] and write [[https://​easybuild.readthedocs.io/​en/​latest/​Writing_easyconfig_files.html|EasyConfig]] files  or 
-contact Leonardo Lenoci (HL409b) ​for a quick tutorial.+contact Leonardo Lenoci (HL409b).
  
 == Working with conda modules == == Working with conda modules ==
  
-Several conda modules are ready-to-use on maris. ​ A possible use of these could be to clone them and extend them with your packages of choice. Mind though that if you run ''​conda init'',​ conda will modify your shell initialisation scripts (e.g. ''​~/​.bashrc''​) to load automatically the chosen conda environment.+Several conda modules are ready-to-use on maris. ​ A possible use of these could be to clone  and extend them with your packages of choice. Mind though that if you run ''​conda init'',​ conda will modify your shell initialisation scripts (e.g. ''​~/​.bashrc''​) to load automatically the chosen conda environment.
 This causes several problems in all cases in which you are supposed to work in a clean environment. This causes several problems in all cases in which you are supposed to work in a clean environment.
  
Line 242: Line 276:
  
 </​code>​ </​code>​
-===== How to run a computation =====+===== How to run a computation ​on Xmaris ​=====
  
-xmaris ​runs the slurm scheduler and resource manager. Computation jobs can be submitted as batch jobs or be run interactively via slurm. Any other jobs will be terminated without prior notice.+Xmaris ​runs the slurm scheduler and resource manager. Computation jobs must be submitted as batch jobs or be run interactively via slurm. Any other jobs will be terminated ​**without prior notice**. Because this is not a slurm manual, you are encouraged to learn the basics by reading [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​|the slurm manual]]. Here we only give you a few simple examples.
  
 ==== Batch jobs ==== ==== Batch jobs ====
-  + 
-To submit a batch job to slurm you must first create a shell script which contains enough instructions to request the needed resources ​to slurm and to execute your program. The script can be written in any known interpreter to the system. ​In a batch script, slurm instructions are prefixed by the interpreter comment symbol and the word ''​SBATCH''​. +Batch jobs are computation jobs that do not execute interactively.  
-For instance a bash batch script ​could be+ 
 +To submit a batch job to Xmaris' ​slurm you must first create a shell script which contains enough instructions to request the needed resources and to execute your program. The script can be written in **any** known interpreter to the system. ​Slurm instructions are prefixed by the chosen ​interpreter comment symbol and the word ''​SBATCH''​. 
 +An example ​bash-batch script ​that will request Xmaris to execute the program ''​hostname''​ on one node is 
 <​code>​ <​code>​
 cat test.sh cat test.sh
Line 259: Line 296:
 </​code>​ </​code>​
  
-Please consult the [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​|slurm manual]] for all possible options. ​Batch scripts are then submitted for execution via ''​sbatch''​+Batch scripts are then submitted for execution via ''​sbatch''​ 
 <​code>​ <​code>​
 sbatch test.sh sbatch test.sh
 sbatch: Submitted batch job 738279474774293 sbatch: Submitted batch job 738279474774293
 </​code>​ </​code>​
 +
 and their status [PENDING|RUNNING|FAILED|COMPLETED] checked using ''​squeue''​. You can recur to the command ''​sstat''​ to display useful information about your running job, such as memory consumption etc... and their status [PENDING|RUNNING|FAILED|COMPLETED] checked using ''​squeue''​. You can recur to the command ''​sstat''​ to display useful information about your running job, such as memory consumption etc...
  
 ssh __shell access to an executing node is__ automatically __granted__ by slurm and can also be used for debugging purposes. ssh __shell access to an executing node is__ automatically __granted__ by slurm and can also be used for debugging purposes.
 +
 +Please consult the [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​|slurm manual]] for all possible ''​sbatch''​ options. ​
  
 ==== Interactive jobs ==== ==== Interactive jobs ====
  
-Interactive jobs give you nodes shell prompts ​in interactive ​mode+Interactive jobs are usually used for debugging purposes and in those cases 
 +in which the computation requires human interaction. Using interactive ​sessions you can gain shell access to a computation node 
  
 <​code>​ <​code>​
Line 276: Line 319:
 </​code>​ </​code>​
  
-===== Parallelism 101 =====+or execute an interactive program
  
-:!: Inexpert users should refrain from attempting to program parallel applications without studying appropriately.+<​code>​ 
 +srun  -p compIntel -N1 -n 1 --mem=4000 --pty python -c "​import sys; data = sys.stdin.readlines();​ print(data)"​ -i 
 +Hello world 
 +^D 
 +['​Hello world\n'​] 
 +</​code>​ 
 +===== Parallelism 101 =====
  
 A parallel job  runs a calculation whose computational subtasks are run simultaneously. The underlying principle is that  A parallel job  runs a calculation whose computational subtasks are run simultaneously. The underlying principle is that 
Line 307: Line 356:
 ===== How to launch a jupyter notebook ===== ===== How to launch a jupyter notebook =====
  
-To launch a jupyter notebook login to [[https://marishead.lorentz.leidenuniv.nl:​4433|xmaris OnDemand]], select ''​Interactive Apps --> Jupyter Notebook''​ and specify the resources needed in the form provided, push ''​Launch''​ and wait until the notebook has launched.+To launch a jupyter notebook login to [[https://xmaris.lorentz.leidenuniv.nl:​4433|xmaris OnDemand]], select ''​Interactive Apps --> Jupyter Notebook''​ and specify the resources needed in the form provided, push ''​Launch''​ and wait until the notebook has launched.
 Now you can interact with your notebook (click on ''​Connect to Jupyter''​),​ open a shell on the executing node (click on ''​Host >​_hostaname''​),​ and analyse notebook log files for debugging purposes (click on ''​Session ID xxxx-xxxx-xxxxx-xxxxxxx-xxx-xx''​). Now you can interact with your notebook (click on ''​Connect to Jupyter''​),​ open a shell on the executing node (click on ''​Host >​_hostaname''​),​ and analyse notebook log files for debugging purposes (click on ''​Session ID xxxx-xxxx-xxxxx-xxxxxxx-xxx-xx''​).
  
Line 331: Line 380:
  
 <​code>​ <​code>​
-https://marishub.lorentz.leidenuniv.nl:​4433/​node/​maris051.lorentz.leidenuniv.nl/​26051/​lab?​+https://xmaris.lorentz.leidenuniv.nl:​4433/​node/​maris051.lorentz.leidenuniv.nl/​26051/​lab?​
 </​code>​ </​code>​
  
Line 395: Line 444:
 xmaris runs the scheduler and resource manager slurm **v18.08.6-2**. Please consult the [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​|official manual]] for detailed information. ​ xmaris runs the scheduler and resource manager slurm **v18.08.6-2**. Please consult the [[https://​slurm.schedmd.com/​archive/​slurm-18.08.6/​|official manual]] for detailed information. ​
  
-:!: The headnode (marishead) is not a compute node. Any user applications running on it will be terminated without notice.+:!: The headnode (xmaris.lorentz.leidenuniv.nl) is not a compute node. Any user applications running on it will be terminated without notice.
  
 Here we report a few useful commands and their outputs to get you started. For the inpatients, look at the following [[https://​www.lorentz.leidenuniv.nl/​RUL-only/​generator/​|slurm batch-script generator]] (__NO RESPONSIBILITIES assumed__! Study the script before submitting it.) which is available only from withing UL IPs. Here we report a few useful commands and their outputs to get you started. For the inpatients, look at the following [[https://​www.lorentz.leidenuniv.nl/​RUL-only/​generator/​|slurm batch-script generator]] (__NO RESPONSIBILITIES assumed__! Study the script before submitting it.) which is available only from withing UL IPs.
institute_lorentz/xmaris.1594625540.txt.gz · Last modified: 2020/07/13 07:32 by lenocil