User Tools

Site Tools


institute_lorentz:xmaris

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
institute_lorentz:xmaris [2022/06/10 09:40]
lenocil [Xmaris scientific software]
institute_lorentz:xmaris [2022/06/10 14:03] (current)
lenocil [Suggested readings]
Line 209: Line 209:
 |TensorFlow-1.15.0-Miniconda3/4.7.10| CPU| All| | |TensorFlow-1.15.0-Miniconda3/4.7.10| CPU| All| |
  
-To create and use a tensorflow-aware jupyter kernel that is compatible with xmaris' OpenOnDemand interface do+The following example shows how you can create a tensorflow-aware jupyter notebook kernel that you can use for instance via the OpenOnDemand interface 
  
 <code bash> <code bash>
-only on maris075 (GPU node)+We use maris075 (GPU node) and load the optimised tf module
 ml load TensorFlow/2.1.0-fosscuda-2019b-Python-3.7.4 ml load TensorFlow/2.1.0-fosscuda-2019b-Python-3.7.4
-pip install --user ipykernel==5.1.2 + 
-pip install --user jupyter-client==5.3.1 +# We install ipykernel, because necessary to run py notebooks 
-ipython kernel install --name=tf210gpuquantum --user+python -m pip install ipykernel --user 
 + 
 +# We create a kernel called TFQuantum based on python from TensorFlow/2.1.0-fosscuda-2019b-Python-3.7.4 
 +python -m ipykernel install --name TFQuantum --display-name "TFQuantum" --user 
 + 
 +# We edit the kernel such that it does not execute python directly 
 +# but via a custom wrapper script 
 +cat $HOME/.local/share/jupyter/kernels/tfquantum/kernel.json 
 + 
 +
 + "argv":
 +  "/home/lenocil/.local/share/jupyter/kernels/tfquantum/wrapper.sh", 
 +  "-m", 
 +  "ipykernel_launcher", 
 +  "-f", 
 +  "{connection_file}" 
 + ], 
 + "display_name": "TFQuantum", 
 + "language": "python", 
 + "metadata":
 +  "debugger": true 
 + } 
 +
 + 
 +# The wrapper script will call python but only after loading any 
 +# appropriate module 
 +cat /home/lenocil/.local/share/jupyter/kernels/tfquantum/wrapper.sh 
 + 
 +#!/bin/env bash 
 +ml load TensorFlow/2.1.0-fosscuda-2019b-Python-3.7.4 
 + 
 +exec python $@ 
 + 
 +# DONE. tfquantum will appear in the dropdown list of kernels 
 +# upon creating a new notebook 
 </code> </code>
  
-When launching a jupyter notebook remember to specify ''TensorFlow/2.1.0-fosscuda-2019b-Python-3.7.4'' as an extra runtime module.+
  
 === TensorFlow with Graphviz === === TensorFlow with Graphviz ===
Line 238: Line 273:
     * via a traditional //configure/make// procedure     * via a traditional //configure/make// procedure
  
-Whatever installation method you might choose, please note that you do not have administrative rights to the cluster. +Whatever installation method you might choose, please note that you **do not have** administrative rights to the cluster. 
  
  
Line 266: Line 301:
 </code> </code>
  
-:!: The environment variable ''EASYBUILD_OPTARCH'' instructs EasyBuild to compile software in a generic way so that it can be used on different CPUs. This is rather convenient in heterogeneous clusters such as xmaris to avoid recompilations of the same softwares on different compute nodes. This convenience comes of course at a cost; the executables so produced will not be as efficient as they would be on a given CPU. For more info read [[https://easybuild.readthedocs.io/en/latest/Controlling_compiler_optimization_flags.ht +|:!: The environment variable ''EASYBUILD_OPTARCH'' instructs EasyBuild to compile software in a generic way so that it can be used on different CPUs. This is rather convenient in heterogeneous clusters such as xmaris to avoid recompilations of the same softwares on different compute nodes. This convenience comes of course at a cost; the executables so produced will not be as efficient as they would be on a given CPU. For more info read [[https://easybuild.readthedocs.io/en/latest/Controlling_compiler_optimization_flags.ht 
-ml|here]].+ml|here]].|
  
-:!: When compiling OpenBLAS it is not sufficient to define ''EASYBUILD_OPTARCH'' to ''GENERIC'' to achieve portability of the executables. Some extra steps must be taken as described in https://github.com/easybuilders/easybuild/blob/master/docs/Controlling_compiler_optimization_flags.rst. A list of targets supported by OpenBLAS can be found [[https://github.com/xianyi/OpenBLAS/blob/develop/TargetList.txt|here]].+|:!: When compiling OpenBLAS it is not sufficient to define ''EASYBUILD_OPTARCH'' to ''GENERIC'' to achieve portability of the executables. Some extra steps must be taken as described in https://github.com/easybuilders/easybuild/blob/master/docs/Controlling_compiler_optimization_flags.rst. A list of targets supported by OpenBLAS can be found [[https://github.com/xianyi/OpenBLAS/blob/develop/TargetList.txt|here]].|
  
 Then execute  Then execute 
Line 279: Line 314:
 to make available to the ''module'' comamnd any of the softwares built in your EasyBuild userspace. to make available to the ''module'' comamnd any of the softwares built in your EasyBuild userspace.
  
-:!: ''module use <path>'' will prepend <path> to your  ''MODULEPATH''. Should you want to append it instead, then add the option ''-a''. To remove <path> from ''MODULEPATH'' execute ''module unuse <path>''.+|:!: ''module use <path>'' will prepend <path> to your  ''MODULEPATH''. Should you want to append it instead, then add the option ''-a''. To remove <path> from ''MODULEPATH'' execute ''module unuse <path>''.|
  
 Should you want to customise the building process of a given software please read how to implement [[https://easybuild.readthedocs.io/en/latest/Implementing-easyblocks.html|EasyBlocks]] and write [[https://easybuild.readthedocs.io/en/latest/Writing_easyconfig_files.html|EasyConfig]] files  or  Should you want to customise the building process of a given software please read how to implement [[https://easybuild.readthedocs.io/en/latest/Implementing-easyblocks.html|EasyBlocks]] and write [[https://easybuild.readthedocs.io/en/latest/Writing_easyconfig_files.html|EasyConfig]] files  or 
Line 585: Line 620:
 ===== Suggested readings ===== ===== Suggested readings =====
  
-  * https://slurm.schedmd.com/archive/slurm-18.08.6/+  * https://slurm.schedmd.com/archive/slurm-21.08.8-2/
   * https://osc.github.io/ood-documentation/master/   * https://osc.github.io/ood-documentation/master/
   * https://www.gnu.org/gnu/linux-and-gnu.en.html   * https://www.gnu.org/gnu/linux-and-gnu.en.html
institute_lorentz/xmaris.1654854037.txt.gz · Last modified: 2022/06/10 09:40 by lenocil