User Tools

Site Tools


institute_lorentz:slurm_ipyparallel

This is an old revision of the document!


ipyparallel on maris

This quick tutorial will help you with the setup of your first ipyparallel session on maris' jupyterhub.

Python environments

You can either use maris-jupyterhub' python environment and all the packages therein or

create a custom python environment as the simple session below shows

export PATH=/marisdata/MARISHUB/bin:${PATH}
conda create --name=<name>
source activate <name>
conda install ipyparallel
conda install <all-packages-you-might-need>

Create an ipython parallel profile

To create an ipython parallel* profile type

export PATH=/marisdata/MARISHUB/bin:${PATH}
ipython profile create --parallel --profile=<profile-name>

the session above will create <profile-name> in ${HOME}/.ipython.

To save profiles in alternative locations set the environment variable IPYTHONDIR.

Adjust created ipython parallel profiles

mkdir "/marisdata/$USER/.log"
cp  /marisdata/EXAMPLE_FILES/ipcontroller_config.py "${HOME}/.ipython/profile_<profile-name>"
cp /marisdata/EXAMPLE_FILES/ipcluster_config.py "${HOME}/.ipython/profile_<profile-name>"
cp /marisdata/EXAMPLE_FILES/ipp_tutorial.ipynb "${HOME}/ipp_tutorial.ipynb"

Start the parallel profile

  • Navigate to the `Cluster' tab
  • Start profile <profile-name> by selecting the number of engines
  • Run the example notebook at jupyterhub.lorentz.leidenuniv.nl/user/<username>/notebooks/ipp_tutorial.ipynb
institute_lorentz/slurm_ipyparallel.1505997296.txt.gz ยท Last modified: 2017/09/21 12:34 by lenocil