User Tools

Site Tools


institute_lorentz:slurm_ipyparallel

ipyparallel on maris

This quick tutorial will help you with the setup of your first ipyparallel session on maris' jupyterhub.

Python environments

You can either use maris-jupyterhub' python environment and all the packages therein or

create a custom python environment as the simple session below shows

export PATH=/marisdata/MARISHUB/bin:${PATH}
conda create --name=<name>
source activate <name>
conda install ipyparallel
conda install <all-packages-you-might-need>

Create an ipython parallel profile

To create an ipython parallel* profile type

export PATH=/marisdata/MARISHUB/bin:${PATH}
ipython profile create --parallel --profile=<profile-name>

the session above will create <profile-name> in ${HOME}/.ipython.

To save profiles in alternative locations set the environment variable IPYTHONDIR.

Adjust created ipython parallel profiles

mkdir "/marisdata/$USER/.log"
cp  /marisdata/EXAMPLE_FILES/ipcontroller_config.py "${HOME}/.ipython/profile_<profile-name>"
cp /marisdata/EXAMPLE_FILES/ipcluster_config.py "${HOME}/.ipython/profile_<profile-name>"
cp /marisdata/EXAMPLE_FILES/ipp_tutorial.ipynb "${HOME}/ipp_tutorial.ipynb"

Start the parallel profile

  • Navigate to the `Cluster' tab
  • Start profile <profile-name> by selecting the number of engines
  • Run the example notebook at jupyterhub.lorentz.leidenuniv.nl/user/<username>/notebooks/ipp_tutorial.ipynb
institute_lorentz/slurm_ipyparallel.txt · Last modified: 2017/09/21 12:34 by lenocil