JupyterHub is a multi-user server for Jupyter (AKA IPython) notebooks. Though JupyterHub you can create and share documents that contain live code, equations, visualizations and text notes (comments).

Maris cluster provides a JupyterHub service which uses a dedicated slurm partition accessible only by authorized users. Users should request access to the maris cluster as described here.

JupyterHub spawns single-user notebooks by requesting consumable resources to slurm using hard-coded parameters that define the different types of notebooks available on the system (see below). Once a user logs in the JupyterHub web interface, the notebook resources will only be allocated upon initialising the notebook server. No notebooks will be spawned in the case of insufficient consumable resources (Memory and CPU). Users are advised to check the output of the command squeue -u <username> in case no spawn occurs in a reasonable amount of time.

Maris' JupyterHub can be reached via a web browser at

https://jupyterhub.lorentz.leidenuniv.nl/

Please note that access to the page above is restricted to within the IL network.

Maris' jupyterhub has the feature admin_access enabled. This means admin users have the ability to take actions on users' behalf, such as stopping and restarting their servers.

All users that have access to maris will also be able to use JupyterHub. Launching a notebook is as easy as selecting the resources needed from the dropdown menu that appears upon a successful login. Currently, the choices are

• NOTEBOOK1: NumNodes=1 NumCPUs=4 NumTasks=1 CPUs/Task=4 mem=16G 30 days
• NOTEBOOK2: NumNodes=1 NumCPUs=2 NumTasks=1 CPUs/Task=2 mem=8G 30 days
• NOTEBOOK3: NumNodes=1 NumCPUs=1 NumTasks=1 CPUs/Task=1 mem=4G 30 days
• NOTEBOOK4: NumNodes=1 NumCPUs=16 NumTasks=1 CPUs/Task=16 mem=64G 10 days
• NOTEBOOK5: NumNodes=1 NumCPUs=16 NumTasks=1 CPUs/Task=16 mem=64G 3Days 2 GPUs

Additions to the list above are possible and should be discussed with support.

If the resources requested are available a single-user notebook instance will be launched and you will be ready to run your code interactively.

Jupyterhub will write output and error files relative to the spawning of each notebook instance using the filename scheme ${HOME}/jupyterhub_%u_%j.[log|err], where `HOME' refers to your home directory and %u and %j to your username and slurm job number respectively. Because maris' home disk has an enforced quota of 10GB per user, please remove old log files to save space.

If launching a notebook fails, please read these output files before contacting support.

It has been observed that jupyterhub `forgets' its open sessions when the slurm controller is unreachable for a long period of time. You are highly advised to save the results of your running notebooks to the disk periodically. Failure to do so might make your running notebooks inaccessible via the jupyterhub interface.

The Files tab allows users to create, modify and remove files. Furthermore, it allows to initiate a terminal application and a notebook. Because maris' jupyterhub environment contains only a python3 kernel only python3 notebooks can be instantiated unless you install your own kernels. An example session to create a custom python kernel is given below

# use system's ipython to create a python2 kernel
ipython kernel install --name=my-python-kernel --user
ls ${HOME}/.local/share/jupyter/kernels/
my-python-kernel
# or more appropriately
jupyter kernelspec list

Set the JUPYTER_PATH environment variable to provide extra directories for the kernels search path. JUPYTER_PATH should contain a series of directories, separated by `:'. Directories given in JUPYTER_PATH are searched before other locations.

Users can install their custom python environments (see here) in order to create custom kernels.

This tab shows and controls any running jupyter processes, such as notebooks and terminals.

JupyterHub users can distribute the load of their calculations to different maris' nodes using ipyparallel. This is done by selecting a predefined profile in this tab. Please follow the instructions in this guide to get started after reading our quick slurm tutorial, slurm's setup on maris and http://slurm.schedmd.com/ .

This tab allows users to list and manage conda environments using jupyter's web interface. In maris' setup, conda environments are displayed if they are in the following paths:

/marisdata/${USER}/.conda/envs
${HOME}/.conda/envs

If the creation of a new conda environment through the web interface produces an `xsrf' error, then edit your jupyter configuration file located at ${HOME}/.jupyter/jupyter_notebook_config.py adding the line

c.NotebookApp.disable_check_xsrf = True

The Leiden Institute of Physics (LION) provides a certain number of matlab licenses available to its users. Here follows a simple session that will enable your notebooks to use matlab's engine

cd /software/matlab/matlab-R2017a/extern/engines/python
python3 setup.py build --build-base="/marisdata/<username>" install --user
pip3 install imatlab --user
python3 -m imatlab install --user
# check that the matlab kernel was installed
jupyter kernelspec list
Available kernels:
  python3    
  imatlab    

Then as usual log on to https://jupyterhub.lorentz.leidenuniv.nl select the appropriate notebook profile and then the matlab kernel upon creating a new notebook.

Be aware that if there are NOT any matlab licenses available you will not be able to initiate the matlab kernel.

license availability can be checked using

license matlab

See also: https://github.com/imatlab/imatlab