This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
institute_lorentz:xmaris [2020/02/19 07:51] – [xmaris scientific software] lenocil | institute_lorentz:xmaris [2024/02/29 14:16] (current) – [Xmaris Partitions] jansen | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== | + | ====== |
- | xmaris | + | Xmaris |
- | financed by external research grants. | + | financed by external research grants. |
- | research groups who have purchased | + | research groups who have been awarded |
- | computing time available for others. If you would like to use xmaris | + | |
- | * Xavier Bonet Monroig | + | |Xavier Bonet Monroig| Oort 260| |
- | | + | |Carlo Beenakker|Oort 261| |
- | to see what resources can be made available | + | to discuss |
- | request | + | |
- | The cluster is optimised for [[https:// | + | | Leonardo Lenoci | HL409b| |
- | :!: Lorentz Institute | + | :!: External research groups to the Lorentz Institute are strongly |
- | ===== xmaris features and expected lifetime ===== | + | |
- | xmaris runs CentOS v7.6 and consists of heterogeneous computation nodes. A list of configured | + | Xmaris is optimised for [[https:// |
- | :!: Because xmaris features different CPU types that understand different types of instructions (see [[https:// | ||
- | < | + | Xmaris is the successor of the maris cluster, renamed with a prefix '' |
- | sinfo -o " %n %P %t %C %z %m %f %l %L" -N -n maris077 | + | |
- | | + | |
- | | + | |
- | </ | + | |
- | In the example above maris077 CPUs belong to the '' | + | [[https:// |
+ | ===== Xmaris features and expected cluster lifetime ===== | ||
- | xmaris aims to offer a stable computational environment to its users in the period **Dec 2019 -- Jan 2024**. Within this period, | + | Xmaris runs CentOS v7 and consists for historical reasons of heterogeneous computation nodes. A list of configured nodes and partitions on the cluster can be obtained on the command line using slurm' |
- | ==== compute nodes data disks ==== | + | :!: Because Xmaris features different CPU types that understand different types of instructions (see [[https:// |
- | All compute | + | You can display just one node specs and features or all nodes specs and features with '' |
- | * /scratch (local to each node, therefore faster I/O) | + | <code bash> |
- | | + | # specific |
- | | + | sinfo -o " %n %P %t %C %z %m %f" -N -n maris077 |
+ | # all nodes | ||
+ | sinfo -o " %n %P %t %C %z %m %f" -N | ||
+ | # all nodes (more concise) | ||
+ | sinfo -Nel | ||
+ | </code> | ||
- | Extra scratch | + | Xmaris aims to offer a //stable// computational environment to its users in the period **Dec 2019 -- Jan 2024**. Within this period, the OS might be patched only with important security updates. Past January 2024, all working xmaris nodes will be re-provisioned from scratch |
- | * /IBSSD (maris0[78, | + | ==== Compute nodes data disks ==== |
+ | All compute nodes have **//at least//** access to the following data partitions | ||
- | :!: iSER stands for “iSCSI Extensions for RDMA”. It is an extension of the iSCSI protocol that includes RDMA (Remote Dynamic Memory Access) support. | + | ^Mount Point^ Type ^Notes^ |
+ | |/scratch | HD | **temporary**, | ||
+ | |/marisdata |NetApp| 2TB/user quota, medium-term storage, remote| | ||
+ | |/home |NetApp| 10GB/user quota, medium-term storage, remote| | ||
+ | |/ | ||
- | Backup snapshots of /home directory | + | Extra efficient scratch spaces are available to all nodes on the infiniband network ('' |
+ | |||
+ | ^Mount Point^ Type^ Notes^ | ||
+ | |/IBSSD| SSD |**DISCONTINUED**, | ||
+ | |/PIBSSD| SSD|**temporary**, | ||
+ | |||
+ | |||
+ | Backup snapshots of '' | ||
xmaris users are strongly advised they delete (or at least move to the shared data disk), if any, their data from the compute nodes scratch disks upon completion of their calculations. All data on the scratch disks __might be cancelled without prior notice__. | xmaris users are strongly advised they delete (or at least move to the shared data disk), if any, their data from the compute nodes scratch disks upon completion of their calculations. All data on the scratch disks __might be cancelled without prior notice__. | ||
- | 10GB/user quotas are enforced on the home disk. Please use /marisdata to __temporarily__ store large datafiles. | + | Note that **disk policies might change at any time at the discretion of the cluster owners**. |
- | :!: The **OLD** ''/ | ||
+ | ---- | ||
- | **All data** on the scratch partitions | + | Please also note the following |
+ | | ||
+ | | ||
+ | |||
- | :!: maris' homes are different than IL workstations' | + | ==== Xmaris usage policies ==== |
- | ==== xmaris usage policies | + | Usage policies |
+ | <code bash> | ||
+ | scontrol show config | grep -i priority | ||
+ | </ | ||
- | Usage policies are updated regularly in accordance with the needs of the cluster owners and **may change without notice at any time**. At the moment there is an enforced usage limit of 128 CPUs per user that does not apply to the owners. Cluster usage is regularly monitored to prevent resource abuse. | + | ===== Xmaris |
- | ===== xmaris | + | To monitor live usage of Xmaris you can either |
- | To monitor live usage of xmaris | + | * execute slurm' |
+ | * browse to https://xmaris.lorentz.leidenuniv.nl/ | ||
- | * ssh into xmaris (see below) and use '' | ||
- | * use your web browser to visit https:// | ||
- | :!: The link above is accessible only within the IL workstations network. | + | ===== How to access |
- | ===== How to access | + | Access to Xmaris is not granted automatically to all Lorentz Institute members. Instead, a preliminary approval must be granted to you by the cluster owners (read [[|here]]). |
- | Once you have been authorised to use xmaris, you have two ways to access its services: | + | Once you have been authorised to use Xmaris, there are two ways to access its services: |
- | - terminal access. | + | - using a web browser (**Strongly advised** to the novel HPC user) |
- | - web access via OpenOnDemand. | + | - using an SSH client (Expert users) |
+ | |||
+ | Both methods can provide terminal access, but connections via web browsers offer you extra services such as sftp (drag-and-drop file transfers), jupyter interactive notebooks, virtual desktops and more at the click of your mouse. **We advise** all users either unfamilair with the GNU/Linux terminal or new to HPC to use a web browser to interact with Xmaris. | ||
- | ==== Terminal access | + | ==== Access via an ssh client |
+ | The procedure differs on whether you try to connect with a client connected to the IL intranet or not. | ||
- | Terminal access is provided via login to xmaris' | + | * When within |
+ | <code bash> | ||
+ | ssh xmaris.lorentz.leidenuniv.nl -l username | ||
+ | </ | ||
+ | |||
+ | * When outside the IL network, for instance from home or using a wireless connection, you must first initiate an ssh tunnel to our SSH server and then connect to Xmaris | ||
+ | |||
+ | <code bash> | ||
+ | ssh -o ProxyCommand=" | ||
+ | </ | ||
- | {{ : | ||
- | :!: If you were a maris user prior to the configuration switch to xmaris, you will find out that many terminal functions and programs | + | |:!: If you were a maris user prior to the configuration switch to xmaris, you might find out that many terminal functions and programs |
==== Web access ==== | ==== Web access ==== | ||
- | xmaris | + | Xmaris |
- | {{ : | + | |
- | Similarly to a standard terminal access, xmaris OpenOnDemand is available only for connections within the IL subnetwork. IL users who wish to access OpenOnDemand from their home could instruct their browser to connect via a SOCKS proxy, for instance open a local terminal and type | + | |
- | < | + | {{ : |
- | ssh -ND 7777 < | + | |
+ | Similarly to a traditional shell access, Xmaris OpenOnDemand is available only for connections within the __IL intranet__. IL users who wish to access OpenOnDemand from their remote home locations could for example use the [[: | ||
+ | Open a local terminal and type (substitute username with your IL username) | ||
+ | |||
+ | < | ||
+ | ssh -ND 7777 username@ssh.lorentz.leidenuniv.nl | ||
</ | </ | ||
- | then in your browser settings find the tab relative to the connection type and instruct the browser to use the SOCKS proxy at '' | + | then in your browser settings find the tab relative to the connection type and instruct the browser to use the SOCKS proxy located |
- | xmaris | + | |
+ | Xmaris | ||
* Create/edit files and directories. | * Create/edit files and directories. | ||
* Submit batch jobs to the slurm scheduler/ | * Submit batch jobs to the slurm scheduler/ | ||
* Open a terminal. | * Open a terminal. | ||
- | * Launch interactive jupyter notebooks. | + | * Launch interactive |
* Monitor cluster usage. | * Monitor cluster usage. | ||
* Create and launch your very own OnDemand application (read [[https:// | * Create and launch your very own OnDemand application (read [[https:// | ||
+ | :!: Please do not bookmark any other URL than https:// | ||
+ | ===== Xmaris Partitions ===== | ||
- | ===== xmaris GPUs ===== | + | ^Partition^ Number nodes ^ Timelimit ^ Notes^ |
+ | |compAMD*| 6 | 15 days | | | ||
+ | |compAMDlong| 3 | 60 days | | | ||
+ | |compIntel| 2 | 5 days and 12 hours| | | ||
+ | |gpuIntel| 1 | 3 days and 12 hours | GPU | | ||
+ | |ibIntel | 8 | 7 days | InfiniBand, Multiprocessing | | ||
- | Currently only maris075 features | + | *: default partition |
- | ===== xmaris | + | |
+ | ===== Xmaris | ||
+ | |||
+ | ^Node^Partition^GPUs^CUDA compatibility^ | ||
+ | |maris075 |gpuIntel|2 x Nvidia | ||
+ | |||
+ | Xmaris GPUs must be allocated using slurm' | ||
+ | <code bash> | ||
+ | srun -p gpuIntel --gres=gpu: | ||
+ | </ | ||
+ | ===== Xmaris | ||
- | xmaris | + | Xmaris |
Line 131: | Line 178: | ||
|OpenMPI | |OpenMPI | ||
|Python | |Python | ||
+ | |PyCUDA | ||
|Perl | Programming Language | | |Perl | Programming Language | | ||
|R | R is a Language and Environment for Statistical Computing and Graphics | | |R | R is a Language and Environment for Statistical Computing and Graphics | | ||
|Singularity | Containers software | | |Singularity | Containers software | | ||
+ | | Tensorflow | Machine Learning Platform | | ||
|plc | The Planck Likelihood Code | | |plc | The Planck Likelihood Code | | ||
+ | |cobaya | A code for Bayesian analysis in Cosmology | | ||
|Clang | |Clang | ||
- | | Mathematica| | + | |Graphviz | Graph visualization software | |
+ | |Octave | ||
+ | | Mathematica* | Technical | ||
+ | * Usage of proprietary software is discouraged. | ||
+ | For an up-to-date list of installed software use the '' | ||
+ | Any pre-installed software can be made available in your environment via the '' | ||
- | For an up-to-date list of installed software use the '' | + | It is possible |
- | Any pre-installed software can be sourced by means of the '' | + | < |
- | + | ||
- | Sometimes it is useful | + | |
- | < | + | |
module load mod1 mod2 mod3 mod4 mod5 mod6 | module load mod1 mod2 mod3 mod4 mod5 mod6 | ||
modules save collection1 | modules save collection1 | ||
Line 151: | Line 203: | ||
module savelist | module savelist | ||
</ | </ | ||
+ | ==== TensorFlow Notes ==== | ||
+ | Xmaris has multiple modules that provide TensorFlow. See '' | ||
+ | |||
+ | ^Module^Hardware^Partition^Additional Ops^ | ||
+ | |TensorFlow/ | ||
+ | |TensorFlow/ | ||
+ | |TensorFlow-1.15.0-Miniconda3/ | ||
+ | |||
+ | The following example shows how you can create a tensorflow-aware jupyter notebook kernel that you can use for instance via the OpenOnDemand interface | ||
+ | |||
+ | <code bash> | ||
+ | # We use maris075 (GPU node) and load the optimised tf module | ||
+ | ml load TensorFlow/ | ||
+ | |||
+ | # We install ipykernel, because necessary to run py notebooks | ||
+ | python -m pip install ipykernel --user | ||
+ | |||
+ | # We create a kernel called TFQuantum based on python from TensorFlow/ | ||
+ | python -m ipykernel install --name TFQuantum --display-name " | ||
+ | |||
+ | # We edit the kernel such that it does not execute python directly | ||
+ | # but via a custom wrapper script | ||
+ | cat $HOME/ | ||
+ | |||
+ | { | ||
+ | " | ||
+ | "/ | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | ], | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | " | ||
+ | } | ||
+ | } | ||
+ | |||
+ | # The wrapper script will call python but only after loading any | ||
+ | # appropriate module | ||
+ | cat / | ||
+ | |||
+ | #!/bin/env bash | ||
+ | ml load TensorFlow/ | ||
+ | |||
+ | exec python $@ | ||
+ | |||
+ | # DONE. tfquantum will appear in the dropdown list of kernels | ||
+ | # upon creating a new notebook | ||
+ | |||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | === TensorFlow with Graphviz === | ||
+ | <code bash> | ||
+ | ml load TensorFlow/ | ||
+ | pip install --user pydot | ||
+ | ml load Graphviz/ | ||
+ | python -c " | ||
+ | </ | ||
+ | |||
==== Installing extra software ==== | ==== Installing extra software ==== | ||
- | | + | If you need to run a software that is not present on Xmaris, you might: |
+ | |||
+ | | ||
- Install it yourself | - Install it yourself | ||
- | * via EasyBuild (see instructions below on how to setup your EasyBuild environment) | + | * via EasyBuild (see instructions below on how to setup your personal |
- | * using your own method | + | * via a traditional // |
+ | |||
+ | Whatever installation | ||
- | :!: You do not have administrative rights to the cluster. | ||
=== Installing software via EasyBuild === | === Installing software via EasyBuild === | ||
- | Load the EasyBuild module | + | :!: See also [[: |
+ | |||
+ | In order to use EasyBuild to build a software, you must first set up your development environment. This is usually done by | ||
+ | |||
+ | * Loading | ||
+ | * Indicating | ||
+ | * Specifying EasyBuild' | ||
+ | * Build a software | ||
+ | |||
+ | In their simplest form, the steps outlined above can be translated into the following shell commands | ||
+ | |||
+ | <code bash> | ||
- | < | ||
- | module avail EasyBuild | ||
module load EasyBuild | module load EasyBuild | ||
mkdir / | mkdir / | ||
export EASYBUILD_PREFIX=/ | export EASYBUILD_PREFIX=/ | ||
- | export EASYBUILD_OPTARCH=GENERIC* | + | export EASYBUILD_OPTARCH=GENERIC |
+ | |||
+ | eb -S ^Miniconda | ||
+ | eb Miniconda2-4.3.21.eb -r | ||
</ | </ | ||
- | :!: NOTE 1: The environment variable '' | + | |:!: The environment variable '' |
- | ml|here]]. | + | ml|here]].| |
- | :!: NOTE 2: When compiling OpenBLAS it is not sufficient to define '' | + | |:!: When compiling OpenBLAS it is not sufficient to define '' |
- | Search a software to build, build it and make it available to your environment | + | Then execute |
< | < | ||
- | eb -S ^Miniconda | ||
- | eb Miniconda2-4.3.21.eb -r | ||
module use / | module use / | ||
</ | </ | ||
- | :!: '' | + | to make available to the '' |
+ | |||
+ | |:!: '' | ||
Should you want to customise the building process of a given software please read how to implement [[https:// | Should you want to customise the building process of a given software please read how to implement [[https:// | ||
- | contact Leonardo Lenoci (HL409b) | + | contact Leonardo Lenoci (HL409b). |
- | ===== How to run a computation ===== | + | == Working with conda modules |
- | xmaris | + | Several conda modules are ready-to-use on maris. |
+ | This causes several problems in all cases in which you are supposed to work in a clean environment. | ||
+ | |||
+ | The steps below show as an example how you could skip '' | ||
+ | |||
+ | < | ||
+ | > ml load Miniconda3/ | ||
+ | > # note that if you specify prefix, you cannot specify the name | ||
+ | > conda create [--prefix < | ||
+ | |||
+ | # the following fails | ||
+ | > conda activate TEST | ||
+ | |||
+ | CommandNotFoundError: | ||
+ | To initialize your shell, run | ||
+ | |||
+ | $ conda init < | ||
+ | |||
+ | Currently supported shells are: | ||
+ | - bash | ||
+ | - fish | ||
+ | - tcsh | ||
+ | - xonsh | ||
+ | - zsh | ||
+ | - powershell | ||
+ | |||
+ | See 'conda init --help' | ||
+ | |||
+ | IMPORTANT: You may need to close and restart your shell after running 'conda init' | ||
+ | |||
+ | ## do this instead | ||
+ | > source activate TEST | ||
+ | > # or if you used the --prefix option to crete the env | ||
+ | > # source activate < | ||
+ | > ... | ||
+ | > conda deactivate | ||
+ | |||
+ | </ | ||
+ | ===== How to run a computation on Xmaris ===== | ||
+ | |||
+ | Xmaris | ||
==== Batch jobs ==== | ==== Batch jobs ==== | ||
- | + | ||
- | To submit a batch job to slurm you must first create a shell script which contains enough instructions to request the needed resources | + | Batch jobs are computation jobs that do not execute interactively. |
- | For instance a bash batch script | + | |
+ | To submit a batch job to Xmaris' | ||
+ | An example | ||
< | < | ||
cat test.sh | cat test.sh | ||
Line 207: | Line 381: | ||
</ | </ | ||
- | Please consult the [[https:// | + | Batch scripts are then submitted for execution via '' |
< | < | ||
sbatch test.sh | sbatch test.sh | ||
sbatch: Submitted batch job 738279474774293 | sbatch: Submitted batch job 738279474774293 | ||
</ | </ | ||
+ | |||
and their status [PENDING|RUNNING|FAILED|COMPLETED] checked using '' | and their status [PENDING|RUNNING|FAILED|COMPLETED] checked using '' | ||
- | ssh shell access to an executing node is automatically | + | ssh __shell |
+ | |||
+ | Please consult the [[https:// | ||
==== Interactive jobs ==== | ==== Interactive jobs ==== | ||
- | Interactive jobs give you nodes shell prompts | + | Interactive jobs are usually used for debugging purposes and in those cases |
+ | in which the computation requires human interaction. Using interactive | ||
< | < | ||
Line 224: | Line 404: | ||
</ | </ | ||
- | ===== Parallelism 101 ===== | + | or execute an interactive program |
- | :!: Inexpert users should refrain from attempting to program parallel applications without studying appropriately. | + | < |
+ | srun -p compIntel -N1 -n 1 --mem=4000 --pty python -c " | ||
+ | Hello world | ||
+ | ^D | ||
+ | [' | ||
+ | </ | ||
+ | ===== Parallelism 101 ===== | ||
A parallel job runs a calculation whose computational subtasks are run simultaneously. The underlying principle is that | A parallel job runs a calculation whose computational subtasks are run simultaneously. The underlying principle is that | ||
Line 237: | Line 423: | ||
==== Multithreading ==== | ==== Multithreading ==== | ||
- | In multithreading programming all computation subtasks ('' | + | In multithreading programming all computation subtasks ('' |
Multithreading can be achieved using libraries such as '' | Multithreading can be achieved using libraries such as '' | ||
Line 244: | Line 430: | ||
==== Multiprocessing ==== | ==== Multiprocessing ==== | ||
- | Multiprocessing usually refers to computations subdivided into tasks that run on multiples nodes. This type of programming increases the resources available to your computation by employing several nodes at the same time. '' | + | Multiprocessing usually refers to computations subdivided into tasks that run on multiples nodes. This type of programming increases the resources available |
MPI-enabled applications spawn multiple copies of the program, also called //ranks//, mapping each one of them to a processor. A computation node has usually multiple processors. The MPI interface lets you manage the allocated resources and the communication and synchronisation of the '' | MPI-enabled applications spawn multiple copies of the program, also called //ranks//, mapping each one of them to a processor. A computation node has usually multiple processors. The MPI interface lets you manage the allocated resources and the communication and synchronisation of the '' | ||
Line 255: | Line 441: | ||
===== How to launch a jupyter notebook ===== | ===== How to launch a jupyter notebook ===== | ||
- | To launch a jupyter notebook login to [[https://marishead.lorentz.leidenuniv.nl: | + | To launch a jupyter notebook login to [[https://xmaris.lorentz.leidenuniv.nl: |
- | '' | + | Now you can interact with your notebook (click on '' |
- | {{ : | + | |
- | push '' | + | If your notebook |
- | {{ : | + | * Check the status of your jobs in the queue with '' |
+ | * Examine the notebook log files (click on '' | ||
+ | * If the suggestions above do not help, contact '' | ||
+ | ==== How to launch a jupyter notebook that uses GPUs ==== | ||
+ | |||
+ | Repeat the steps above but make sure you select an appropriate GPU partition. Moreover, you must add an appropriate CUDA module to the field '' | ||
+ | The form field '' | ||
- | Now you can interact with your notebook (click on '' | ||
- | :!: If you want your notebook directory to be different than $HOME, please do '' | + | :!: NOTE1: If you want your notebook directory to be different than $HOME, please do '' |
+ | :!: NOTE2: Any form fields left empty will assume pre-programmed values. For instance, you do not need to | ||
+ | specify your slurm account because it will default to the account of your PI. | ||
==== Launching jupyterlab instead of jupyter notebook ==== | ==== Launching jupyterlab instead of jupyter notebook ==== | ||
Line 273: | Line 465: | ||
< | < | ||
- | https://marishub.lorentz.leidenuniv.nl: | + | https://xmaris.lorentz.leidenuniv.nl: |
</ | </ | ||
Line 321: | Line 513: | ||
conda env remove --name py35 | conda env remove --name py35 | ||
</ | </ | ||
+ | |||
+ | === Install a mathematica (wolfram) kernel === | ||
+ | |||
+ | You can set it up following these notes https:// | ||
+ | |||
+ | * Open an SSH connection to xmaris | ||
+ | * Run ''/ | ||
+ | * The '' | ||
+ | |||
==== Debugging jupyter lab/ | ==== Debugging jupyter lab/ | ||
Line 330: | Line 531: | ||
===== xmaris slurm tips ===== | ===== xmaris slurm tips ===== | ||
- | :!: This is not a slurm manual, you should always | + | :!: This is not a slurm manual, you should always |
xmaris runs the scheduler and resource manager slurm **v18.08.6-2**. Please consult the [[https:// | xmaris runs the scheduler and resource manager slurm **v18.08.6-2**. Please consult the [[https:// | ||
- | :!: The headnode (marishead) is not a compute node. Any user applications running on it will be terminated without notice. | + | :!: The headnode (xmaris.lorentz.leidenuniv.nl) is not a compute node. Any user applications running on it will be terminated without notice. |
- | Here we report a few useful commands and their outputs to get you started. | + | Here we report a few useful commands and their outputs to get you started. For the inpatients, look at the following [[https:// |
==== Determine your slurm account name ==== | ==== Determine your slurm account name ==== | ||
Line 379: | Line 580: | ||
srun --constraint=" | srun --constraint=" | ||
</ | </ | ||
+ | |||
+ | ==== Run a multiprocessing application ==== | ||
+ | |||
+ | Xmaris supports OpenMPI in combination with slurm' | ||
+ | First of all, make sure that the '' | ||
+ | <code bash> | ||
+ | # ulimit -l | ||
+ | unlimited | ||
+ | </ | ||
+ | |||
+ | If that is NOT the case, please contact the IT support. | ||
+ | |||
+ | To run an MPI application see the session below | ||
+ | |||
+ | <code bash> | ||
+ | # login to the headnode and request resources | ||
+ | $ salloc | ||
+ | salloc: Granted job allocation 564086 | ||
+ | salloc: Waiting for resource configuration | ||
+ | salloc: Nodes maris[078-083] are ready for job | ||
+ | # load the needed modules for the app to run | ||
+ | $ ml load OpenMPI/ | ||
+ | # execute the app (note that the default MPI is set to pmi2) | ||
+ | $ srun ./ | ||
+ | Hello world! | ||
+ | 11.000000 -9.000000 5.000000 -9.000000 21.000000 -1.000000 5.000000 -1.000000 3.000000 | ||
+ | Hello world! | ||
+ | 11.000000 -9.000000 5.000000 -9.000000 21.000000 -1.000000 5.000000 -1.000000 3.000000 | ||
+ | Hello world! | ||
+ | 11.000000 -9.000000 5.000000 -9.000000 21.000000 -1.000000 5.000000 -1.000000 3.000000 | ||
+ | Hello world! | ||
+ | 11.000000 -9.000000 5.000000 -9.000000 21.000000 -1.000000 5.000000 -1.000000 3.000000 | ||
+ | Hello world! | ||
+ | 11.000000 -9.000000 5.000000 -9.000000 21.000000 -1.000000 5.000000 -1.000000 3.000000 | ||
+ | Hello world! | ||
+ | 11.000000 -9.000000 5.000000 -9.000000 21.000000 -1.000000 5.000000 -1.000000 3.000000 | ||
+ | |||
+ | </ | ||
+ | |||
===== Suggested readings ===== | ===== Suggested readings ===== | ||
- | https:// | + | * https:// |
+ | * https:// | ||
+ | * https:// | ||
+ | * https:// | ||
+ | * https:// | ||
+ | * https:// | ||
+ | * http:// | ||
+ | * https:// | ||
+ | * https:// | ||
+ | * https:// | ||
+ | ===== Requesting help ===== | ||
- | https://osc.github.io/ | + | Please use this [[https://helpdesk.lorentz.leidenuniv.nl|helpdesk]] form or email '' |
- | https:// | + | ===== Recent Scientific Publications from maris ===== |
- | https:// | + | ==== Quantum physics and Quantum computing ==== |
- | https:// | + | * [[https:// |
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
- | https:// | + | ==== Statistical, |
- | http://www.tldp.org/LDP/GNU-Linux-Tools-Summary/GNU-Linux-Tools-Summary.pdf | + | * [[https://journals.aps.org/pre/abstract/ |
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
- | https://easybuild.readthedocs.io/en/latest/ | + | ==== Particles, fields, gravitation, |
+ | * [[https://journals.aps.org/prd/abstract/10.1103/ | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
- | https:// | + | ==== String theory ==== |
- | https://en.wikipedia.org/wiki/Parallel_computing | + | * [[https://link.springer.com/article/10.1007%2FJHEP01%282020%29151|Isolated zeros destroy Fermi surface in holographic models with a lattice]] |
- | ===== Requesting help ===== | + | |
- | Please use this [[https:// |