User Tools

Site Tools


working_with_python

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
working_with_python [2020/10/05 09:39]
lenocil [METHOD 2: virtualenv]
working_with_python [2020/12/22 14:06] (current)
jansen
Line 81: Line 81:
     export PYTHONUSERBASE=$HOME/.local-rhel7     export PYTHONUSERBASE=$HOME/.local-rhel7
   fi   fi
-For users of the ''tcsh'' shell, add this to your .cshrc in stead:+For users of the ''tcsh'' shell, add this to your .tcshrc in stead:
   if (! -f /etc/fedora-release) then   if (! -f /etc/fedora-release) then
     setenv PYTHONUSERBASE $HOME/.local-rhel7     setenv PYTHONUSERBASE $HOME/.local-rhel7
Line 138: Line 138:
 virtualenv --help virtualenv --help
 </code> </code>
-==== METHOD 3: easy_install ====+ 
 +==== METHOD 3: easy_install with the `--user' option ====
 Easy Install is a python module (easy_install) that lets you automatically download, build, install, and manage Python packages.  Easy Install is a python module (easy_install) that lets you automatically download, build, install, and manage Python packages. 
-By default, easy_install installs python packages into Pythons main site-packages directory, and manages them using a custom .pth file in that same directory. Very often though, a user or developer wants easy_install to install and manage python packages in an alternative location, so +By default, easy_install installs python packages into Python's main site-packages directory, and manages them using a custom .pth file in that same directory. Very often though, a user or developer wants easy_install to install and manage python packages in an alternative location. This possible via the ''--user'' option in similar fashion to pip's
-to install package locally type+
 <code bash> <code bash>
 easy_install -N --user pymatlab easy_install -N --user pymatlab
 </code> </code>
-This will install pymatlab in ~/.local/lib ready to be imported in your next python session. +This will install pymatlab in ''${HOME}/.local/'' ready to be imported in your next python session. 
-Note: If you want to install your package in a different location than ~/.local, then set the environment variable  $PYTHONUSERBASE to a custom location, e.g, +If you want to install your package in a different location than ''$HOME/.local'', then set the environment variable  $PYTHONUSERBASE to a custom location, e.g, 
 <code bash> <code bash>
-export PYTHONUSERBASE=/home/user/some/where/I/can/write+export PYTHONUSERBASE=/home/user/some/where/I/can/write # alternative location
 </code> </code>
  
Line 156: Line 156:
 python -m easy_install --help python -m easy_install --help
 </code> </code>
 +
 +==== Migrating packages between python versions ====
 +Another issue when using personal installs may arrise on operating system upgrades, when a newer version of python is made the default (eg, moving from python 3.7 to python 3.9).
 +Notes copied from the [[https://docs.fedoraproject.org/en-US/fedora/f33/release-notes/developers/Development_Python/#_notes_on_migrating_user_installed_pip_packages|Fedora release notes]]:
 +  - Make a list of installed packages in the old python version:
 +<code bash>  
 +  python3.7 -m pip freeze > installed.txt 
 +</code>
 +  - Reinstall for the current python version:
 +<code bash>
 +  python3.9 -m pip install --user -r installed.txt
 +</code>
 +  - Optionally, uninstall the packages from the old python version and/or remove the obsolete directory under $HOME/.local/lib/python3.7
 ===== Example: how to let python search arbitrary library paths  ===== ===== Example: how to let python search arbitrary library paths  =====
-Create/edit+For instance for python v2.7 installations, create or edit
  
-''~/.local/lib/python2.7/site-packages/my-super-library.pth''+''$HOME/.local/lib/python2.7/site-packages/my-super-library.pth''
  
-by appending the path of your choice, for instance+by appending the path of your choice
  
 <code> <code>
-echo "/my/home/sweet/home/library" >> ~/.local/lib/python2.7/site-packages/my-super-library.pth+echo "/my/home/sweet/home/library" >> $HOME/.local/lib/python2.7/site-packages/my-super-library.pth
 </code> </code>
  
 All ''.pth'' files will be sourced by python provided they are in the right location. All ''.pth'' files will be sourced by python provided they are in the right location.
  
-===== Example: how to create python environment module  =====+===== Example: how to create your own python environment module  =====
  
-First enable the module package to search also private module directories +Please read [[easybuild_environment|here]].
-<code bash> +
-module load use.own +
-</code>+
  
-the line above will create a $HOME/privatemodules if it does not exist and its path will be searched for the presence of environment modules files. 
- 
-Let us  now install some packages to an arbitrary location and upgrade (__only in $PYTHONUSERBASE__) an already system-wide installed package  
- 
-<code bash> 
-export PYTHONUSERBASE=/somewhere/I/can/write/to 
-pip install --user SomePackage 
-pip install -I --user SomePackageThatWASInstalledSystemwide 
-</code> 
- 
-Create a file, say `$HOME/privatemodules/super-module' with the following contents 
-<code bash> 
-#%Module 1.0 
-# 
-#   
- 
-prepend-path  PATH          /somewhere/I/can/write/to/bin # if executables were installed 
-prepend-path  PYTHONPATH    /somewhere/I/can/write/to/lib/python2.7/site-packages 
-</code> 
- 
-and type  
- 
-<code bash> 
-module load super-module 
-</code> 
-and you are ready to use your newly created python environment. Note that is similar procedure can be repeated using python3. 
 ===== Example: numpy with openBLAS ===== ===== Example: numpy with openBLAS =====
 In this example we create a python2 virtual environment in which we will install the latest version of numpy that will use the openBLAS library.  In this example we create a python2 virtual environment in which we will install the latest version of numpy that will use the openBLAS library. 
Line 281: Line 265:
 ====== Anaconda/Miniconda ====== ====== Anaconda/Miniconda ======
 Another way of using a private python install (separate versions etc), is to install and use [[conda|Anaconda/Miniconda]]. Since these environments can encompass much more than just python, they deserve their own page (especially since they come with their own share of pitfalls). Another way of using a private python install (separate versions etc), is to install and use [[conda|Anaconda/Miniconda]]. Since these environments can encompass much more than just python, they deserve their own page (especially since they come with their own share of pitfalls).
 +
 +
 +====== Jupyter Notebooks ======
 +Depending on your operating system (Fedora or RedHat) you might get a different python kernel version as the standard kernel. If you get ''python2'' as the default kernel and only option, but wish the use the ''python3'' kernel you need to add this kernel to you local environment. This can be done by executing:
 +    python3 -m ipykernel install –user
 +Once this command has run successfully, it will have installed python3 as a jupyter kernel.
 +
 +After starting ''jupyter notebook'' you can select ''python3'' as kernel.
working_with_python.1601890798.txt.gz · Last modified: 2020/10/05 09:39 by lenocil