| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| easybuild_environment [2020/10/05 08:10] – [Python extensions] lenocil | easybuild_environment [2021/02/08 16:37] (current) – jansen |
|---|
| |
| ^Where ^OS^Mount Point^Remote Location^Protocol^ | ^Where ^OS^Mount Point^Remote Location^Protocol^ |
| |Workstations \\ & Servers|Fedora 31| /easybuild/easybuild/fc31| Software Server| NFS| | |Workstations \\ & Servers|Fedora 33| /easybuild/easybuild/fc31| Software Server| NFS| |
| | ::: | RHEL7| /easybuild/easybuild/el7| Software Server| NFS| | | ::: | RHEL7| /easybuild/easybuild/el7| Software Server| NFS| |
| | ::: | RHEL7/Lustre| /easybuild/easybuild/el7_lustre| Software Server| NFS| | |
| | ::: | RHEL8| /easybuild/easybuild/el8| Software Server| NFS| | | ::: | RHEL8| /easybuild/easybuild/el8| Software Server| NFS| |
| | Xmaris| CentOS 7| /marisdata/easybuild| Marisdata | NFS| | | Xmaris| CentOS 7| /marisdata/easybuild| Marisdata | NFS| |
| Failing to do so, can result in the production of __non-portable__ softwares. On the other hand, we advise you build hardware-bound softwares in all cases in which execution performance is paramount. | Failing to do so, can result in the production of __non-portable__ softwares. On the other hand, we advise you build hardware-bound softwares in all cases in which execution performance is paramount. |
| |
| Put particular attention if you are planning to build //OpenBLAS// via EasyBuild. In this case defining EASYBUILD_OPTARCH=GENERIC __is not sufficient__ to produce portable software (CPU independent). Use both ''export EASYBUILD_OPTARCH=GENERIC'' in your setup and ''--try-amend=buildopts='TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 BINARY=64 USE_THREAD=1 USE_OPENMP=1 CC="$CC" FC="$F77"' '' as EasyBuild (''eb'') runtime option. | Put particular attention if you are planning to build //OpenBLAS// via EasyBuild. In this case defining EASYBUILD_OPTARCH=GENERIC __is not sufficient__ to produce portable software (CPU independent). Use both ''export EASYBUILD_OPTARCH=GENERIC'' in your setup and ''--try-amend=buildopts='TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 BINARY=64 USE_THREAD=1 USE_OPENMP=1 CC="$CC" FC="$F77"' '' as EasyBuild (''eb'') runtime option. If your compilation fails you can try '' --try-amend=buildopts='TARGET=CORE2 BINARY=64 USE_THREAD=1 USE_OPENMP=1 CC="$CC" FC="$F77"' '' instead. |
| |
| |
| |
| ==== Heterogeneous environments ==== | ==== Heterogeneous environments ==== |
| If you are planning to use your EasyBuild-built software on a variety of CPUs, do not forget to instruct EasyBuild to do so via ''export EASYBUILD_OPTARCH=GENERIC'' and ''eb ... –try-amend=buildopts='TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 BINARY=64 USE_THREAD=1 USE_OPENMP=1 CC=“$CC” FC=“$F77”' '' as EasyBuild (eb) runtime option. | If you are planning to use your EasyBuild-built software on a variety of CPUs, do not forget to instruct EasyBuild to do so via ''export EASYBUILD_OPTARCH=GENERIC'' and ''eb ... –try-amend=buildopts='TARGET=CORE2 DYNAMIC_ARCH=1 DYNAMIC_OLDER=1 BINARY=64 USE_THREAD=1 USE_OPENMP=1 CC=“$CC” FC=“$F77”' '' as EasyBuild (eb) runtime option if your are building OpenBLAS ((https://easybuild.readthedocs.io/en/latest/Controlling_compiler_optimization_flags.html#build-environment-vs-hardcoding-in-build-scripts)). |
| |
| Do you want to know on what hardware you are? '' gcc -march=native -Q --help=target | awk '/march/{print $2}' '' | Do you want to know on what hardware you are? '' gcc -march=native -Q --help=target | awk '/march/{print $2}' '' |
| Always consult existing EasyBuild recipes and learn from them. ''grep -ri pythonbundle /easybuild/easybuild/fc31/software/EasyBuild/*/easybuild/easyconfigs'' on a workstation will return a list of easyconfigs from which you can learn all sorts of tricks that concern the ''pythonbundle'' easyblock. | Always consult existing EasyBuild recipes and learn from them. ''grep -ri pythonbundle /easybuild/easybuild/fc31/software/EasyBuild/*/easybuild/easyconfigs'' on a workstation will return a list of easyconfigs from which you can learn all sorts of tricks that concern the ''pythonbundle'' easyblock. |
| |
| | ==== Environment conflicts ==== |
| | Should any preset environment variables conflict with the correct execution of a program or should you want to modify the environment at all, you can do this directly in your [[https://easybuild.readthedocs.io/en/latest/version-specific/easyconfig_parameters.html#modules-parameters|easyconfig file]]. In the example below, we show how to //unset// the environment variable LD_LIBRARY_PATH upon module loading |
| | |
| | |
| | <code python> |
| | #... |
| | modtclfooter = "unsetenv LD_LIBRARY_PATH" |
| | modluafooter = 'unsetenv("LD_LIBRARY_PATH")' |
| | #... |
| | </code> |
| ==== Python extensions ==== | ==== Python extensions ==== |
| When you install a python package as an extension, EasyBuild checks if the extension is working properly by //python-importing// the extension name. This means that for extensions such as ''PyYAML'', the building process will fail because no module exists named ''PyYAML''. You can overcome the default behaviour by either giving the extension a custom ''modulename'' | When you install a python package as an extension, EasyBuild checks if the extension is working properly by //python-importing// the extension name. This means that for extensions such as ''PyYAML'', the building process will fail because no module exists named ''PyYAML''. You can overcome the default behaviour by either giving the extension a custom ''modulename'' |
| eb my_easyconfig.eb --skip --rebuild | eb my_easyconfig.eb --skip --rebuild |
| </code> | </code> |
| | |
| | Read about [[https://easybuild.readthedocs.io/en/latest/Partial_installations.html|partial installations]]. |
| ==== GPUs ==== | ==== GPUs ==== |
| If you are building software with GPU support, do so on a workstation/server with GPUs and specify the [[https://developer.nvidia.com/cuda-gpus|CUDA compute capability]] of the attached GPU(s), for instance ''eb ... --cuda-compute-capabilities=6.0''. | If you are building software with GPU support, do so on a workstation/server with GPUs and specify the [[https://developer.nvidia.com/cuda-gpus|CUDA compute capability]] of the attached GPU(s), for instance ''eb ... --cuda-compute-capabilities=6.0''. |
