This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
institute_lorentz:irods_fair_storage [2022/10/20 07:29] – [FAIR Storage Access] lenocil | institute_lorentz:irods_fair_storage [2023/06/13 07:36] (current) – lenocil | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== FAIR Storage Service ====== | ====== FAIR Storage Service ====== | ||
- | |||
- | |:!: DRAFT: Doc in preparation | | ||
- | |Comments, suggestions and corrections are really appreciated | | ||
Lorentz Institute offers its members a data storage option that complies with the //**FAIR** Guiding Principles for scientific data management and stewardship.// | Lorentz Institute offers its members a data storage option that complies with the //**FAIR** Guiding Principles for scientific data management and stewardship.// | ||
Line 18: | Line 15: | ||
===== FAIR Storage Access ===== | ===== FAIR Storage Access ===== | ||
- | Access | + | Writing access |
Once access has been granted, you can interact with the server via any of the [[https:// | Once access has been granted, you can interact with the server via any of the [[https:// | ||
Line 24: | Line 21: | ||
* [[https:// | * [[https:// | ||
* [[https:// | * [[https:// | ||
+ | |||
+ | Additionally our [[https:// | ||
iCommands is the most flexible and powerful client, but it might require a basic prior knowledge of the GNU/Linux command line. | iCommands is the most flexible and powerful client, but it might require a basic prior knowledge of the GNU/Linux command line. | ||
Line 75: | Line 74: | ||
Please not that even if you specify a default resource name in your iRODS configuration file, it is always possible to overwrite the resource destination during iCommands operations by specifying the option '' | Please not that even if you specify a default resource name in your iRODS configuration file, it is always possible to overwrite the resource destination during iCommands operations by specifying the option '' | ||
- | In a terminal window, load the iCommands module | + | In a terminal window |
<code bash> | <code bash> | ||
Line 164: | Line 163: | ||
== Metadata == | == Metadata == | ||
+ | Metadata are one of the most important features of iRODS that makes this software compliant | ||
+ | |||
iRODS metadata are defined by Attribute-Value-Unit (AVU) triplets, for instance | iRODS metadata are defined by Attribute-Value-Unit (AVU) triplets, for instance | ||
Length 10 meters | Length 10 meters | ||
Line 184: | Line 185: | ||
|Grant Other IL User Recursive READ Access to Collection | ichmod -r read IL_USERNAME collection_path| | |Grant Other IL User Recursive READ Access to Collection | ichmod -r read IL_USERNAME collection_path| | ||
|List Existing Groups |iquest " | |List Existing Groups |iquest " | ||
+ | |||
+ | == Integrity: Checksums == | ||
+ | ^More Info:^ **iput -h, irsync -h, ichksum -h**^ | ||
+ | |Check Object Integrity during Transfer| iput -[r]K object| | ||
+ | |Check Object Integrity during Transfer| irsync -[r]K source i: | ||
+ | |||
== External Collaborators == | == External Collaborators == | ||
Line 189: | Line 196: | ||
iRODS lets you easily share your data with external collaborators (users unknown to the IL systems). The only requirement is that they have access to iCommands or have a web browser. See the examples below | iRODS lets you easily share your data with external collaborators (users unknown to the IL systems). The only requirement is that they have access to iCommands or have a web browser. See the examples below | ||
- | iCommands: Tickets | + | __iCommands: Tickets__ |
iRODS tickets constitute a powerful and flexible way to share your data with external collaborators who have access to iCommands at their institutions. In the example session below an IL user creates a // | iRODS tickets constitute a powerful and flexible way to share your data with external collaborators who have access to iCommands at their institutions. In the example session below an IL user creates a // | ||
Line 229: | Line 236: | ||
</ | </ | ||
- | If your collaborator has no access to icommands, then place the object you would like to share in the folder ''/ | + | If your collaborator has no access to icommands, then place the object you would like to share in the folder ''/ |
- | == Integrity: Checksums == | ||
- | ^More Info:^ **iput -h, irsync -h, ichksum -h**^ | ||
- | |Check Object Integrity during Transfer| iput -[r]K object| | ||
- | |Check Object Integrity during Transfer| irsync -[r]K source i: | ||
+ | __Web Browsers: webdav__ | ||
- | = Web Browsers: webdav | + | Without creating a ticket for a data object, you could login to our //FAIR// storage system place any object you would like to share in a special collection called ''/ |
+ | |||
+ | Access via the ticket system using a web browser will follow the scheme | ||
+ | < | ||
+ | https:// | ||
+ | # example | ||
+ | https:// | ||
+ | </ | ||
- | Without creating a ticket for a data object, you could login to our //FAIR// storage system place any object you would like to share in a special collection called ''/ | ||
=== Custom Lorentz Institute Ingestion Rules === | === Custom Lorentz Institute Ingestion Rules === | ||
Line 261: | Line 271: | ||
</ | </ | ||
- | and uploaded | + | and upload |
<code bash> | <code bash> | ||
Line 275: | Line 285: | ||
|version| Data Versioning Number | //0.0.2//| | |version| Data Versioning Number | //0.0.2//| | ||
|tags|Keywords to your Data| //Blackhole Gravity Quantum Computers// | |tags|Keywords to your Data| //Blackhole Gravity Quantum Computers// | ||
- | |doi | If related to published material, Digital object Identifier((https:// | + | |dois | If related to published material, Digital object Identifier((https:// |
|pi| Name of Principal Investigator| //John B. Smith//| | |pi| Name of Principal Investigator| //John B. Smith//| | ||
|funder| Name organization funding this research | //NWO//| | |funder| Name organization funding this research | //NWO//| | ||
Line 288: | Line 298: | ||
=== Web Browser Access === | === Web Browser Access === | ||
- | |URL | https:// | + | |URL | https:// |
|Username| IL Username| | |Username| IL Username| | ||
|Password| IL Password| | |Password| IL Password| | ||
Line 296: | Line 306: | ||
Anonymous access | Anonymous access | ||
- | |URL | https:// | + | |URL | https:// |
Line 310: | Line 320: | ||
Menu -> Places -> Connect to Server((This may vary depending on the OS. If in trouble search for Connect To Server.)) | Menu -> Places -> Connect to Server((This may vary depending on the OS. If in trouble search for Connect To Server.)) | ||
- | |Server| access.lorentz.leidenuniv.nl| | + | |Server| access.lorentz.leidenuniv.nl/davrods| |
|Port | 443| | |Port | 443| | ||
|Type|Secure WebDAV (HTTPS)| | |Type|Secure WebDAV (HTTPS)| | ||
Line 320: | Line 330: | ||
Finder -> Go -> Connect to Server | Finder -> Go -> Connect to Server | ||
- | |Server Address| https:// | + | |Server Address| https:// |
Specify your IL credentials when prompted and click on Connect. | Specify your IL credentials when prompted and click on Connect. | ||
+ | |||
+ | ==== Rclone Access ==== | ||
+ | |||
+ | :!: This access method works only on our remote desktop system | ||
+ | |||
+ | Open a terminal | ||
+ | |||
+ | <code bash> | ||
+ | $ rclone config | ||
+ | No remotes found - make a new one | ||
+ | n) New remote | ||
+ | s) Set configuration password | ||
+ | q) Quit config | ||
+ | n/s/q> n | ||
+ | name> iRODS | ||
+ | Type of storage to configure. | ||
+ | Enter a string value. Press Enter for the default ("" | ||
+ | Choose a number from below, or type in your own value | ||
+ | 1 / 1Fichier | ||
+ | \ " | ||
+ | 2 / Alias for an existing remote | ||
+ | \ " | ||
+ | 3 / Amazon Drive | ||
+ | \ " | ||
+ | 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, and Tencent COS | ||
+ | \ " | ||
+ | ... | ||
+ | 36 / Union merges the contents of several upstream fs | ||
+ | \ " | ||
+ | 37 / Webdav | ||
+ | \ " | ||
+ | 38 / Yandex Disk | ||
+ | \ " | ||
+ | 39 / Zoho | ||
+ | \ " | ||
+ | 40 / http Connection | ||
+ | \ " | ||
+ | 41 / premiumize.me | ||
+ | \ " | ||
+ | 42 / seafile | ||
+ | \ " | ||
+ | Storage> 37 | ||
+ | |||
+ | ** See help for webdav backend at: https:// | ||
+ | |||
+ | URL of http host to connect to | ||
+ | Enter a string value. Press Enter for the default ("" | ||
+ | Choose a number from below, or type in your own value | ||
+ | 1 / Connect to example.com | ||
+ | \ " | ||
+ | url> https:// | ||
+ | Name of the Webdav site/ | ||
+ | Enter a string value. Press Enter for the default ("" | ||
+ | Choose a number from below, or type in your own value | ||
+ | 1 / Nextcloud | ||
+ | \ " | ||
+ | 2 / Owncloud | ||
+ | \ " | ||
+ | 3 / Sharepoint Online, authenticated by Microsoft account. | ||
+ | \ " | ||
+ | 4 / Sharepoint with NTLM authentication. Usually self-hosted or on-premises. | ||
+ | \ " | ||
+ | 5 / Other site/ | ||
+ | \ " | ||
+ | vendor> iRODS | ||
+ | User name. In case NTLM authentication is used, the username should be in the format ' | ||
+ | Enter a string value. Press Enter for the default ("" | ||
+ | user> YOUR_IL_USERNAME | ||
+ | Password. | ||
+ | y) Yes type in my own password | ||
+ | g) Generate random password | ||
+ | n) No leave this optional password blank (default) | ||
+ | y/g/n> y | ||
+ | Enter the password: | ||
+ | password: | ||
+ | Confirm the password: | ||
+ | password: | ||
+ | Bearer token instead of user/pass (e.g. a Macaroon) | ||
+ | Enter a string value. Press Enter for the default ("" | ||
+ | bearer_token> | ||
+ | Edit advanced config? (y/n) | ||
+ | y) Yes | ||
+ | n) No (default) | ||
+ | y/n> n | ||
+ | Remote config | ||
+ | -------------------- | ||
+ | [iRODS] | ||
+ | type = webdav | ||
+ | url = https:// | ||
+ | vendor = iRODS | ||
+ | user = YOUR_IL_USERNAME | ||
+ | pass = *** ENCRYPTED *** | ||
+ | -------------------- | ||
+ | y) Yes this is OK (default) | ||
+ | e) Edit this remote | ||
+ | d) Delete this remote | ||
+ | y/e/d> y | ||
+ | Current remotes: | ||
+ | |||
+ | Name Type | ||
+ | ==== ==== | ||
+ | iRODS webdav | ||
+ | |||
+ | e) Edit existing remote | ||
+ | n) New remote | ||
+ | d) Delete remote | ||
+ | r) Rename remote | ||
+ | c) Copy remote | ||
+ | s) Set configuration password | ||
+ | q) Quit config | ||
+ | e/ | ||
+ | |||
+ | $ | ||
+ | |||
+ | |||
+ | </ | ||
+ | |||
+ | The navigate to https:// |