Configuring Institutional File Systems (SMB/SFTP/iRODS/S3)

Configuring institutional File Systems is a task for system administrators (to see the current File System configuration you need a sysadmin account). As a sysadmin, navigate to the System tab, then click on ‘Configuration’ and ‘Institutional File Systems’ button.

Example File System configuration

Managing File Systems

The ‘institutional File Systems’ page allows you to:

• add new File Systems
• modify existing File Systems (clicking on ‘Details’ button loads settings of that File System)
• delete unused File Systems (that operation fails if users have already created Filestores for that File System)

Adding or modifying a File System

Selecting ‘Details’ of existing File System, or clicking on ‘Add new File System’ button opens new panel, where details of the File System can be set.

Each File System is a set of a following settings:

1. Client type – RSpace supports connecting through Samba, SFTP, iRODS or S3 protocols
2. Name – that’s the label for the File System that will be displayed to the users
3. URL – address of the File System server
4. Status – whether system is enabled and users can use it
5. Authentication Type – different authentication methods are supported for different client types
   a) username/password – user is asked for external system credentials on their first connection to the file system in current session
   b) public key authentication – supported for SFTP connector; user can generate an SSH key pair with RSpace and register it with local key exchange service. This authentication method is tailored for University of Edinburgh public key infrastructure, so please contact RSpace first if you want to use it in your deployment.
   c) server-wide access - default option for S3 connector; the authentication is handled globally (on RSpace level) and so every RSpace user can access the file system after one is enabled by the System Administrator

Samba File System configuration

RSpace provides two connectors allowing connection to Samba servers, both supporting different versions of Samba protocol. To use the original Samba protocol (JCIFS) select 'SMBv1' as a protocol type. If your storage server supports newer Samba protocol (SMB2/3), then use 'SMBv2/3' type.

You need to provide a Samba Domain that users using this File System belong to. For SMBv2/3 connector you also need to provide the Share Name.

SFTP File System configuration

When configuring SFTP File System you’ll see additional input field for SFTP server public key.

Example File System – SFTP protocol

You need to provide a fingerprint of sftp server public key, so the secure connection can be established by RSpace. The fingerprint has to be in rsa-sha format (not ecdsa).

On the RSpace server it can be obtained by connecting to the file server with

ssh -oHostKeyAlgorithms='ssh-rsa' username@mysftpserver.org

          

then checking .ssh/known_hosts file on RSpace server. You will see a new entry like this:

|1|R8QLOZYLyYkJNpTKp8hMwGE/aMY=|KW1FpJupAPFHYWr2xi+bPSt/+P4= ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIvx0F8VJ9oOLvGNoHTwWW4KRfyU5R5cursUET53aRHTnzDG5X/L8NiP3fqKDhU15uSnbev/zdLR8hJFdqXdTUmRSzCLqr3gn8ADnc7f1/5v5g1/EfZ1Zdx1vV3Bu6OcoSpkvdOxJCgxjKzQP/PsgisCkMf6dWZva90Hrs4SIucq6LH7kJS5aV4vXcW8t6nS5SjU+bnsbyhuDV42L2xFIM89PCkJKEY+rWfiWt/CGiVgWdgmJsgugcXEYhsYYfqgAN+E1UIoWVej/pz7RVPhhYPHEhlXGZFPa+/n/6KofxI+Wz86Gt58+IOsxhlbswet3ubyMWF2Sd7Lshw1RPi0xR

          

Copy everything after, but not including ssh-rsa

In this example, you would enter the following into the RSpace dialog:

AAAAB3NzaC1yc2EAAAADAQABAAABAQDIvx0F8VJ9oOLvGNoHTwWW4KRfyU5R5cursUET53aRHTnzDG5X/L8NiP3fqKDhU15uSnbev/zdLR8hJFdqXdTUmRSzCLqr3gn8ADnc7f1/5v5g1/EfZ1Zdx1vV3Bu6OcoSpkvdOxJCgxjKzQP/PsgisCkMf6dWZva90Hrs4SIucq6LH7kJS5aV4vXcW8t6nS5SjU+bnsbyhuDV42L2xFIM89PCkJKEY+rWfiWt/CGiVgWdgmJsgugcXEYhsYYfqgAN+E1UIoWVej/pz7RVPhhYPHEhlXGZFPa+/n/6KofxI+Wz86Gt58+IOsxhlbswet3ubyMWF2Sd7Lshw1RPi0xR

          

For this mechanism to work, it must be possible for RSpace users to authenticate to the file server using username/password credentials. You may need to edit /etc/ssh/sshd_configto enable challenge based-authentication:

ChallengeResponseAuthentication yes

iRODS File System configuration

RSpace can connect to instances of the iRODS data management software (see iRODS Integration for full details).

Select the iRODS option from the Institutional File Systems configuration page and enter the details of the iRODS instance you wish to connect to as detailed below:

S3 File System configuration (new in RSpace 2.22/1.122)

RSpace can connect to S3-compatible storages, be it AWS or another systems. Users can subsequently link the S3 resources in their documents, download the linked files through RSpace UI, and include them in archive export (if deployment properties configuration allows that).

Select the S3 option from the Institutional File Systems configuration page and enter the details of the S3 storage you wish to connect to.

When selecting AWS S3 storage, you don't need to provide the url - that's because AWS has one central deployment. For other S3 providers, you need to provide the URL, and decide if 'path-style URL access' should be supported on connection (provider-specific option).

S3 Authentication: "Server-Wide access" option

RSpace needs IAM credentials (access and secret key) to be able to connect to your S3 provider. These credentials need to be provided through deployment properties:

netfilestores.s3.global.credentials.accessKey=<access key of IAM user>
netfilestores.s3.global.credentials.secretKey=<secret key of IAM user>

After migration of an Institutional File System

During the lifetime of RSpace deployment it may happen that Institutional File System is moved to a different location (URL), or that the folder structure is changed. In some cases it may be possible to keep the filestore links added to RSpace documents to work after migration, but that depends on how different the new paths are.

RSpace users interact with Institutional File Systems by creating links to resources on the Institutional File System server. The path to resource is saved in RSpace as 3 fragments:

1. URL to the File System server - configurable by system administrator on ‘System’ screen.
2. Filestore path - path fragment chosen by user to point to their starting location on a given File System
3. path to the specific linked file - within selected Filestore

Consider user-created SMB url to the test.txt file, with full path like this:

smb://howler/samba-folder/Bio Multi Meßsystem/otherFolder/test.txt

For this example, the 3 fragments saved in RSpace are:

1. smb://howler/samba-folder is a File System server part, configured by System Admin on System screen:

1. /Bio Multi Meßsystem/otherFolder/ is the filestore path, selected by the user when creating Filestore to use for the particular File System:

   

2. /test.txt is the path to actual resource, on the selected Filestore, selected by the user when writing up an RSpace document

Now let's assume the files at smb://howler system were migrated to a different server.

1. If the change only affects File System part of the path, System Admin can just update details of pre-existing File System, and the pre-existing links created by the users should keep working.
   E.g. if the file smb://howler/samba-folder/Bio Multi Meßsystem/otherFolder/test.txt is now accessible at smb://pangolin/migrated-samba-folder/Bio Multi Meßsystem/otherFolder/test.txt it'll be enough if System Admin updates URL and share-name of the server on System configuration page.
2. If the change affects part of the path saved with Filestore, there is currently no way in RSpace UI to fix these links. User should re-create the links pointing to new File System and newly created Filestore.
   E.g. if the file smb://howler/samba-folder/Bio Multi Meßsystem/otherFolder/test.txt is now accessible at smb://pangolin/migrated-samba-folder/migrated-bio/otherFolder/test.txt the only way to keep the links working would be for System Admin to update URL and share-name of the server on System configuration page, then to run SQL command updating NfsFileStore.path column in database so filestores previously pointing to Bio Multi Meßsystem/otherFolder are now pointing to migrated-bio/otherFolder.
3. If the change affects part of the path saved with ELN document, there is no way to fix these links. User should re-create the links pointing to new File System, Filestore and exact path.
   E.g. if the file smb://howler/samba-folder/Bio Multi Meßsystem/otherFolder/test.txt is now accessible at smb://pangolin/migrated-samba-folder/migrated-bio/otherFolder/migrated-test.txt there is currently no way to mass update such links.