Galaxy Integration

Updated by Tilo Mathes

Overview

The integration between RSpace and Galaxy is under active development. Further improvements and new functionality will be added to this page as they become available. Read more about it in our recent blog post and check out this video of an example workflow.

Galaxy is an open-source workflow platform designed to make computational methods accessible, reusable, and reproducible. Originally developed for computational biology, Galaxy has evolved into a general framework utilized across various scientific disciplines. Further information about Galaxy can be found at galaxyproject.org.

RSpace currently integrates with Galaxy in two ways:

  • You can efficiently exchange files between RSpace and Galaxy by configuring RSpace as a file source in Galaxy
  • Data attached to RSpace documents can be uploaded directly to Galaxy, where its use in analyses can be tracked and linked back to the document it originated from.

Exchanging data and provenance information between RSpace and Galaxy

To efficiently exchange files between RSpace and Galaxy, you can configure RSpace as a file source in Galaxy with the release of Galaxy version 25.0:

  1. On Galaxy, ensure your Galaxy administrator has enabled the RSpace integration.
  2. In RSpace, if you don't have one already, create an API key in your My RSpace section (see also Obtaining your RSpace API key).
  3. On Galaxy, configure a new RSpace file source under User Preferences > Manage Your Repositories > Create > RSpace.
  4. Assign a name to your RSpace file source, enter the URL to your RSpace instance, your RSpace API key, and click Create.

Once RSpace is set up as a file source in Galaxy, you can import data from the RSpace Gallery into Galaxy using the upload functionality in Galaxy.

Furthermore, you can export data and histories from Galaxy into the RSpace Gallery using the dataset export tool and the history export tool in Galaxy, respectively.

For further information about this integration can be found here: Galaxy blog post.

Uploading data to Galaxy from RSpace and tracking its use in Galaxy

The RSpace-Galaxy integration connects experimental notes with computational workflows, making it easier for researchers to document their research with provenance information about their data and computational analyses. Data attached to RSpace documents can be uploaded directly to Galaxy, where its use in analyses is tracked and linked back to the RSpace document it originated from.

Set up and configuration

  1. The new Galaxy Integration may now be made available by system administrators from the RSpace System -> Configuration page under 'Analysis and Programing':

  1. System admins must create a 'whitelist' of allowed Galaxy servers and aliases to be displayed to users for those servers. This whitelist is added as an RSpace property in the RSpace property files under 'galaxy.server.config. The following example allows the Galaxy EU and US servers: galaxy.server.config=[{"alias": "galaxy eu server", "url": "https://usegalaxy.eu"}, {"alias": "galaxy us server", "url": "https://usegalaxy.org"}]
    If you edit the value for galaxy.server.config, do not delete any existing alias values or else users who have uploaded to those Galaxy servers will see error messages
  2. Individual RSpace users must enable Galaxy from the Apps page and enter their API Key for the corresponding Galaxy instance. You may enter one key for each Galaxy server in the above whitelist configured by the sysadmin. Deleting a 'key' results in that Galaxy server instance no longer being available for uploads of RSpace data. Exisiting links to the Galaxy server in RSpace will continue to function as normal, using the deleted key to eg, track progress of workflows on the remote server.

Using the Galaxy integration

  1. A new icon will now be visible when editing text fields in RSpace documents. There will also be a compute workflow icon visible on the right of any field which has already been used to upload data to Galaxy.
  1. After clicking the Galaxy Integration button, the popup displays a table containing all data that was attached to the field in RSpace that is being edited:

  1. You must choose one of your configured Galaxy servers as the destination for the uploaded files. The Galaxy EU server is the initial default choice. Once you have selected at least one attached file, the 'Upload to Galaxy' button is enabled. Upload first creates a new 'History' in Galaxy, with a name derived from the metadata of the RSpace document and RSpace Field being edited. The history name uses the following convention:
    "RSPACE_" + document name + "_" + global ID of document + "_" + name of field data was attached to + "_" + global ID of that field.
    if you repeatedly upload data from the same field, the history will have a numeric value appended. The first upload will be called eg 'history_X_fieldname', subsequent uploads will be 'history_X_fieldname_1'.
  2. After upload, a link is displayed that will open the new history in another tab.

If you are not logged into Galaxy, clicking on these links will just open a history called 'unnamed history' which does not contain any data. You must be logged into the corresponding Galaxy instance for the links to work.

All data uploaded to the new history will be combined into a 'list dataset'. The list dataset uses exactly the same name as the history.

Finally, RSpace files uploaded to Galaxy will have annotation metadata generated for them linking back to 1) The RSpace document 2) The Gallery item 3) A download link for the RSpace data.

  1. You can now use the uploaded data to perform your computational work in Galaxy!
  2. Any fields in an RSpace document which have had attached data uploaded to Galaxy will be decorated with a new Workflow icon.

When initially opening an RSpace documentm, data is loaded from Galaxy and the icon is disabled:

Once data has completely loaded the icon will be abled with a correct count of the number of ivocations made in Galaxy using data attached to this RSpace Field.

The workflow icon also has a badge number on it, which is displayed if any data uploaded from this RSpace field is used in a Galaxy Invocation. The badge number is only updated when 1) a user opens the RSpace document or refreshes the page or 2) When a user clicks on the workflow icon.

  1. Clicking on the workflow icon opens a table displaying all data uploaded to Galaxy along with any invocations that have used the data. The state of any invocations and their created date is also shown.
Workflows can be in the following states: Running, Complete, Cancelled, Failed, Paused and Unknown. Sometimes, just after starting, a Galaxy Invocation does not have any state information associated - we display 'Unknown' for it's state in that situation. There is no guarantee that a workflow which in the Paused state will resume running again.

The table is static while it is open - closing and re-opening will update the values in the table.

A Galaxy user can run invocations or add other data manually in a history which has been created by upload of RSpace data, and those invocations might not be using any of the uploaded RSpace data. Invocations that do not use any data uploaded from RSpace will not be tracked in the above table.

How did we do?

Powered by HelpDocs (opens in a new tab)