There is an implementation for using the AWS S3 object store as the data
store for a given Galaxy instance. The implementation is located here
and it offers several config options in universe_wsgi.ini.
The data stored in S3 is locally cached while it's being operated on but
always synced with the back end object store.
Pulsar seems to have some support for S3 but, as the docs say in the
implementation, it's explicitly beta:
As a side note, there are some planned enhancements to how the object store
implementation is handled and there will hopefully be quite a bit of
activity on this topic in the near future (eg, https://trello.com/c/YynQKq8m
Hope this at least clarifies the state of object store support,
On Mon, Aug 25, 2014 at 10:24 AM, Raknes Inge Alexander <
> I have a few questions about object stores in Galaxy:
> 1: Can all Galaxy data sets be stored in an object store?
> 2: If so, does Galaxy still need to maintain a local copy of the data?
> 3: Is LWR or Pulsar able to get the data directly from the object store,
> or does it still have to go through Galaxy?
> We are planning to let users of our Galaxy installation handle large
> input/output files (~30G) and we expect that the VM containing our Galaxy
> installation will become a bottleneck if all data needs to travel
> through that node.
> - Inge Alexander Raknes
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
> To search Galaxy mailing lists use the unified search at:
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
To search Galaxy mailing lists use the unified search at: