In my case I ended modifying the API workflow_execute.py script and
used it to create symlinks I could use to run processes outside of
galaxy. In case the new processing machine doesn't have access to the
directory tree and use rsync with '--copy-links' to download the
This is the important part of my script:
r = get(api_key, galaxy_url)
# iterate over each data set in the library or history
for item in r:
# check if the name matchs the regular expression
if item['type'] == 'file' and match(regex, item['name']) is not None:
# if so go and get the details for the data set
item_details = get(api_key, galaxy_url + '/' +
# link to the data set real file name
# if this is a bam file, also sym link to the bam index
if item_details['data_type'] == 'bam':
item_details['name'] + '.bai')
A few advantages of this approach over simply "scpying". First you get
the same name you are using in galaxy and not a generic
dataset_000.dat. Also, you can copy multiple files matching a provided
regular expression in their name, with scp you would have to copy one
at a time.
If you are interested I can send you the whole script, which I'm sure
you will find several ways to break it, but in general it does its
On Wed, Jun 6, 2012 at 11:20 AM, Hans-Rudolf Hotz <h...@fmi.ch> wrote:
> Hi Jean-Francois
> My random guess: this is a web browser issue, struggling to download big
> files? - although, 3000 lines is not big
> If required, we use scp on the command line to get a copy of the dataset
> Regards, Hans
> On 06/06/2012 03:53 PM, Jean-Francois Payotte wrote:
>> Hi folks,
>> Some of our local Galaxy instance users seem to be experiencing some
>> strange behaviour lately. I searched the mailing-list archive but I
>> didn't found anything related, so I'd be interested to know if somebody
>> already had the same issue.
>> The problem is that sometimes, when people are trying to download their
>> datasets from their history, although the file seems to download
>> successfully, it appears that the downloaded file is incomplete (for
>> example a 3000 lines text file will show only maybe 2000 lines at the
>> first download, 1600 lines at the second download, and so on... and
>> eventually, the file will download completely.
>> This issue happened with more than one user and with different tools.
>> Does anybody ever had this kind of issue? Or does somebody would have an
>> idea of where to look to solve this problem?
>> Best regards,
>> Please keep all replies on the list by using "reply all"
>> in your mail client. To manage your subscriptions to this
>> and other Galaxy lists, please use the interface at:
> Please keep all replies on the list by using "reply all"
> in your mail client. To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at: