Re: [galaxy-dev] Tools that make datasets

2015-10-21 Thread Peter van Heusden
These names have meaning in CollectedDatasetMatch: designation means designation if that exists else name name means name dbkey means dbkey ext means ext visible means visible On 21 October 2015 at 08:44, Steve Cassidy wrote: > Thanks, yes that works, though I'm not

Re: [galaxy-dev] Tools that make datasets

2015-10-21 Thread Steve Cassidy
A brief writeup of my experiences: http://web.science.mq.edu.au/~cassidy/wordpress/2015/10/21/galaxy-tool-generating-datasets/ Steve On 21 October 2015 at 18:27, Peter van Heusden wrote: > These names have meaning in CollectedDatasetMatch: > > designation means designation

Re: [galaxy-dev] Tools that make datasets

2015-10-20 Thread Peter van Heusden
Just a quick check - did you refresh your history to confirm that the dataset *is* empty? We had the same thing at SANBI but it turns out that Galaxy creates an empty output collection and then only populates it sometime after job completion (this is a know UI bug). See:

Re: [galaxy-dev] Tools that make datasets

2015-10-20 Thread Steve Cassidy
Thanks Peter, I did see that proviso somewhere but no, refreshing doesn't help. That page was one of those that I referred to getting to this point. Steve On 20 October 2015 at 18:33, Peter van Heusden wrote: > Just a quick check - did you refresh your history to confirm

Re: [galaxy-dev] Tools that make datasets

2015-10-20 Thread Peter van Heusden
I suspect that the problem might be in the then. I'm not an export on this, but "__name_and_ext__" turns into the regexp r"(?P.*)\.(?P[^\.]+)?" in lib/galaxy/tools/parameters/output_collect.py, and is used by the DatasetCollector (line 358). This looks like it should match the filenames you're

Re: [galaxy-dev] Tools that make datasets

2015-10-20 Thread Steve Cassidy
Yes, I'm sure that's where the problem lies. Writing out to the current directory doesn't work. The files get written to 'job_working_directory/000/1/' but if I run the Upload File tool the result is placed in 'files/000/'. I think I need to work out where to write the files, I found some

[galaxy-dev] Tools that make datasets

2015-10-20 Thread Steve Cassidy
Hi all, I'm trying to understand how to write a tool that generates a dataset rather than a single output file. I've tried following all of the examples but I'm stuck, so I thought I would distil down the simplest example I could write and ask for help here. So here's my example:

Re: [galaxy-dev] Tools that make datasets

2015-10-20 Thread Steve Cassidy
Ah, thankyou, yes, I can now get results by using patterns to match the output. I used your example but prepend 'simple' to the filename and then searches for that with: this solves the problem for the sample script but not generally since in general I can't predict the filenames that will be

Re: [galaxy-dev] Tools that make datasets

2015-10-20 Thread Peter van Heusden
I poked around at your tool XML and the code a bit and the problem is directory="$job_name". Galaxy expects to collect files from the job's working directory - basically the current working directory the job runs in. The directory= argument doesn't have variables expanded as far as I can tell. In