Re: [galaxy-dev] . Output file to history (Sebastian Luna Valero)
I have opened a pull request that I believe could significantly ease the operations like this (for instance grabbing a directory of similar output files all of the same type based on a pattern specified by the tool instead of by Galaxy). There are lots of details of in the pull request body and some examples in the form of both functional and unit tests: https://bitbucket.org/galaxy/galaxy-central/pull-request/356/enhancements-for-runtime-discovered Let me know (either on the pull request or here) if you would like to see changes or if this is "good enough". These kinds of dynamic outputs are not quite as feature-full as I would like - but hopefully these limitations are addressed by forthcoming dataset collection work which I believe will be able to leverage the work in this pull request. -John On Sat, Mar 22, 2014 at 6:19 PM, Ross wrote: > Hi, Jun and Sebastian, > > This is not trivial, but it is already being done with some tools using the > method described at > https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Single_history_output_Html_file_with_links_to_any_number_of_output_files_and_images > if that can solve your problem. The Galaxy Html composite datatype can be > used to expose arbitrary outputs without creating a separate history item > for each one if that's what you want. The tool executable or the wrapper > must write valid and complete Html content to the path Galaxy provides as > the Html file for the tool and the sanitize_all_html switch must be turned > off in universe_wsgi.ini (which is a potential security problem for public > sites!) for the page to be pretty, but since the script itself is generating > all the output it is possible to write code to create a nicely laid out page > of links and images for the user. Composite datatypes (like Html) are > documented in the wiki and contributions to the wiki are always welcomed - > but at present, it's an advanced and not well documented possibility. > > If there really is demand for this functionality to be exposed, I have been > thinking about an autoHtml datatype that would do what the tool factory does > - look inside the job working directory at the end of the job and arrange > every file it finds there into a simple Html page of links. If > https://trello.com/c/vNQLZnSk gets enough upvotes, we'll put it into the > development pipeline? > > > > On Sun, Mar 23, 2014 at 4:43 AM, Jun Fan wrote: >> >> Hi Sebastian, >> >> I am not an expert in galaxy developing. So the following content is >> just my personal opinion. I do not think it is possible to meet you >> requirement. I reckon Galaxy searches the certain folder, defined in the >> universal_wsgi.ini by a parameter called #collect_outputs_from = >> new_file_path,job_working_directory. On top of that, a job id is required, >> which is the reason in the dynamic multiple output job id is part of the >> file name. If your script.py cannot follow the rules, Galaxy for sure cannot >> recognize the outputs. Normally in this kind of case, I write an additional >> script to execute the script.py with its original parameter and do the file >> manipulation afterwards. Of course this is not ideal, but I do nothing about >> python. It may be possible to write some complicated python code within the >> command tag of your wrapper file to do this. I am here also waiting for >> experts' opinions/comments. >> >> Best regards! >> Jun >> >> On 3/21/14 9:28 AM, Sebastian Luna Valero wrote: >> > >> > Dear All, >> > >> > I am trying to add a new tool in Galaxy and I have the following >> > problem. >> > >> > Let me explain a simplified example. Let's imagine that my script >> > works from CLI as follows: >> > >> > python script.py --input "input-file" --pattern "output-pattern" >> > >> > After processing "input-file", the script writes several output files >> > and their name is given according to "output-pattern". >> > >> > The number of output files depends on the content of "input-file". The >> > output files are written to the current working directory where >> > script.py is located. >> > >> > In the simplest scenario, I get only one output file in the working >> > directory. >> > >> > My problem is that I would like to see the output files in Galaxy's >> > history without modifying the CLI interface. >> > >> > To solve this problem, I have looked at the wiki page: >> > >> > https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#N >> > umber_of_Output_datasets_cannot_be_determined_until_tool_run >> > >> > and the email here: >> > >> > http://dev.list.galaxyproject.org/Multiple-output-files-do-not-appear- >> > in-history-td4660470.html >> > >> > However, I think that mine is a different scenario. I do not want to >> > add new output parameters in the CLI. I would like Galaxy to bring >> > these output files to my history without modifying the CLI options, is >> > that possible? >> > >> > Many thanks in advance for your help! >> > >> > Best regards, >> > Seba
Re: [galaxy-dev] . Output file to history (Sebastian Luna Valero)
Hi, Jun and Sebastian, This is not trivial, but it is already being done with some tools using the method described at https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#Single_history_output_Html_file_with_links_to_any_number_of_output_files_and_images if that can solve your problem. The Galaxy Html composite datatype can be used to expose arbitrary outputs without creating a separate history item for each one if that's what you want. The tool executable or the wrapper must write valid and complete Html content to the path Galaxy provides as the Html file for the tool and the sanitize_all_html switch must be turned off in universe_wsgi.ini (which is a potential security problem for public sites!) for the page to be pretty, but since the script itself is generating all the output it is possible to write code to create a nicely laid out page of links and images for the user. Composite datatypes (like Html) are documented in the wiki and contributions to the wiki are always welcomed - but at present, it's an advanced and not well documented possibility. If there really is demand for this functionality to be exposed, I have been thinking about an autoHtml datatype that would do what the tool factory does - look inside the job working directory at the end of the job and arrange every file it finds there into a simple Html page of links. If https://trello.com/c/vNQLZnSk gets enough upvotes, we'll put it into the development pipeline? On Sun, Mar 23, 2014 at 4:43 AM, Jun Fan wrote: > Hi Sebastian, > > I am not an expert in galaxy developing. So the following content is > just my personal opinion. I do not think it is possible to meet you > requirement. I reckon Galaxy searches the certain folder, defined in the > universal_wsgi.ini by a parameter called #collect_outputs_from = > new_file_path,job_working_directory. On top of that, a job id is required, > which is the reason in the dynamic multiple output job id is part of the > file name. If your script.py cannot follow the rules, Galaxy for sure > cannot recognize the outputs. Normally in this kind of case, I write an > additional script to execute the script.py with its original parameter and > do the file manipulation afterwards. Of course this is not ideal, but I do > nothing about python. It may be possible to write some complicated python > code within the command tag of your wrapper file to do this. I am here also > waiting for experts' opinions/comments. > > Best regards! > Jun > > On 3/21/14 9:28 AM, Sebastian Luna Valero wrote: > > > > Dear All, > > > > I am trying to add a new tool in Galaxy and I have the following problem. > > > > Let me explain a simplified example. Let's imagine that my script > > works from CLI as follows: > > > > python script.py --input "input-file" --pattern "output-pattern" > > > > After processing "input-file", the script writes several output files > > and their name is given according to "output-pattern". > > > > The number of output files depends on the content of "input-file". The > > output files are written to the current working directory where > > script.py is located. > > > > In the simplest scenario, I get only one output file in the working > > directory. > > > > My problem is that I would like to see the output files in Galaxy's > > history without modifying the CLI interface. > > > > To solve this problem, I have looked at the wiki page: > > > > https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#N > > umber_of_Output_datasets_cannot_be_determined_until_tool_run > > > > and the email here: > > > > http://dev.list.galaxyproject.org/Multiple-output-files-do-not-appear- > > in-history-td4660470.html > > > > However, I think that mine is a different scenario. I do not want to > > add new output parameters in the CLI. I would like Galaxy to bring > > these output files to my history without modifying the CLI options, is > > that possible? > > > > Many thanks in advance for your help! > > > > Best regards, > > Sebastian. > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] . Output file to history (Sebastian Luna Valero)
Hi Sebastian, I am not an expert in galaxy developing. So the following content is just my personal opinion. I do not think it is possible to meet you requirement. I reckon Galaxy searches the certain folder, defined in the universal_wsgi.ini by a parameter called #collect_outputs_from = new_file_path,job_working_directory. On top of that, a job id is required, which is the reason in the dynamic multiple output job id is part of the file name. If your script.py cannot follow the rules, Galaxy for sure cannot recognize the outputs. Normally in this kind of case, I write an additional script to execute the script.py with its original parameter and do the file manipulation afterwards. Of course this is not ideal, but I do nothing about python. It may be possible to write some complicated python code within the command tag of your wrapper file to do this. I am here also waiting for experts' opinions/comments. Best regards! Jun On 3/21/14 9:28 AM, Sebastian Luna Valero wrote: > > Dear All, > > I am trying to add a new tool in Galaxy and I have the following problem. > > Let me explain a simplified example. Let's imagine that my script > works from CLI as follows: > > python script.py --input "input-file" --pattern "output-pattern" > > After processing "input-file", the script writes several output files > and their name is given according to "output-pattern". > > The number of output files depends on the content of "input-file". The > output files are written to the current working directory where > script.py is located. > > In the simplest scenario, I get only one output file in the working > directory. > > My problem is that I would like to see the output files in Galaxy's > history without modifying the CLI interface. > > To solve this problem, I have looked at the wiki page: > > https://wiki.galaxyproject.org/Admin/Tools/Multiple%20Output%20Files#N > umber_of_Output_datasets_cannot_be_determined_until_tool_run > > and the email here: > > http://dev.list.galaxyproject.org/Multiple-output-files-do-not-appear- > in-history-td4660470.html > > However, I think that mine is a different scenario. I do not want to > add new output parameters in the CLI. I would like Galaxy to bring > these output files to my history without modifying the CLI options, is > that possible? > > Many thanks in advance for your help! > > Best regards, > Sebastian. ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Output file naming
On Fri, Oct 26, 2012 at 3:47 PM, Peter Cock wrote: > i.e. This isn't something each tool author should be doing, > but something general that should be in Galaxy itself. Yes, we would very much like a visual/graphical indicator of dataset ancestry, and possibly an alternative history view that was graph based. On the big list once the new History UI framework is done. (I imagine people will still want to customize labels though). -- jt ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming
On Fri, Oct 26, 2012 at 7:53 PM, Anthonius deBoer wrote: > and if you use something like ... > > I do this for all the tools in galaxy (bit of a pain, but worth it) so they > produce more intelligent names than "12: X on data 10" etc. > > Thon These kind of tricks are quite handy, but in general I'd like Galaxy to make the ancestry of each dataset more visible (I would say "history" but Galaxy already uses that for the collection of current datasets), both in the peek view, but also perhaps a detail view reusing some of the graph layout code used in workflows? i.e. This isn't something each tool author should be doing, but something general that should be in Galaxy itself. Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming
and if you use something like;label="#echo os.path.splitext ( str ( $input.name ) ) [ 0 ] #-BWA.bam"You can change "SomeFile.fastq" into "SomeFile-BWA.bam".This allows you to remove the extension (.fastq in this case), insert some extra text after the original name and set the right extension (Not needed for Galaxy, but we are all used to seeing those labels)...I do this for all the tools in galaxy (bit of a pain, but worth it) so they produce more intelligent names than "12: X on data 10" etc.ThonOn Oct 26, 2012, at 08:12 AM, Joachim Jacob wrote:Hi, You can use label="$input.name" Cheers, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 10/26/2012 04:38 PM, Darin wrote: > Hello, > > I'm new to Galaxy, and I've been asked to tweak some tools so that > output file names are related to input file names. For example, an > input file of myData.abc might lead to an output file named > myData_afterMyTool.xyz. > > I think the user is really just interested in the labels showing up in > the history panel, so that gives me some flexibility. > > In my development environment, I've changed one tool's xml file's > output tag to this: > > > > > > ... and that seems to work, but it's using an internal file name, > instead of the label for the input file. > > So - does anyone know if there's an easy way to get the label from the > input file into the output label, instead of the path/dataset_n.dat > file format for this name? If there's a better way to approach this, > I'd love any suggestions on that, too. > > Thanks, > > Darin > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming
Hi, You can use label="$input.name" Cheers, Joachim Joachim Jacob, PhD Rijvisschestraat 120, 9052 Zwijnaarde Tel: +32 9 244.66.34 Bioinformatics Training and Services (BITS) http://www.bits.vib.be @bitsatvib On 10/26/2012 04:38 PM, Darin wrote: Hello, I'm new to Galaxy, and I've been asked to tweak some tools so that output file names are related to input file names. For example, an input file of myData.abc might lead to an output file named myData_afterMyTool.xyz. I think the user is really just interested in the labels showing up in the history panel, so that gives me some flexibility. In my development environment, I've changed one tool's xml file's output tag to this: ... and that seems to work, but it's using an internal file name, instead of the label for the input file. So - does anyone know if there's an easy way to get the label from the input file into the output label, instead of the path/dataset_n.dat file format for this name? If there's a better way to approach this, I'd love any suggestions on that, too. Thanks, Darin ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] output file
change your tool to use explicitly named output folder and files bash mytool.sh $input $output1.extra_files_path $output1 $output2 where $output1.extra_files_path will be a folder (e.g. working dir) $output1, $output2, etc. are files that are to be brought into your history in galaxy, to be used by other tools or viewable by the user. mv files from working dir to these locations before exiting. files in the folder that are not moved to the $output1 location will be saved but not visible to the user also see compound datatypes (although they're usually not necessary). On Sun, May 6, 2012 at 2:56 AM, 張詩婷 wrote: > Hi, > > After execution, my tool created many files in a directory , how can I to > find this directory? > > command: bash mytool.sh $input > > > > > > > > > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ > ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file permissions different pbs:/// VS local:/// job runners
On Jul 5, 2012, at 9:21 PM, Rob Syme wrote: > Hi all. > > When I run the FastQC tool using the default local job runner, it produces a > folder of files with permission 664: > > -rw-rw-r-- 1 galaxy galaxy 17K Jul 5 17:29 duplication_levels.png > -rw-rw-r-- 1 galaxy galaxy 1.6K Jul 5 17:29 error.png > -rw-rw-r-- 1 galaxy galaxy 7.5K Jul 5 17:29 fastqc_data.txt > > > When I switch to using the(local) pbs queue, it produces a folder with > permissions 600: > > -rw--- 1 galaxy galaxy 15K Jul 6 09:10 duplication_levels.png > -rw--- 1 galaxy galaxy 1.6K Jul 6 09:10 error.png > -rw--- 1 galaxy galaxy 8.8K Jul 6 09:10 fastqc_data.txt > > > The final dat file (in database/files/000/dataset_xx.dat) is fine, it's just > the files in the corresponding folder (database/files/000/dataset_xx_files/*) > that have the incorrect permissions. > These more restrictive permissions mean that the files cannot be read by > apache, resulting a bunch of broken links when viewing the tool's output. > > Does anybody know why this might be happening or how to resolve it? Hi Rob, In your cluster user's .bash_profile, try setting `umask 022` to cause new files to be created with world-readable permissions. --nate > > Many thanks. > > -r > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > > http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming / dataset's label
Hi, Thanks for the tips and CC! Cheers, Tanguy ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming / dataset's label
On 04/13/2011 12:30 PM, Dave Walton wrote:
Is there anyway to access the value of fields that are part of the workflow. Taking the
"Input Dataset" example, that is an input file that goes into my next module. I'd like
to grab the name of that input file, and then append it with a new extension. So if the module is
grooming fastq, I'd like to grab my file name that is given to the fastq groomer, and append
".fastqsanger" to it, and have that be my output file name.
Is this possible?
Right now, no, none of those attributes are available through the
parameter replacement mechanism, it's currently just a simple text
substitution. I do like the idea of exposing certain attributes though,
that's something I can look into.
I suppose, for now, you could do something like
"${input_filename_stub}.fastqsanger" for your example step, combining
the parameter tag with some static text, and that would work. The
entire input doesn't have to get replaced.
-Dannon
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming / dataset's label
I get it now!!
I did my Rename Dataset/output_file with a variable ${input}.
I chose that because I thought that would give me the name of the input file to
that module, which I'm looking to carry on through out the run of my workflow
(though actually I'd like the file name minus the file extension, but that's
another problem).
When I did this, I ended up with a field "Input" at the top of my workflow form
when I went to run it. I typed a value in and sure enough, all my output files
now contained what I typed into the field.
So, this is certainly useful, but the question I have, and I believe it's what
Matthew is getting at too is:
Is there anyway to access the value of fields that are part of the workflow.
Taking the "Input Dataset" example, that is an input file that goes into my
next module. I'd like to grab the name of that input file, and then append it
with a new extension. So if the module is grooming fastq, I'd like to grab my
file name that is given to the fastq groomer, and append ".fastqsanger" to it,
and have that be my output file name.
Is this possible?
Thanks,
Dave
On Apr 13, 2011, at 11:20 AM, Dannon Baker wrote:
> Ahh, I understand your confusion now. The previous bit you pointed out
> was an error indeed, but not the one you were running into. The
> variable name should be whatever you want it to be, it doesn't have to
> map to an existing value like "Input Dataset", but there's an issue with
> the way the tags are handled for parameter replacement that causes a
> failure when the name contains spaces. Instead of ${Input Dataset}, try
> ${input_dataset}. I have a fix for this that should be available in the
> near future, as part of a larger set of changes that should make
> parameters much more accessible.
>
>
>
> On 04/13/2011 10:11 AM, Matthew Conte wrote:
>> Hi Dannon,
>>
>> Sorry, I guess I still don't quite get it. So I should keep the Input
>> Dataset name the default which is "Input Dataset". In the next subsequent
>> step in the workflow I then edit step actions Rename Dataset/output_file
>> with "${Input Dataset} groomed", for example. This gives me the same error
>> as before. It is still unclear to me what the actual variable name is for
>> the input dataset so that I can use it to rename output in later steps. Do
>> you have an example workflow that uses the input data set name to name the
>> output of downstream workflow steps that you could share with me as an
>> example so I can see the usage? Even a simple two step workflow would do.
>>
>> Thanks again,
>> Matt
>>
>> On Wed, Apr 13, 2011 at 6:09 AM, Dannon Baker wrote:
>>
>>> The input dataset steps do not use workflow parameters as they don't
>>> actually create output datasets, but rather act as more of a temporary
>>> handle. Remove the parameter tags from the input steps and just referring
>>> to those same parameter names in the subsequent steps you wanted to use them
>>> with should work and I think will allow you to do what you want.
>>>
>>> Thank you for pointing out the error though, this should definitely be more
>>> gracefully handled and supported with some help text in the editor, I'll see
>>> what I can do about that. Let me know if this doesn't get up and running
>>> and I'll do what I can to help!
>>>
>>> -Dannon
>>>
>>>
>>>
>>>
>>> On 04/12/2011 06:20 PM, Matthew Conte wrote:
>>>
I'd like to do the same thing that Tanguy described. I'm trying to figure
out the syntax to this within the workflow editor, but can't quite get it
right. In your example Dannon, would you just set ${dataset_name} as the
"Name" for your "Input dataset" to capture the input dataset name as a
variable? When I do that and then try to run the workflow I get the
following error:
Error -: 'dataset_name'
URL: http://localhost:8080/galaxy/workflow/run?id=f597429621d6eb2b
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
line 143 in __call__
app_iter = self.application(environ, start_response)
File '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/path-to-galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in
__call__
body = method( trans, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/controllers/workflow.py', line
1397 in run
incoming=kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 650
in fill_template
return self.fill_template_mako( filename, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 661
in fill_template_mako
return template.render( **d
Re: [galaxy-dev] Output file naming / dataset's label
Ahh, I understand your confusion now. The previous bit you pointed out
was an error indeed, but not the one you were running into. The
variable name should be whatever you want it to be, it doesn't have to
map to an existing value like "Input Dataset", but there's an issue with
the way the tags are handled for parameter replacement that causes a
failure when the name contains spaces. Instead of ${Input Dataset}, try
${input_dataset}. I have a fix for this that should be available in the
near future, as part of a larger set of changes that should make
parameters much more accessible.
On 04/13/2011 10:11 AM, Matthew Conte wrote:
Hi Dannon,
Sorry, I guess I still don't quite get it. So I should keep the Input
Dataset name the default which is "Input Dataset". In the next subsequent
step in the workflow I then edit step actions Rename Dataset/output_file
with "${Input Dataset} groomed", for example. This gives me the same error
as before. It is still unclear to me what the actual variable name is for
the input dataset so that I can use it to rename output in later steps. Do
you have an example workflow that uses the input data set name to name the
output of downstream workflow steps that you could share with me as an
example so I can see the usage? Even a simple two step workflow would do.
Thanks again,
Matt
On Wed, Apr 13, 2011 at 6:09 AM, Dannon Baker wrote:
The input dataset steps do not use workflow parameters as they don't
actually create output datasets, but rather act as more of a temporary
handle. Remove the parameter tags from the input steps and just referring
to those same parameter names in the subsequent steps you wanted to use them
with should work and I think will allow you to do what you want.
Thank you for pointing out the error though, this should definitely be more
gracefully handled and supported with some help text in the editor, I'll see
what I can do about that. Let me know if this doesn't get up and running
and I'll do what I can to help!
-Dannon
On 04/12/2011 06:20 PM, Matthew Conte wrote:
I'd like to do the same thing that Tanguy described. I'm trying to figure
out the syntax to this within the workflow editor, but can't quite get it
right. In your example Dannon, would you just set ${dataset_name} as the
"Name" for your "Input dataset" to capture the input dataset name as a
variable? When I do that and then try to run the workflow I get the
following error:
Error -: 'dataset_name'
URL: http://localhost:8080/galaxy/workflow/run?id=f597429621d6eb2b
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
line 143 in __call__
app_iter = self.application(environ, start_response)
File '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/path-to-galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in
__call__
body = method( trans, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/controllers/workflow.py', line
1397 in run
incoming=kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 650
in fill_template
return self.fill_template_mako( filename, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 661
in fill_template_mako
return template.render( **data )
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/template.py',
line
133 in render
return runtime._render(self, self.callable_, args, data)
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
line
364 in _render
_render_context(template, callable_, context, *args,
**_kwargs_for_callable(callable_, data))
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
line
381 in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
line
414 in _exec_template
callable_(context, *args, **kwargs)
File '/path-to-galaxy-dist/database/compiled_templates/base.mako.py',
line
40 in render_body
__M_writer(unicode(next.body()))
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
line
255 in
return lambda *args, **kwargs:callable_(self.context, *args, **kwargs)
File '/path-to-galaxy-dist/database/compiled_templates/workflow/
run.mako.py',
line 172 in render_body
pja_ss = pja_ss.replace(rematch, '%s' % (wf_parms[rematch[2:-1]],
rematch[2:-1], rematch[2:-1]))
KeyError: 'dataset_name'
Perhaps I'm missing something obvious?
Thanks,
Matt
\
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming / dataset's label
Hi Dannon,
Sorry, I guess I still don't quite get it. So I should keep the Input
Dataset name the default which is "Input Dataset". In the next subsequent
step in the workflow I then edit step actions Rename Dataset/output_file
with "${Input Dataset} groomed", for example. This gives me the same error
as before. It is still unclear to me what the actual variable name is for
the input dataset so that I can use it to rename output in later steps. Do
you have an example workflow that uses the input data set name to name the
output of downstream workflow steps that you could share with me as an
example so I can see the usage? Even a simple two step workflow would do.
Thanks again,
Matt
On Wed, Apr 13, 2011 at 6:09 AM, Dannon Baker wrote:
> The input dataset steps do not use workflow parameters as they don't
> actually create output datasets, but rather act as more of a temporary
> handle. Remove the parameter tags from the input steps and just referring
> to those same parameter names in the subsequent steps you wanted to use them
> with should work and I think will allow you to do what you want.
>
> Thank you for pointing out the error though, this should definitely be more
> gracefully handled and supported with some help text in the editor, I'll see
> what I can do about that. Let me know if this doesn't get up and running
> and I'll do what I can to help!
>
> -Dannon
>
>
>
>
> On 04/12/2011 06:20 PM, Matthew Conte wrote:
>
>> I'd like to do the same thing that Tanguy described. I'm trying to figure
>> out the syntax to this within the workflow editor, but can't quite get it
>> right. In your example Dannon, would you just set ${dataset_name} as the
>> "Name" for your "Input dataset" to capture the input dataset name as a
>> variable? When I do that and then try to run the workflow I get the
>> following error:
>>
>> Error -: 'dataset_name'
>> URL: http://localhost:8080/galaxy/workflow/run?id=f597429621d6eb2b
>> File
>>
>> '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
>> line 143 in __call__
>> app_iter = self.application(environ, start_response)
>> File '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
>> line 80 in __call__
>> return self.application(environ, start_response)
>> File
>> '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
>> line 632 in __call__
>> return self.application(environ, start_response)
>> File '/path-to-galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in
>> __call__
>> body = method( trans, **kwargs )
>> File '/path-to-galaxy-dist/lib/galaxy/web/controllers/workflow.py', line
>> 1397 in run
>> incoming=kwargs )
>> File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 650
>> in fill_template
>> return self.fill_template_mako( filename, **kwargs )
>> File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 661
>> in fill_template_mako
>> return template.render( **data )
>> File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/template.py',
>> line
>> 133 in render
>> return runtime._render(self, self.callable_, args, data)
>> File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
>> line
>> 364 in _render
>> _render_context(template, callable_, context, *args,
>> **_kwargs_for_callable(callable_, data))
>> File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
>> line
>> 381 in _render_context
>> _exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
>> File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
>> line
>> 414 in _exec_template
>> callable_(context, *args, **kwargs)
>> File '/path-to-galaxy-dist/database/compiled_templates/base.mako.py',
>> line
>> 40 in render_body
>> __M_writer(unicode(next.body()))
>> File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py',
>> line
>> 255 in
>> return lambda *args, **kwargs:callable_(self.context, *args, **kwargs)
>> File '/path-to-galaxy-dist/database/compiled_templates/workflow/
>> run.mako.py',
>> line 172 in render_body
>> pja_ss = pja_ss.replace(rematch, '> class="wfpspan wf_parm__%s pja_wfp">%s' % (wf_parms[rematch[2:-1]],
>> rematch[2:-1], rematch[2:-1]))
>> KeyError: 'dataset_name'
>>
>> Perhaps I'm missing something obvious?
>>
>> Thanks,
>> Matt
>>
> \
>
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming / dataset's label
The input dataset steps do not use workflow parameters as they don't
actually create output datasets, but rather act as more of a temporary
handle. Remove the parameter tags from the input steps and just
referring to those same parameter names in the subsequent steps you
wanted to use them with should work and I think will allow you to do
what you want.
Thank you for pointing out the error though, this should definitely be
more gracefully handled and supported with some help text in the editor,
I'll see what I can do about that. Let me know if this doesn't get up
and running and I'll do what I can to help!
-Dannon
On 04/12/2011 06:20 PM, Matthew Conte wrote:
I'd like to do the same thing that Tanguy described. I'm trying to figure
out the syntax to this within the workflow editor, but can't quite get it
right. In your example Dannon, would you just set ${dataset_name} as the
"Name" for your "Input dataset" to capture the input dataset name as a
variable? When I do that and then try to run the workflow I get the
following error:
Error -: 'dataset_name'
URL: http://localhost:8080/galaxy/workflow/run?id=f597429621d6eb2b
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
line 143 in __call__
app_iter = self.application(environ, start_response)
File '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/path-to-galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in
__call__
body = method( trans, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/controllers/workflow.py', line
1397 in run
incoming=kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 650
in fill_template
return self.fill_template_mako( filename, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 661
in fill_template_mako
return template.render( **data )
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/template.py', line
133 in render
return runtime._render(self, self.callable_, args, data)
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
364 in _render
_render_context(template, callable_, context, *args,
**_kwargs_for_callable(callable_, data))
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
381 in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
414 in _exec_template
callable_(context, *args, **kwargs)
File '/path-to-galaxy-dist/database/compiled_templates/base.mako.py', line
40 in render_body
__M_writer(unicode(next.body()))
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
255 in
return lambda *args, **kwargs:callable_(self.context, *args, **kwargs)
File '/path-to-galaxy-dist/database/compiled_templates/workflow/run.mako.py',
line 172 in render_body
pja_ss = pja_ss.replace(rematch, '%s' % (wf_parms[rematch[2:-1]],
rematch[2:-1], rematch[2:-1]))
KeyError: 'dataset_name'
Perhaps I'm missing something obvious?
Thanks,
Matt
\
___
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
Re: [galaxy-dev] Output file naming / dataset's label
I'd like to do the same thing that Tanguy described. I'm trying to figure
out the syntax to this within the workflow editor, but can't quite get it
right. In your example Dannon, would you just set ${dataset_name} as the
"Name" for your "Input dataset" to capture the input dataset name as a
variable? When I do that and then try to run the workflow I get the
following error:
Error - : 'dataset_name'
URL: http://localhost:8080/galaxy/workflow/run?id=f597429621d6eb2b
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/exceptions/errormiddleware.py',
line 143 in __call__
app_iter = self.application(environ, start_response)
File '/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/recursive.py',
line 80 in __call__
return self.application(environ, start_response)
File
'/path-to-galaxy-dist/eggs/Paste-1.6-py2.6.egg/paste/httpexceptions.py',
line 632 in __call__
return self.application(environ, start_response)
File '/path-to-galaxy-dist/lib/galaxy/web/framework/base.py', line 145 in
__call__
body = method( trans, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/controllers/workflow.py', line
1397 in run
incoming=kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 650
in fill_template
return self.fill_template_mako( filename, **kwargs )
File '/path-to-galaxy-dist/lib/galaxy/web/framework/__init__.py', line 661
in fill_template_mako
return template.render( **data )
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/template.py', line
133 in render
return runtime._render(self, self.callable_, args, data)
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
364 in _render
_render_context(template, callable_, context, *args,
**_kwargs_for_callable(callable_, data))
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
381 in _render_context
_exec_template(inherit, lclcontext, args=args, kwargs=kwargs)
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
414 in _exec_template
callable_(context, *args, **kwargs)
File '/path-to-galaxy-dist/database/compiled_templates/base.mako.py', line
40 in render_body
__M_writer(unicode(next.body()))
File '/path-to-galaxy-dist/eggs/Mako-0.2.5-py2.6.egg/mako/runtime.py', line
255 in
return lambda *args, **kwargs:callable_(self.context, *args, **kwargs)
File '/path-to-galaxy-dist/database/compiled_templates/workflow/run.mako.py',
line 172 in render_body
pja_ss = pja_ss.replace(rematch, '%s' % (wf_parms[rematch[2:-1]],
rematch[2:-1], rematch[2:-1]))
KeyError: 'dataset_name'
Perhaps I'm missing something obvious?
Thanks,
Matt
On Wed, Dec 8, 2010 at 9:38 AM, Dannon Baker wrote:
> Tanguy,
>
> My apologies for the delay in response. For now, the best way to do
> something like this would be to add rename dataset actions to each step
> where you want to modify the dataset name. You can do this within the
> workflow editor, in the right panel under "Edit Step Actions".
>
> A new feature we've been working on will allow you to use variables within
> the rename action, among other places. Depending on how often you reuse
> workflows, it could save you a lot of time renaming things. An example
> would look something like this, for a rename dataset action: "Filtering
> ${dataset_name} on col2 > ${threshold}", where ${dataset_name} and
> ${threshold} are variables that are set when you run a workflow. These
> variables get replaced anywhere they are seen in the workflow tool
> parameters and step actions. This feature is not available yet, but we do
> expect it to be ready soon.
>
> Thanks for using Galaxy,
>
> Dannon
>
> On Oct 28, 2010, at 4:40 AM, Tanguy Le Carrour wrote:
>
> > Dear Galaxy Team,
> >
> > I started using Galaxy few months ago. I'm now setting up my first "real"
> workflows.
> > I would like to keep the name of my input file through the various steps
> as dataset's label, maybe also modify it a bit, and finally give a "proper"
> name to the result file downloaded.
> >
> > A simple example:
> > 1) date-sample_name.gff3 --[step1]-> date-sample_name-step1.tsv
> > 2) date-sample_name-step1.tsv --[step2]->
> date-sample_name-step1-step2.txt
> > 3 download date-sample_name-step1-step2.txt
> >
> > I've seen that I can use the 'label' attribute in the tool config "data"
> tag to do so. But I don't know how to access the ${input.label} (?) and how
> to modify it.
> >
> > I doesn't stop me using my workflows, but it would definitely help my
> users not to get lost when they use them.
> >
> > Best regards,
> > Tanguy Le Carrour
> >
> > --
> > Tanguy LE CARROUR
> >
> > Functional Genomics Center Zurich
> > ETH Zurich / University of Zurich
> > Winterthurerstrasse 190
> > 8057 Zurich
> > Switzerland
> >
> >
> >
> > ___
> > galaxy-dev mailing list
> > galaxy-dev@lists.bx.psu.edu
> > http://lists.bx.psu.edu/listinfo/galaxy-dev
>
> ___
> galaxy-dev mailing
