Re: [galaxy-dev] Variable number of inputs in Workflow (or optional inputs)
Hi Jennifer, Thanks for the reply. However, my question was not about building tools. My question was about building a workflow (using the workflow editor with existing tools), as a user of galaxy. Using the workflow editor, when I create a workflow using a tool that allows a variable number of inputs, it seems that I have to define the number of inputs during workflow creation and not when I run the workflow. I was just asking if it would be feasible to have a variable number of inputs (only defined when a running instance of a workflow is created). It feels to me that the answer is no. Thanks again, Daniel On Mon 05 Mar 2012 05:56:16 PM WET, Jennifer Jackson wrote: Hello Daniel, Examining existing tools that have multiple/optional inputs can be a good way to see how this is done. One example is the tool: NGS: QC and manipulation - Manipulate FASTQ. The 'Manipulate FASTQ' tool source is here: http://bitbucket.org/galaxy/galaxy-central/src/e58a87c91bc4/tools/fastq/ fastq_manipulation.py fastq_manipulation.xml The tool tag set is defined in this wiki; http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax See repeat, conditional, and when for more details and examples. Questions about tools are best sent to the galaxy-...@bx.psu.edu mailing list (the galaxy-user list is primarily for questions about data/tools usages on the main public instance). I am going to forward your question over there so that the development community can add to my reply in case they have a simpler way of doing this or other advice. Best, Jen Galaxy team On 3/5/12 7:32 AM, Daniel Sobral wrote: Hi, I wanted to build a workflow where the first step would be a tool that can have a variable number of inputs (e.g. a series). It seems that I need to predefine apriori the number of inputs to give to the tool. Is there a way to define the number of inputs at runtime? E.g. a series-like input? Thanks, Daniel ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Variable number of inputs in Workflow (or optional inputs)
Hello Daniel, I had the same problem when I started to work with Galaxy workflows. My workaround is: 1) Modify the first tool of the workflow Make it work on a simple text file that contains a list of input file paths (one by line) instead of several files. 2) Create a galaxy tools that can create this list from a galaxy history 3) Build a workflow with only one input slot for the first tool (the list) Hope this helps :) Have a nice day, Aurélien Le 06/03/2012 10:01, Daniel Sobral a écrit : Hi Jennifer, Thanks for the reply. However, my question was not about building tools. My question was about building a workflow (using the workflow editor with existing tools), as a user of galaxy. Using the workflow editor, when I create a workflow using a tool that allows a variable number of inputs, it seems that I have to define the number of inputs during workflow creation and not when I run the workflow. I was just asking if it would be feasible to have a variable number of inputs (only defined when a running instance of a workflow is created). It feels to me that the answer is no. Thanks again, Daniel On Mon 05 Mar 2012 05:56:16 PM WET, Jennifer Jackson wrote: Hello Daniel, Examining existing tools that have multiple/optional inputs can be a good way to see how this is done. One example is the tool: NGS: QC and manipulation - Manipulate FASTQ. The 'Manipulate FASTQ' tool source is here: http://bitbucket.org/galaxy/galaxy-central/src/e58a87c91bc4/tools/fastq/ fastq_manipulation.py fastq_manipulation.xml The tool tag set is defined in this wiki; http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax Seerepeat,conditional, andwhen for more details and examples. Questions about tools are best sent to the galaxy-...@bx.psu.edu mailing list (the galaxy-user list is primarily for questions about data/tools usages on the main public instance). I am going to forward your question over there so that the development community can add to my reply in case they have a simpler way of doing this or other advice. Best, Jen Galaxy team On 3/5/12 7:32 AM, Daniel Sobral wrote: Hi, I wanted to build a workflow where the first step would be a tool that can have a variable number of inputs (e.g. a series). It seems that I need to predefine apriori the number of inputs to give to the tool. Is there a way to define the number of inputs at runtime? E.g. a series-like input? Thanks, Daniel ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Aurélien Bernard IE Bioprogrammeur - CNRS Université des sciences Montpellier II Institut des Sciences de l'Evolution Bâtiment 22 - CC 064 Place Eugène Bataillon 34095 Montpellier cedex 5 France Tel : 04 67 14 32 61 ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Variable number of inputs in Workflow (or optional inputs)
Hi Daniel, My apologies! I misunderstood what your question was about. You are correct, the required inputs are defined when the workflow is created. For tools that accept a variable number of inputs, when that tool is added to a workflow, the run-time parameters, including expected inputs, are defined. Did you have a particular tool in mind? We can ask Dannon for some suggestions/comments if you want to share your ideas about how this sort of processing would flow. I am glad you wrote back. Please send more details if you want and we can try to offer more (better!) help, Best, Jen Galaxy team On 3/6/12 1:01 AM, Daniel Sobral wrote: Hi Jennifer, Thanks for the reply. However, my question was not about building tools. My question was about building a workflow (using the workflow editor with existing tools), as a user of galaxy. Using the workflow editor, when I create a workflow using a tool that allows a variable number of inputs, it seems that I have to define the number of inputs during workflow creation and not when I run the workflow. I was just asking if it would be feasible to have a variable number of inputs (only defined when a running instance of a workflow is created). It feels to me that the answer is no. Thanks again, Daniel On Mon 05 Mar 2012 05:56:16 PM WET, Jennifer Jackson wrote: Hello Daniel, Examining existing tools that have multiple/optional inputs can be a good way to see how this is done. One example is the tool: NGS: QC and manipulation - Manipulate FASTQ. The 'Manipulate FASTQ' tool source is here: http://bitbucket.org/galaxy/galaxy-central/src/e58a87c91bc4/tools/fastq/ fastq_manipulation.py fastq_manipulation.xml The tool tag set is defined in this wiki; http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax Seerepeat,conditional, andwhen for more details and examples. Questions about tools are best sent to the galaxy-...@bx.psu.edu mailing list (the galaxy-user list is primarily for questions about data/tools usages on the main public instance). I am going to forward your question over there so that the development community can add to my reply in case they have a simpler way of doing this or other advice. Best, Jen Galaxy team On 3/5/12 7:32 AM, Daniel Sobral wrote: Hi, I wanted to build a workflow where the first step would be a tool that can have a variable number of inputs (e.g. a series). It seems that I need to predefine apriori the number of inputs to give to the tool. Is there a way to define the number of inputs at runtime? E.g. a series-like input? Thanks, Daniel ___ The Galaxy User list should be used for the discussion of Galaxy analysis and other features on the public server at usegalaxy.org. Please keep all replies on the list by using reply all in your mail client. For discussion of local Galaxy instances and the Galaxy source code, please use the Galaxy Development list: http://lists.bx.psu.edu/listinfo/galaxy-dev To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ -- Jennifer Jackson http://usegalaxy.org http://galaxyproject.org/wiki/Support ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Variable number of inputs in Workflow (or optional inputs)
Good Morning, We are also interested in this capability. To give a concrete example, we sometimes receive multiple sequence runs 1* for the same sample. We would like to be able to process each run of the sample through BWA and then merge them together, post process it a bit, and then send the merged bam file through the rest of our standard pipeline. Ideally this would be automated. Aurélien, I am interested in your workaround this might get us part of the way there, as we could concatenate the fastq files together and run BWA once. Would you be willing to share some of your custom tools for us to iterate on? But ideally, for performance, we would run the bwa steps in parallel. Is the galaxy team looking into these types of features or do other people have custom solutions they are using? Thanks so much, Ann ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
Re: [galaxy-dev] Variable number of inputs in Workflow (or optional inputs)
This seems to fall within the recent discussion of map/reduce operators for workflows, e.g. breaking large jobs up for embarrassingly parallel tasks, then merging them back at some later point. Dannon mentioned some basic functionality does exist within Galaxy to do this, but it's at an early stage of development. Peter Cock did get some things to work with it. See this thread: http://thread.gmane.org/gmane.science.biology.galaxy.devel/4502/focus=4502 chris On Mar 6, 2012, at 9:56 AM, Ann Black wrote: Good Morning, We are also interested in this capability. To give a concrete example, we sometimes receive multiple sequence runs 1…* for the same sample. We would like to be able to process each run of the sample through BWA and then merge them together, post process it a bit, and then send the merged bam file through the rest of our standard pipeline. Ideally this would be automated. Aurélien, I am interested in your workaround – this might get us part of the way there, as we could concatenate the fastq files together and run BWA once. Would you be willing to share some of your custom tools for us to iterate on? But ideally, for performance, we would run the bwa steps in parallel. Is the galaxy team looking into these types of features or do other people have custom solutions they are using? Thanks so much, Ann ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ ___ Please keep all replies on the list by using reply all in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/
