Hello Galaxy People,
I have put together some batch workflow enhancements that I think
could start to address some of the use cases mentioned at the Galaxy
conference. This pull request
(https://bitbucket.org/galaxy/galaxy-central/pull-request/75/enhanced-batch-workflow-mode)
implements the backend for allowing multiple inputs to be selected on
multiple different params when running a workflow (currently Galaxy
only allows one input parameter to be batched). The UI piece has not
really been addressed but I was hoping to start a conversation on
that.
The backend currently has two processing "modes". I will illustrate
them with examples I heard about at the conference.
One thing I heard mentioned is batching processing paired FASTQ files
is difficult, and there were some hacks mentioned to address that. If
you pull in the above changeset, remove this line from run.mako:
$('.multiinput').addClass('disabled'); and ensure that this element is
set to "matched" in the form:
<input type="hidden" name="multiple_input_mode" value="matched" />
Then you can select multiple inputs on any number of parameters and
they will be matched up (based on the order they appear in the
history) and the workflow will be run for each match. Right know the
UI won't ensure that the same number of items is selected on each so
that piece should be added.
Someone else had mentioned doing sort of all versus all workflows (in
the context of microarray data). This is what I called product mode.
Again if you remove the JavaScript line mentioned above and ensure
that this element is set to product in the form:
<input type="hidden" name="multiple_input_mode" value="product" />
Then you can select multiple inputs on any number of parameters and
the workflow will be run on each combination of the inputs.
To illustrate the difference - if user selected 5 inputs for one param
and 5 inputs for another in matched mode that will be run the workflow
5 times matching each input and in product mode it will run the
workflow 25 times with every combination of input pairs. If user
selects 6 inputs for one param and 4 for another, in product mode 24
workflows will run and in matched mode the submission will fail.
It should be straight forward to pull in the changeset and hack up
your Galaxy to do one or the other. I guess though we should allow
both and let the user decide :). I was hoping for some ideas on the UI
front, what the best way to communicate the differences, can we do
this without cluttering up the UI, should there be a separate advanced
workflow run mode, how do other workflow applications handle this
distinction, etc....
One thought I had was allowing a choice between these two modes on a
per workflow basis is a nice step forward from where we are now, but
maybe another good step forward would be to allow the decision on a
per parameter basis. What I was thinking was maybe when you select the
little icon for multiple inputs, a broken link chain icon would appear
next to the multi-input icon indicating "product" mode would be
enabled for that input (i.e. the inputs are not linked), and then if
the user clicked the little broken chain on some parameters and those
chains would close (and maybe glow) and those parameters would become
"linked" (or matched).
Then you could mix and match, for instance you could run n*2 fastq
files paired off each against m different parameter files.
Mixing and matching would take a little more work, but if that makes
the UI more straight forward it might be worth it (I would be happy to
implement these extensions). Though I am pretty bad at UI so I assume
I am fairly off base on this link idea.
Thanks,
-John
------------------------------------------------
John Chilton
Senior Software Developer
University of Minnesota Supercomputing Institute
Office: 612-625-0917
Cell: 612-226-9223
Bitbucket: https://bitbucket.org/jmchilton
Github: https://github.com/jmchilton
Web: http://jmchilton.net
___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client. To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
http://lists.bx.psu.edu/
