Re: [galaxy-dev] Using Mesos to Enable distributed computing under Galaxy?

2013-10-28 Thread Kyle Ellrott
I don't think implementation will be very difficult. The bigger question is this a technology people are open to? The nearest competitor is YARN ( http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/YARN.html). Mesos seems a bit more geared toward general purpose usage (with several

Re: [galaxy-dev] Using Mesos to Enable distributed computing under Galaxy?

2013-10-28 Thread Kyle Ellrott
You probably are a good person to get an opinion from. My plan isn't to write new frameworks, but rather use existing libraries that can communicate with Mesos to setup their parallel environments. But for Swift, you would probably want to write a new framework. Just looking at Swift, I imagine

Re: [galaxy-dev] Why are Custom Builds disabled when

2013-10-28 Thread Jeremy Goecks
Why are Custom Builds only enabled if you're *not* using remote_user ? Is this because they're related to user preferences (as per the code in lib/galaxy/webapps/galaxy/controllers/user.py)? This may be an artifact of a time when user preferences were immutable. Do Custom Builds and

[galaxy-dev] What to do with toolshed tools that get stuck Installing

2013-10-28 Thread Ira Cooke
Hi All, The short version of my problem is that I often seem to find myself in a situation where my tool gets stuck in the Installing phase ... and I don't know how to get feedback on it to see where it is failing. What I guess I'm after is just the stdout of the tool installation .. but

Re: [galaxy-dev] What to do with toolshed tools that get stuck Installing

2013-10-28 Thread Björn Grüning
Hi Ira, I can reproduce that error here, will try to study it tomorrow more deeply. For the meantime can you try to remove the following lines? action type=set_environment_for_install repository name=package_ruby_2_0 owner=bgruening package name=ruby

Re: [galaxy-dev] What to do with toolshed tools that get stuck Installing

2013-10-28 Thread Ira Cooke
Hi Bjoern, Thanks ... I thought I had to do that in order to make the GALAXY_RUBY_HOME variable available ... nice that it's not necessary. I've updated the tool on the test toolshed and have rerun my test but unfortunately the problem is still there. Cheers Ira On 29/10/2013, at 10:06 AM,

[galaxy-dev] tardis job splitter

2013-10-28 Thread McCulloch, Alan
dear all, There have been a few posts lately about doing distributed computing via Galaxy - i.e. job splitters etc - below a contribution of some ideas we have developed and applied in our work, where we have arranged for some Galaxy tools to execute in parallel on our cluster. We have