We are looking at integrating some tools into Galaxy for a domain where many of the existing programs are Windows only. My desire is to write tool files and have the certain jobs run remotely on a Windows host (our Galaxy servers are Linux). It wasn't clear how to do this with the existing job runners since they all seem to require a shared file system and absolute file paths that are the same on the Galaxy server as they are on the remote execution hosts.

I developed a new job runner that allows for the remote execution of jobs that also rewrites paths and stages files as part of the job execution. There is a server application that needs to be running on the remote machine (*nix or Windows). This application is likely easier to setup/configure than SGE/PBS/etc... because it is a simple Python paste application, with just a few configuration options. There may be valid use cases for this approach for non-Windows applications as well - if you would like a lighter weight server component on the remote system, if shared file systems are impossible/impractical, etc....

The path rewrites are not going to cover every possible way an absolute path can sneak into the execution, so this is only useful for a subset of tools. But it does handle normal file inputs and outputs, config files, and scripts in the tool definition directory. I welcome input/advice on this approach or the implementation. If there is any way I can improve the code base to the point where it can be included in galaxy-central I would be more than happy to make the desired changes. If not, it is out there for people to use if they have a similar use case.

The server application as well as a patch to add the runner/client to a galaxy-central installation can be found at:


Thanks for your time,

John Chilton
Software Developer
University of Minnesota Supercomputing Institute
Office: 612-625-0917
Cell: 612-226-9223
E-Mail: chil...@msi.umn.edu
galaxy-dev mailing list

Reply via email to