Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
On Thu, Oct 24, 2013 at 5:53 PM, James Taylor wrote: >> So far $GALAXY_SLOTS seems to be working nicely for me. >> >> However, I am wondering if it would be possible to use it inside >> the section? Is that run at the time of job creation >> on the Galaxy server (where determining the number of threads >> may be hard) or as part of job execution (e.g. on the cluster, >> when $GALAXY_SLOTS would be known). > > Unfortunately, the former, and there is no easy way to change this. Ah. That was what I suspected :( > One could do some simple variable substitution on the config file when > the job runs though (sed would do the job here, not pretty but it > works). Yeah - I was thinking about something like that as a work around, although since I have a wrapper Python script anyway I can do the edit there: https://github.com/peterjc/pico_galaxy/tree/master/tools/mira4 Thanks for such a prompt answer, Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
> So far $GALAXY_SLOTS seems to be working nicely for me. > > However, I am wondering if it would be possible to use it inside > the section? Is that run at the time of job creation > on the Galaxy server (where determining the number of threads > may be hard) or as part of job execution (e.g. on the cluster, > when $GALAXY_SLOTS would be known). Unfortunately, the former, and there is no easy way to change this. One could do some simple variable substitution on the config file when the job runs though (sed would do the job here, not pretty but it works). ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
On Thu, Oct 17, 2013 at 8:37 AM, Bjoern Gruening wrote: > Am Samstag, den 12.10.2013, 19:42 +0100 schrieb Peter Cock: >> On Thu, Aug 1, 2013 at 10:27 AM, Nicola Soranzo wrote: >> > Il 2013-07-30 17:18 Peter Cock ha scritto: >> >> >> >> Hello all, >> >> >> >> Re: >> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html >> >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html >> >> >> >> Something I raised during the GCC2013, and we talked about >> >> via Twitter as well was a Galaxy environment variable for use >> >> within Tool Wrappers setting the number of threads/CPUs to >> >> use. >> >> >> >> The idea is that you can configure a default value, and then >> >> override this per runner or per tool etc. >> > >> > >> > Thanks Peter for pushing this idea, I totally support this proposal. >> > In the mean time, I've been using for my tools the solution by >> > Jim Johnson for its CD-HIT wrapper: >> > >> > http://toolshed.g2.bx.psu.edu/view/jjohnson/cdhit >> > >> > But this requires the system administrator to modify both the tool >> > env.sh and job_conf.xml to be in sync. >> > >> >> Is there an open Trello card for this? >> > >> > A Trello card would be useful indeed. >> > >> > Nicola >> >> Better than a Trello card, we now have a pull request from John: >> https://bitbucket.org/galaxy/galaxy-central/pull-request/236/job-runner-enhancements-galaxy_slots/diff > > And thanks to John its merged! Time for testing and migrating our > tools :) So far $GALAXY_SLOTS seems to be working nicely for me. However, I am wondering if it would be possible to use it inside the section? Is that run at the time of job creation on the Galaxy server (where determining the number of threads may be hard) or as part of job execution (e.g. on the cluster, when $GALAXY_SLOTS would be known). I have tried using \$GALAXY_SLOTS but it remains in the file generated by as $GALAXY_SLOTS rather than being substituted. Thanks, Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
Am Samstag, den 12.10.2013, 19:42 +0100 schrieb Peter Cock: > On Thu, Aug 1, 2013 at 10:27 AM, Nicola Soranzo wrote: > > Il 2013-07-30 17:18 Peter Cock ha scritto: > >> > >> Hello all, > >> > >> Re: > >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html > >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html > >> > >> Something I raised during the GCC2013, and we talked about > >> via Twitter as well was a Galaxy environment variable for use > >> within Tool Wrappers setting the number of threads/CPUs to > >> use. > >> > >> The idea is that you can configure a default value, and then > >> override this per runner or per tool etc. > > > > > > Thanks Peter for pushing this idea, I totally support this proposal. > > In the mean time, I've been using for my tools the solution by > > Jim Johnson for its CD-HIT wrapper: > > > > http://toolshed.g2.bx.psu.edu/view/jjohnson/cdhit > > > > But this requires the system administrator to modify both the tool > > env.sh and job_conf.xml to be in sync. > > > >> Is there an open Trello card for this? > > > > A Trello card would be useful indeed. > > > > Nicola > > Better than a Trello card, we now have a pull request from John: > https://bitbucket.org/galaxy/galaxy-central/pull-request/236/job-runner-enhancements-galaxy_slots/diff And thanks to John its merged! Time for testing and migrating our tools :) > Peter > ___ > Please keep all replies on the list by using "reply all" > in your mail client. To manage your subscriptions to this > and other Galaxy lists, please use the interface at: > http://lists.bx.psu.edu/ > > To search Galaxy mailing lists use the unified search at: > http://galaxyproject.org/search/mailinglists/ ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
On Thu, Aug 1, 2013 at 10:27 AM, Nicola Soranzo wrote: > Il 2013-07-30 17:18 Peter Cock ha scritto: >> >> Hello all, >> >> Re: >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html >> http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html >> >> Something I raised during the GCC2013, and we talked about >> via Twitter as well was a Galaxy environment variable for use >> within Tool Wrappers setting the number of threads/CPUs to >> use. >> >> The idea is that you can configure a default value, and then >> override this per runner or per tool etc. > > > Thanks Peter for pushing this idea, I totally support this proposal. > In the mean time, I've been using for my tools the solution by > Jim Johnson for its CD-HIT wrapper: > > http://toolshed.g2.bx.psu.edu/view/jjohnson/cdhit > > But this requires the system administrator to modify both the tool > env.sh and job_conf.xml to be in sync. > >> Is there an open Trello card for this? > > A Trello card would be useful indeed. > > Nicola Better than a Trello card, we now have a pull request from John: https://bitbucket.org/galaxy/galaxy-central/pull-request/236/job-runner-enhancements-galaxy_slots/diff Peter ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
Re: [galaxy-dev] Defining $GALAXY_SLOTS for use in tool wrappers
Il 2013-07-30 17:18 Peter Cock ha scritto: Hello all, Re: http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-June/010153.html http://lists.bx.psu.edu/pipermail/galaxy-dev/2012-October/011557.html Something I raised during the GCC2013, and we talked about via Twitter as well was a Galaxy environment variable for use within Tool Wrappers setting the number of threads/CPUs to use. The idea is that you can configure a default value, and then override this per runner or per tool etc. Thanks Peter for pushing this idea, I totally support this proposal. In the mean time, I've been using for my tools the solution by Jim Johnson for its CD-HIT wrapper: http://toolshed.g2.bx.psu.edu/view/jjohnson/cdhit But this requires the system administrator to modify both the tool env.sh and job_conf.xml to be in sync. Is there an open Trello card for this? A Trello card would be useful indeed. Nicola ___ Please keep all replies on the list by using "reply all" in your mail client. To manage your subscriptions to this and other Galaxy lists, please use the interface at: http://lists.bx.psu.edu/ To search Galaxy mailing lists use the unified search at: http://galaxyproject.org/search/mailinglists/
