Re: [galaxy-dev] CloudMan with HTCondor and Slurm & mounting pathes

[Enis Afgan]

Changing the default job manager is not interchangeable in an automated
fashion. SGE is currently the main job manager that Galaxy has been
configured to be used with. Slurm will replace SGE in the next Galaxy
CloudMan release. HTCondor has been added as an experimental module but due
to the lack of interest from the community (ie, no interest until now), it
hasn't seen any development and I'm really not even sure if it's working as
expected any more. Relevant documentation is available here
https://wiki.galaxyproject.org/CloudMan/HTCondor so if you can try this out
first and see what happens; we can then go from there.

Having said that, if you install any of the desired job managers manually,
Galaxy can be configured to use that one by setting appropriate
configuration option in */mnt/galaxy/galaxy-app/universe_wsgi.ini*. See
this page for more details on configuring Galaxy:
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster


Regarding the use of an S3 bucket as a data source - the intent (largely
mandated by the technical characteristics of an object store) is for this
to be used as a source of data rather than the default data store for
Galaxy. With that, it is advisable to use it as a source location for
importing shared datasets and ultimately storing the resulting ones. What
this means from within Galaxy is that you could create a data library with
the mounted S3 bucket being a data source and then utilize it for the
analyses. Take a look at slides #25-28 in this presentation on how to do
this:
http://www.slideshare.net/afgane/data-analysis-with-galaxy-on-the-cloud
If you would still like to use the S3 bucket as a comprehensive data store,
this can be done by setting *file_path* in
*/mnt/galaxy/galaxy-app/universe_wsgi.ini* to the desired path.

Hope this clarifies some questions. Let us know what you decide to do and
how things develop,
Enis

On Tue, Oct 14, 2014 at 6:35 AM, Abdularahman Azab  wrote:

>  Dear all,
>
>
>  I've installed a galaxy cloud cluster using usergalaxy.org/cloudlaunch
>
>
>  It runs fine, but I need to know how to let the cluster run on THCondor
> or slurm instead of SGE. I can see that Condor is already installed on the
> galaxy-master node since the condor_status command is there. But how can I
> switch the default galaxy job-runner between SGE, HTCondor, and slurm
>
>
>  Also regarding the mounting option of a e.g. s3 bucket on the master
> node. It works fine and it can be accessed from the command line. But how
> to let galaxy see this mounted path for e.g, reading input files and
> storing resulted history elements in it?
>
>
>  Thank you,
>
>  Yours sincerely,
> Abdulrahman Azab
>
>  Head engineer, ELIXIR.NO / The Genomic HyperBrowser team
>  Department of Informatics, University of Oslo, Boks 1072 Blindern,
> NO-0316 OSLO, Norway
>  Email: a...@ifi.uio.no , Cell-phone: +47
> 46797339
>  
>  Senior Lecturer in Computer Engineering
>  Faculty of Engineering, University of Mansoura, 35516-Mansoura, Egypt
>  Email: abdulrahman.a...@mans.edu.eg
> 
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   http://lists.bx.psu.edu/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] CloudMan with HTCondor and Slurm & mounting pathes

[Abdularahman Azab]

Dear all,


I've installed a galaxy cloud cluster using usergalaxy.org/cloudlaunch


It runs fine, but I need to know how to let the cluster run on THCondor or 
slurm instead of SGE. I can see that Condor is already installed on the 
galaxy-master node since the condor_status command is there. But how can I 
switch the default galaxy job-runner between SGE, HTCondor, and slurm


Also regarding the mounting option of a e.g. s3 bucket on the master node. It 
works fine and it can be accessed from the command line. But how to let galaxy 
see this mounted path for e.g, reading input files and storing resulted history 
elements in it?


Thank you,

Yours sincerely,
Abdulrahman Azab

Head engineer, ELIXIR.NO / The Genomic HyperBrowser team
Department of Informatics, University of Oslo, Boks 1072 Blindern, NO-0316 
OSLO, Norway
Email: a...@ifi.uio.no, Cell-phone: +47 46797339

Senior Lecturer in Computer Engineering
Faculty of Engineering, University of Mansoura, 35516-Mansoura, Egypt
Email: abdulrahman.a...@mans.edu.eg

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  http://lists.bx.psu.edu/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/