David,

Thank you for the valuable feedback.  I will look into the galaxy-pulsar server 
app and determine how we can implement it to connect our galaxy instance to the 
working nodes.  Perhaps I’ll direct  a few relevant questions while we get on 
board with the implementation, if you don’t mind.    If you have any notes or 
lessons learned that you could share that would be very helpful to us. However, 
this idea seems already like a good solution for us.    I’ll keep all my notes 
and setup a step by step instructions after we are done.


Carlos.


From: David Trudgian 
<david.trudg...@utsouthwestern.edu<mailto:david.trudg...@utsouthwestern.edu>>
Date: Tuesday, September 22, 2015 at 11:53 AM
To: Nicola Soranzo <nsora...@tiscali.it<mailto:nsora...@tiscali.it>>, Carlos 
Lijeron <clije...@hunter.cuny.edu<mailto:clije...@hunter.cuny.edu>>, 
"galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>" 
<galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>>
Subject: RE: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration

Hi Carlos,

We aren’t using Active Directory here – but we do use an OpenLDAP directory 
which our cluster authenticates against, as does Galaxy. In order to track 
cluster usage for users running Galaxy jobs we use galaxy-pulsar between Galaxy 
itself and our SLURM cluster. Pulsar is configured to submit all jobs as a real 
user on the cluster, via SLURM-DRMAA. This means that all of the Galaxy usage 
on the cluster appears in our SLURM accounting database just like any other job 
would.  The complexity here is file ownership. Pulsar has to copy all input 
files into a staging directory, and change ownership to the real user for the 
job to run on the cluster. It is/was(?) a little complex to setup, as there are 
more parts involved then a typical Galaxy install, but it works great for us 
here.

When I was getting this setup John Chilton mentioned adding a PulsarEmbedded 
runner into Galaxy at some point. Not sure whether this has/is happening, but 
it’d make these situations easier:

https://trello.com/c/4YwVZBtq/1865-embedded-pulsar-job-runner


DT

--
David Trudgian Ph.D.
Computational Scientist, BioHPC
UT Southwestern Medical Center
Dallas, TX 75390-9039
Tel: (214) 648-4833

From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] On Behalf 
Of Nicola Soranzo
Sent: Tuesday, September 22, 2015 5:59 AM
To: Carlos Lijeron <clije...@hunter.cuny.edu<mailto:clije...@hunter.cuny.edu>>; 
galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>
Subject: Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration

Hi Carlos,
we are using Active Directory LDAP for user authentication, which works pretty 
well, but only the Galaxy user to submit jobs to the LSF cluster queue, so I 
can't help with the resource tracking.

Cheers,
Nicola
On 21/09/15 20:19, Carlos Lijeron wrote:
Everyone,

We are setting up Galaxy to work with our cluster and SLURM as the work 
manager.   The cluster itself authenticates to our local Active Directory, so 
I’m wondering if the best way to track resource utilization of Galaxy users on 
the cluster is to also have Galaxy authenticate to the same Active Directory 
LDAP.

Is anyone on this list using the same configuration and tracking resource 
utilization from Galaxy users submitting jobs to the cluster nodes?   Please 
advise.

Thank you all !


Carlos.




___________________________________________________________

Please keep all replies on the list by using "reply all"

in your mail client.  To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

  https://lists.galaxyproject.org/



To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/


________________________________

UT Southwestern


Medical Center



The future of medicine, today.

___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Reply via email to