Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP configuration

2015-09-24 Thread Nicola Soranzo

Hi Carlos!

No, you don't need Apache. You can also use multiple authentication 
mechanisms at the same time, e.g. LDAP for users of a particular domain 
and the usual Galaxy database auth for the others. The configuration 
file you need to modify is config/auth_conf.xml, the options are 
documented on the .sample file (look at 
https://github.com/galaxyproject/galaxy/blob/dev/config/auth_conf.xml.sample 
for the most updated version).


Cheers,
Nicola

On 23/09/15 20:10, Carlos Lijeron wrote:

Good day Nicola,

Thank you for the advise on LDAP authentication.  Did  you need to run 
Galaxy on Apache to enable this, or did you simply change some 
configuration files to enable this.  I think we’ll be able to write 
 some scripts using SLURM commands to pull data about resource 
utilization, but my main concern is whether we need Apache to enable 
LDAP authentication of galaxy users.


Thanks again !


Carlos

From: Nicola Soranzo <nicola.sora...@gmail.com 
<mailto:nicola.sora...@gmail.com>> on behalf of Nicola Soranzo 
<nsora...@tiscali.it <mailto:nsora...@tiscali.it>>

Date: Tuesday, September 22, 2015 at 6:59 AM
To: Carlos Lijeron <clije...@hunter.cuny.edu 
<mailto:clije...@hunter.cuny.edu>>, 
"galaxy-dev@lists.galaxyproject.org 
<mailto:galaxy-dev@lists.galaxyproject.org>" 
<galaxy-dev@lists.galaxyproject.org 
<mailto:galaxy-dev@lists.galaxyproject.org>>
Subject: Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration


Hi Carlos,
we are using Active Directory LDAP for user authentication, which 
works pretty well, but only the Galaxy user to submit jobs to the LSF 
cluster queue, so I can't help with the resource tracking.


Cheers,
Nicola

On 21/09/15 20:19, Carlos Lijeron wrote:

Everyone,

We are setting up Galaxy to work with our cluster and SLURM as the 
work manager.   The cluster itself authenticates to our local Active 
Directory, so I’m wondering if the best way to track resource 
utilization of Galaxy users on the cluster is to also have Galaxy 
authenticate to the same Active Directory LDAP.


Is anyone on this list using the same configuration and tracking 
resource utilization from Galaxy users submitting jobs to the cluster 
nodes?   Please advise.


Thank you all !


Carlos.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP configuration

2015-09-23 Thread Carlos Lijeron
Good day Nicola,

Thank you for the advise on LDAP authentication.  Did  you need to run Galaxy 
on Apache to enable this, or did you simply change some configuration files to 
enable this.  I think we’ll be able to write  some scripts using SLURM commands 
to pull data about resource utilization, but my main concern is whether we need 
Apache to enable LDAP authentication of galaxy users.

Thanks again !


Carlos

From: Nicola Soranzo 
<nicola.sora...@gmail.com<mailto:nicola.sora...@gmail.com>> on behalf of Nicola 
Soranzo <nsora...@tiscali.it<mailto:nsora...@tiscali.it>>
Date: Tuesday, September 22, 2015 at 6:59 AM
To: Carlos Lijeron <clije...@hunter.cuny.edu<mailto:clije...@hunter.cuny.edu>>, 
"galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>" 
<galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>>
Subject: Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration

Hi Carlos,
we are using Active Directory LDAP for user authentication, which works pretty 
well, but only the Galaxy user to submit jobs to the LSF cluster queue, so I 
can't help with the resource tracking.

Cheers,
Nicola

On 21/09/15 20:19, Carlos Lijeron wrote:
Everyone,

We are setting up Galaxy to work with our cluster and SLURM as the work 
manager.   The cluster itself authenticates to our local Active Directory, so 
I’m wondering if the best way to track resource utilization of Galaxy users on 
the cluster is to also have Galaxy authenticate to the same Active Directory 
LDAP.

Is anyone on this list using the same configuration and tracking resource 
utilization from Galaxy users submitting jobs to the cluster nodes?   Please 
advise.

Thank you all !


Carlos.



___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP configuration

2015-09-23 Thread Carlos Lijeron
David,

Thank you for the valuable feedback.  I will look into the galaxy-pulsar server 
app and determine how we can implement it to connect our galaxy instance to the 
working nodes.  Perhaps I’ll direct  a few relevant questions while we get on 
board with the implementation, if you don’t mind.If you have any notes or 
lessons learned that you could share that would be very helpful to us. However, 
this idea seems already like a good solution for us.I’ll keep all my notes 
and setup a step by step instructions after we are done.


Carlos.


From: David Trudgian 
<david.trudg...@utsouthwestern.edu<mailto:david.trudg...@utsouthwestern.edu>>
Date: Tuesday, September 22, 2015 at 11:53 AM
To: Nicola Soranzo <nsora...@tiscali.it<mailto:nsora...@tiscali.it>>, Carlos 
Lijeron <clije...@hunter.cuny.edu<mailto:clije...@hunter.cuny.edu>>, 
"galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>" 
<galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>>
Subject: RE: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration

Hi Carlos,

We aren’t using Active Directory here – but we do use an OpenLDAP directory 
which our cluster authenticates against, as does Galaxy. In order to track 
cluster usage for users running Galaxy jobs we use galaxy-pulsar between Galaxy 
itself and our SLURM cluster. Pulsar is configured to submit all jobs as a real 
user on the cluster, via SLURM-DRMAA. This means that all of the Galaxy usage 
on the cluster appears in our SLURM accounting database just like any other job 
would.  The complexity here is file ownership. Pulsar has to copy all input 
files into a staging directory, and change ownership to the real user for the 
job to run on the cluster. It is/was(?) a little complex to setup, as there are 
more parts involved then a typical Galaxy install, but it works great for us 
here.

When I was getting this setup John Chilton mentioned adding a PulsarEmbedded 
runner into Galaxy at some point. Not sure whether this has/is happening, but 
it’d make these situations easier:

https://trello.com/c/4YwVZBtq/1865-embedded-pulsar-job-runner


DT

--
David Trudgian Ph.D.
Computational Scientist, BioHPC
UT Southwestern Medical Center
Dallas, TX 75390-9039
Tel: (214) 648-4833

From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] On Behalf 
Of Nicola Soranzo
Sent: Tuesday, September 22, 2015 5:59 AM
To: Carlos Lijeron <clije...@hunter.cuny.edu<mailto:clije...@hunter.cuny.edu>>; 
galaxy-dev@lists.galaxyproject.org<mailto:galaxy-dev@lists.galaxyproject.org>
Subject: Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration

Hi Carlos,
we are using Active Directory LDAP for user authentication, which works pretty 
well, but only the Galaxy user to submit jobs to the LSF cluster queue, so I 
can't help with the resource tracking.

Cheers,
Nicola
On 21/09/15 20:19, Carlos Lijeron wrote:
Everyone,

We are setting up Galaxy to work with our cluster and SLURM as the work 
manager.   The cluster itself authenticates to our local Active Directory, so 
I’m wondering if the best way to track resource utilization of Galaxy users on 
the cluster is to also have Galaxy authenticate to the same Active Directory 
LDAP.

Is anyone on this list using the same configuration and tracking resource 
utilization from Galaxy users submitting jobs to the cluster nodes?   Please 
advise.

Thank you all !


Carlos.




___

Please keep all replies on the list by using "reply all"

in your mail client.  To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

  https://lists.galaxyproject.org/



To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/




UT Southwestern


Medical Center



The future of medicine, today.

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP configuration

2015-09-22 Thread David Trudgian
Hi Carlos,

We aren’t using Active Directory here – but we do use an OpenLDAP directory 
which our cluster authenticates against, as does Galaxy. In order to track 
cluster usage for users running Galaxy jobs we use galaxy-pulsar between Galaxy 
itself and our SLURM cluster. Pulsar is configured to submit all jobs as a real 
user on the cluster, via SLURM-DRMAA. This means that all of the Galaxy usage 
on the cluster appears in our SLURM accounting database just like any other job 
would.  The complexity here is file ownership. Pulsar has to copy all input 
files into a staging directory, and change ownership to the real user for the 
job to run on the cluster. It is/was(?) a little complex to setup, as there are 
more parts involved then a typical Galaxy install, but it works great for us 
here.

When I was getting this setup John Chilton mentioned adding a PulsarEmbedded 
runner into Galaxy at some point. Not sure whether this has/is happening, but 
it’d make these situations easier:

https://trello.com/c/4YwVZBtq/1865-embedded-pulsar-job-runner


DT

--
David Trudgian Ph.D.
Computational Scientist, BioHPC
UT Southwestern Medical Center
Dallas, TX 75390-9039
Tel: (214) 648-4833

From: galaxy-dev [mailto:galaxy-dev-boun...@lists.galaxyproject.org] On Behalf 
Of Nicola Soranzo
Sent: Tuesday, September 22, 2015 5:59 AM
To: Carlos Lijeron <clije...@hunter.cuny.edu>; 
galaxy-dev@lists.galaxyproject.org
Subject: Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP 
configuration

Hi Carlos,
we are using Active Directory LDAP for user authentication, which works pretty 
well, but only the Galaxy user to submit jobs to the LSF cluster queue, so I 
can't help with the resource tracking.

Cheers,
Nicola
On 21/09/15 20:19, Carlos Lijeron wrote:
Everyone,

We are setting up Galaxy to work with our cluster and SLURM as the work 
manager.   The cluster itself authenticates to our local Active Directory, so 
I’m wondering if the best way to track resource utilization of Galaxy users on 
the cluster is to also have Galaxy authenticate to the same Active Directory 
LDAP.

Is anyone on this list using the same configuration and tracking resource 
utilization from Galaxy users submitting jobs to the cluster nodes?   Please 
advise.

Thank you all !


Carlos.




___

Please keep all replies on the list by using "reply all"

in your mail client.  To manage your subscriptions to this

and other Galaxy lists, please use the interface at:

  https://lists.galaxyproject.org/



To search Galaxy mailing lists use the unified search at:

  http://galaxyproject.org/search/mailinglists/




UT Southwestern


Medical Center



The future of medicine, today.

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy on a Cluster -- Active Directory LDAP configuration

2015-09-22 Thread Nicola Soranzo

Hi Carlos,
we are using Active Directory LDAP for user authentication, which works 
pretty well, but only the Galaxy user to submit jobs to the LSF cluster 
queue, so I can't help with the resource tracking.


Cheers,
Nicola

On 21/09/15 20:19, Carlos Lijeron wrote:

Everyone,

We are setting up Galaxy to work with our cluster and SLURM as the 
work manager.   The cluster itself authenticates to our local Active 
Directory, so I’m wondering if the best way to track resource 
utilization of Galaxy users on the cluster is to also have Galaxy 
authenticate to the same Active Directory LDAP.


Is anyone on this list using the same configuration and tracking 
resource utilization from Galaxy users submitting jobs to the cluster 
nodes?   Please advise.


Thank you all !


Carlos.


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/


___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/