[galaxy-dev] Galaxy configuration

[Clinton Chee]

Dear Galaxy devs / support,

I am trying to understand the following snippet from job_conf.xml (I 
want to use drmaa v1)



galaxy@merc
 



Based on someone's configuration above, I am trying to customize to my 
system.


I'm trying to understand where/how the parameters are associated?
- tags?
- destination in param id? is this the queue?  I also checked the DRMAA 
specification but cannot find any "destination" as a keyword.
- galaxy@merc?   is merc pointing to the hostname, and galaxy the queue 
name? (I know the instution's cluster is called "merc", but I don't know 
if the "merc" in the tag is for convenience or is being read as the 
hostname?



I've looked through:
https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer
https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
https://wiki.galaxyproject.org/Admin/Config/Jobs - describes job_conf.xml

but cannot make sense, eg. for "tags" is says "Tags to which this 
destination belongs."

Eg. "tags="longwalltime,bigcluster""


Do you have more detailed documention on configuration?

Thanks
Clinton



--

*PBS WorksTM named #1 software product!*
HPCwire Reader’s Choice Awards 2014
*
For more information on Altair’s award-winning HPC
workload management suite, please visit pbsworks.com 
 *


Clinton Chee (PhD)
PBS Application Engineer
Altair - Innovation Intelligence
Mob: 61 (0)402 058 983
Email: c...@altair.com
Skype: live:clinton.chee
Twitter: @xtechnotes 
!function(d,s,id){var 
js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore 
(js,fjs);}}(document, 
'script', 'twitter-wjs');
PBSWorks: Facebook | Google+ 
| Linkedin 
 | Twitter 
| YouTube 


Australia Toll Free: 1800 174 396
Urgent / Emergency: pbs-supp...@india.altair.com




___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Galaxy configuration

[Nate Coraor]

Hi Clinton,

On Mon, Sep 7, 2015 at 2:23 AM, Clinton Chee  wrote:

> Dear Galaxy devs / support,
>
> I am trying to understand the following snippet from job_conf.xml (I want
> to use drmaa v1)
> 
> 
> galaxy@merc
>  
> 
>
>
> Based on someone's configuration above, I am trying to customize to my
> system.
>
> I'm trying to understand where/how the parameters are associated?
> - tags?
>

Tags are identifiers used to map tools to destinations. You probably do not
need to use them, but if you wanted to have a tool (or the default
destination) send jobs to more than one destination (selected at random),
tags are used for that. They can also be used for setting job concurrency
limits.


> - destination in param id? is this the queue?  I also checked the DRMAA
> specification but cannot find any "destination" as a keyword.
>

I am not sure where this example came from but I don't believe it would
work - the "destination" param works with the "pbs" runner, but the pbs
runner only works with Torque, as far as I know. The drmaa runner expects
params that conform to the DRMAA specification, the most common of which is
`nativeSpecification`. If you wanted to set the destination for PBS Pro
using the drmaa runner, the param would be:

-q queue@server_name

- galaxy@merc?   is merc pointing to the hostname, and galaxy the queue
> name? (I know the instution's cluster is called "merc", but I don't know if
> the "merc" in the tag is for convenience or is being read as the hostname?
>

That format is a PBS destination identifier, so `galaxy` is the queue on
the PBS server `merc`.

Hope this helps, please let us know if this doesn't clear things up.

--nate


>
> I've looked through:
> https://wiki.galaxyproject.org/Admin/Config/Performance/ProductionServer
> https://wiki.galaxyproject.org/Admin/Config/Performance/Cluster
> https://wiki.galaxyproject.org/Admin/Config/Jobs - describes job_conf.xml
>
> but cannot make sense, eg. for "tags" is says "Tags to which this
> destination belongs."
> Eg. "tags="longwalltime,bigcluster""
>
>
> Do you have more detailed documention on configuration?
>
> Thanks
> Clinton
>
>
> --
> --
> *PBS WorksTM named #1 software product!*
> HPCwire Reader’s Choice Awards 2014
>
>
> * For more information on Altair’s award-winning HPC workload management
> suite, please visit pbsworks.com  *
> --
> Clinton Chee (PhD)
> PBS Application Engineer
> Altair - Innovation Intelligence
> Mob: 61 (0)402 058 983
> Email: c...@altair.com
> Skype: live:clinton.chee
> Twitter: @xtechnotes !function(d,s,id){var
> js,fjs=d.getElementsByTagName(s)[0],p=/^
> http:/.test(d.location)?'http':'https';if(!d.getElementById(id))
> {js=d.createElement(s);js.id=id;js.src=p+'://
> platform.twitter.com/widgets.js';fjs.parentNode.insertBefore
> (js,fjs);}}(document,
> 'script', 'twitter-wjs');
> PBSWorks: Facebook | Google+
> | Linkedin
>  | Twitter
> | YouTube
> 
> Australia Toll Free: 1800 174 396
> Urgent / Emergency: pbs-supp...@india.altair.com
>
>
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] External User Authenticaion

[Ryan G]

Yes, I have a test server I'm going to check this one.  thanks for the
link, that's perfect...I'll add some debugging code in here to see what's
going on.

On Tue, Sep 8, 2015 at 1:46 PM, Dannon Baker  wrote:

> Do you have a way to verify the "HTTP_MAIL" header is actually being
> passed through your proxy server?
>
> The problem is that Galaxy still doesn't think it's receiving the expected
> headers, so there isn't a good way that it can tell you more about what
> might be going on.  If you're able to tweak Galaxy (using a test server)
> and add a few logging statements the code, this would be good places to
> check what's going on (print the `environ` dictionary associated with that
> request, along with self.remote_user_header to see what Galaxy is actually
> trying to use):
>
>
> https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/web/framework/middleware/remoteuser.py#L49
>
> -Dannon
>
> On Thu, Sep 3, 2015 at 1:51 PM, Ryan G 
> wrote:
>
>> It turns out our authentication system passes a header 'HTTP_MAIL' which
>> contains the users email address.  In galaxy.ini, I have
>>
>> use_remote_user = True
>> remote_user_header = HTTP_MAIL
>>
>> After restarting,Galaxy still gives the same error.
>>
>> On Mon, Aug 31, 2015 at 3:44 PM, Dannon Baker 
>> wrote:
>>
>>> Hi Ryan,
>>>
>>> It may be that Galaxy is looking for a different remote user header than
>>> your proxy is setting.  I believe by default we look for HTTP_REMOTE_USER,
>>> but this is configurable in galaxy.ini (so, you could set yours to
>>> HTTP_USER there).  Let me know if this doesn't sort it out for you and we
>>> can dig deeper!
>>>
>>> -Dannon
>>>
>>> On Mon, Aug 31, 2015 at 3:42 PM, Ryan G 
>>> wrote:
>>>
 Hi all - I'm trying to use external user authentication with Galaxy.
 The external authentication passes to Galaxy the username with the mail
 domain at HTTP_USER.

 In galaxy.ini, I enable:
 use_remote_user = True


 When I try to access Galaxy, I get the message:
 Galaxy is configured to authenticate users via an external method (such
 as HTTP authentication in Apache), but a username was not provided by the
 upstream (proxy) server. This is generally due to a misconfiguration in the
 upstream server.

 But nothing in paster.log indicating what the error is.

 How do I track this down?



 ___
 Please keep all replies on the list by using "reply all"
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
   https://lists.galaxyproject.org/

 To search Galaxy mailing lists use the unified search at:
   http://galaxyproject.org/search/mailinglists/

>>>
>>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] External User Authenticaion

[Dannon Baker]

Do you have a way to verify the "HTTP_MAIL" header is actually being passed
through your proxy server?

The problem is that Galaxy still doesn't think it's receiving the expected
headers, so there isn't a good way that it can tell you more about what
might be going on.  If you're able to tweak Galaxy (using a test server)
and add a few logging statements the code, this would be good places to
check what's going on (print the `environ` dictionary associated with that
request, along with self.remote_user_header to see what Galaxy is actually
trying to use):

https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/web/framework/middleware/remoteuser.py#L49

-Dannon

On Thu, Sep 3, 2015 at 1:51 PM, Ryan G  wrote:

> It turns out our authentication system passes a header 'HTTP_MAIL' which
> contains the users email address.  In galaxy.ini, I have
>
> use_remote_user = True
> remote_user_header = HTTP_MAIL
>
> After restarting,Galaxy still gives the same error.
>
> On Mon, Aug 31, 2015 at 3:44 PM, Dannon Baker 
> wrote:
>
>> Hi Ryan,
>>
>> It may be that Galaxy is looking for a different remote user header than
>> your proxy is setting.  I believe by default we look for HTTP_REMOTE_USER,
>> but this is configurable in galaxy.ini (so, you could set yours to
>> HTTP_USER there).  Let me know if this doesn't sort it out for you and we
>> can dig deeper!
>>
>> -Dannon
>>
>> On Mon, Aug 31, 2015 at 3:42 PM, Ryan G 
>> wrote:
>>
>>> Hi all - I'm trying to use external user authentication with Galaxy.
>>> The external authentication passes to Galaxy the username with the mail
>>> domain at HTTP_USER.
>>>
>>> In galaxy.ini, I enable:
>>> use_remote_user = True
>>>
>>>
>>> When I try to access Galaxy, I get the message:
>>> Galaxy is configured to authenticate users via an external method (such
>>> as HTTP authentication in Apache), but a username was not provided by the
>>> upstream (proxy) server. This is generally due to a misconfiguration in the
>>> upstream server.
>>>
>>> But nothing in paster.log indicating what the error is.
>>>
>>> How do I track this down?
>>>
>>>
>>>
>>> ___
>>> Please keep all replies on the list by using "reply all"
>>> in your mail client.  To manage your subscriptions to this
>>> and other Galaxy lists, please use the interface at:
>>>   https://lists.galaxyproject.org/
>>>
>>> To search Galaxy mailing lists use the unified search at:
>>>   http://galaxyproject.org/search/mailinglists/
>>>
>>
>>
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] SFTP configuration with ProFTPD

[Jose Juan Almagro Armenteros]

Hello everyone,

I am a bit stuck trying to set up a SFTP on my local Galaxy. I already have
an FTP server as described in the Galaxy wiki and in this post "
http://galacticengineer.blogspot.co.uk/2015/02/ftp-upload-to-galaxy-using-proftpd-and.html;,
which works correctly with the user authentication.

The problem is that when I switched to SFTP (using some posts that
described how to do this on ProFTPD with mod_sftp.c), the user
authentication stopped working. I don't really know what specific
modifications should I do to the proftpd.conf file, which is the same as
the one in the wiki, to make it works.

I would really appreciate if you have any idea about how to change from FTP
to SFTP in Galaxy or if any of you have a config file working for SFTP.

Thank you and best regards,

Jose
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] SFTP configuration with ProFTPD

[Nate Coraor]

On Tue, Sep 8, 2015 at 2:05 PM, Jose Juan Almagro Armenteros <
jjalma...@gmail.com> wrote:

> Hello everyone,
>
> I am a bit stuck trying to set up a SFTP on my local Galaxy. I already
> have an FTP server as described in the Galaxy wiki and in this post "
> http://galacticengineer.blogspot.co.uk/2015/02/ftp-upload-to-galaxy-using-proftpd-and.html;,
> which works correctly with the user authentication.
>
> The problem is that when I switched to SFTP (using some posts that
> described how to do this on ProFTPD with mod_sftp.c), the user
> authentication stopped working. I don't really know what specific
> modifications should I do to the proftpd.conf file, which is the same as
> the one in the wiki, to make it works.
>
> I would really appreciate if you have any idea about how to change from
> FTP to SFTP in Galaxy or if any of you have a config file working for SFTP.
>

Hi Jose,

Are you certain that when you're testing SFTP, you are using ProFTPD's SFTP
server, and not an sftp service (i.e. via OpenSSH) already running on the
system? One important distinction: SFTP and FTPS are not the same thing.

--nate


>
> Thank you and best regards,
>
> Jose
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/