date:20120119

Re: [galaxy-dev] Error download composite dataset from history

2012-01-19 Thread Louise-Amélie Schmitt


Hi Ross

Thanks a million, I hg-pull'ed it solved the problem!! Please push it to 
galaxy-dist? ;)


I ran in another little bug though, but nothing serious, you might even 
have already spotted it by now:
In lib/galaxy/web/framework/__init__.py the line 873 made the whole 
thing crash due to a wrong arguments number, so I changed it from:

self._ensure_logged_in_user( environ )
to:
self._ensure_logged_in_user( environ, session_cookie )
and it works fine now!

Best,
L-A


Le 19/01/2012 02:13, Ross a écrit :

Hi, L-A

Thanks for reporting this - I just pushed a fix in galaxy-central
6556:ab1a5ab063ce
Please let us know if that resolves the problem?


2012/1/19 Louise-Amélie Schmittlouise-amelie.schm...@embl.de:

Hello!

I'm running into an error when trying to download a composite dataset from
the history with the floppy disk icon. Here is the error message in the
logs:


galaxy.web.controllers.dataset ERROR 2012-01-18 16:32:10,324 Unable to
remove temporary library download archive and directory
Traceback (most recent call last):
  File /g/funcgen/galaxy-dev/lib/galaxy/web/controllers/dataset.py, line
285, in archive_composite_dataset
os.rmdir( tmpd )
OSError: [Errno 39] Directory not empty: './database/tmp/tmpkRv28I'


Then I tried replacing the os.rmdir by a shutil.rmtree to see if it would
solve the issue, and here is what I got:


galaxy.web.controllers.dataset ERROR 2012-01-18 16:34:30,239 Unable to
remove temporary library download archive and directory
Traceback (most recent call last):
  File /g/funcgen/galaxy-dev/lib/galaxy/web/controllers/dataset.py, line
286, in archive_composite_dataset
shutil.rmtree( tmpd )
  File
/g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/shutil.py,
line 221, in rmtree
onerror(os.remove, fullname, sys.exc_info())
  File
/g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/shutil.py,
line 219, in rmtree
os.remove(fullname)
OSError: [Errno 16] Device or resource busy:
'./database/tmp/tmpwpsShD/.nfs0289ccea0138'


But when I check the contents of the direcory, there is no hidden
.nfs-something file.

Does anyone have an idea why this could occur? Both our testing instance and
our production instance have the same issue, the first being on
galaxy-central and the second on galaxy-dist.

Thanks,
L-A

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/





___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] problems with location of filter bar in trackster

2012-01-19 Thread Hans-Rudolf Hotz


Hi Jeremy

Thank you very much for your help. I will look into your suggestions.


Regards, Hans

On 01/18/2012 10:08 PM, Jeremy Goecks wrote:


it works fine on the public server


Good, this narrows down the problem to installing the -central script on your 
instance b/c main's copy of the script is very similar to -central's.


what do you mean by packed Trackster script


All JavaScript files are compressed in size--comments/whitespace removed and 
variables renamed--to minimize their transfer time. These are available in 
/static/scripts/packed

If you have directives in your proxy server config to use the packed scripts, 
you'll want to update the packed script from -central as well.

The other thing to consider is browser caching: you may need to clear your 
browser cache to load the new Trackster script.

Good luck,
J.

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] Error download composite dataset from history

2012-01-19 Thread Ross

Hi, L-A,

2012/1/19 Louise-Amélie Schmitt louise-amelie.schm...@embl.de:
 Hi Ross

 Thanks a million, I hg-pull'ed it solved the problem!! Please push it to
 galaxy-dist? ;)

Glad to hear it works - thanks for reporting back.

 I ran in another little bug though, but nothing serious, you might even have
 already spotted it by now:
 In lib/galaxy/web/framework/__init__.py the line 873 made the whole thing
 crash due to a wrong arguments number, so I changed it from:
            self._ensure_logged_in_user( environ )
 to:
            self._ensure_logged_in_user( environ, session_cookie )
 and it works fine now!

No, I hadn't encountered that - AFAIK, the function definition and
call are correct in the current tip of galaxy-central @
https://bitbucket.org/galaxy/galaxy-central/src/ab1a5ab063ce/lib/galaxy/web/framework/__init__.py
The tip of galaxy-dist on bitbucket currently looks consistent -
neither definition nor call have the session_cookie parameter - so I'm
not sure where the error you saw at line 873 comes from - in
https://bitbucket.org/galaxy/galaxy-dist/src/b258de1e6cea/lib/galaxy/web/framework/__init__.py
the call is at line 860...perhaps you were working with an older
revision or one that's been merged with central at some stage?


 Best,
 L-A


 Le 19/01/2012 02:13, Ross a écrit :

 Hi, L-A

 Thanks for reporting this - I just pushed a fix in galaxy-central
 6556:ab1a5ab063ce
 Please let us know if that resolves the problem?


 2012/1/19 Louise-Amélie Schmittlouise-amelie.schm...@embl.de:

 Hello!

 I'm running into an error when trying to download a composite dataset
 from
 the history with the floppy disk icon. Here is the error message in the
 logs:


 galaxy.web.controllers.dataset ERROR 2012-01-18 16:32:10,324 Unable to
 remove temporary library download archive and directory
 Traceback (most recent call last):
  File /g/funcgen/galaxy-dev/lib/galaxy/web/controllers/dataset.py, line
 285, in archive_composite_dataset
    os.rmdir( tmpd )
 OSError: [Errno 39] Directory not empty: './database/tmp/tmpkRv28I'


 Then I tried replacing the os.rmdir by a shutil.rmtree to see if it would
 solve the issue, and here is what I got:


 galaxy.web.controllers.dataset ERROR 2012-01-18 16:34:30,239 Unable to
 remove temporary library download archive and directory
 Traceback (most recent call last):
  File /g/funcgen/galaxy-dev/lib/galaxy/web/controllers/dataset.py, line
 286, in archive_composite_dataset
    shutil.rmtree( tmpd )
  File

 /g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/shutil.py,
 line 221, in rmtree
    onerror(os.remove, fullname, sys.exc_info())
  File

 /g/steinmetz/collaboration/software/CentOS5/opt/Python-2.6.5/lib/python2.6/shutil.py,
 line 219, in rmtree
    os.remove(fullname)
 OSError: [Errno 16] Device or resource busy:
 './database/tmp/tmpwpsShD/.nfs0289ccea0138'


 But when I check the contents of the direcory, there is no hidden
 .nfs-something file.

 Does anyone have an idea why this could occur? Both our testing instance
 and
 our production instance have the same issue, the first being on
 galaxy-central and the second on galaxy-dist.

 Thanks,
 L-A

 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/







-- 
Ross Lazarus MBBS MPH;
Associate Professor, Harvard Medical School;
Head, Medical Bioinformatics, BakerIDI; Tel: +61 385321444;

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] LDAP authentification

2012-01-19 Thread Sarah Maman


Hello,

I'm trying to install galaxy on a apache server,and i have the same 
error message than David HOOVER when i tried to generate the 
authentification files. Thanks to David Hoover who gives me some helpful 
explanation.


When i read the thread on gmod, it seems,that the problem is solved 
(http://gmod.827538.n3.nabble.com/trouble-with-authentication-through-Apache-td1809681.html 
http://gmod.827538.n3.nabble.com/trouble-with-authentication-through-Apache-td1809681.html) 
but I did not find the solution in my case.


I use LDAP authentification and Galaxy is implemented on a VM.

Could you please explain me how to solve it.  I really need help.

I modified universe file ( in galaxy-dist) and i created .htaccess file 
in galaxy-dist/ :

* in universe_wsgi.ini:

use_remote_user = True
remote_user_maildomain = example.org http://example.org

* in .htaccess file in galaxy-dist/ Directory :

  1 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_1
 # Define Galaxy as a valid Proxy
  2 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_2
 Proxy http://localhost:8080
  3 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_3
 Order deny,allow
  4 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_4
 Allow from all
  5 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_5
 /Proxy
  6 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_6
 #!highlight apache
  7 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_7
 Location /
  8 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_8
 AuthType Basic
  9 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_9
 AuthBasicProvider *ldap*
 10 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_10
 Auth*LDAP*URL 
*ldap*://server:389/ou=People,dc=example,dc=org?uid?sub?(objectClass=person)
 11 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_11
 Authz*LDAP*Authoritative off
 12 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_12
 Require valid-user
 13 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_13
 # Set the REMOTE_USER header to the contents of the *LDAP* query response's 
uid attribute
 14 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_14
 RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
 15 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%29#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_15
 /Location

with location, AuthBasicProvider and proxy competed

But i have the error :

Access to Galaxy is denied 

Galaxy is configured to authenticate users via an external method (such 
as HTTP authentication in Apache), but a username was not provided by 
the upstream (proxy) server. This is generally due to a misconfiguration 
in the upstream server. 


Thank you, and best regards

Sarah Maman


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Langhorst, Brad

Hi Sarah:

I don't know what kind of LDAP you're talking to... I'm talking to Active
Directory and this configuration works in that situation.

Order allow,deny
allow from all

AuthType Basic
AuthName NEB Credentials
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPBindDN ccaloo...@neb.com
AuthLDAPBindPassword password
AuthLDAPURL
ldap://ldap.domain.com:389/dc=domain,dc=com?sAMAccountName
require valid-user

RewriteCond %{IS_SUBREQ} ^false$
RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
RequestHeader set REMOTE_USER
%{AUTHENTICATE_sAMAccountName}e


-- 
Brad Langhorst
New England Biolabs
langho...@neb.com

 





On 1/19/12 8:59 AM, Sarah Maman sarah.ma...@toulouse.inra.fr wrote:

Hello,

I'm trying to install galaxy on a apache server,and i have the same
error message than David HOOVER when i tried to generate the
authentification files. Thanks to David Hoover who gives me some helpful
explanation.

When i read the thread on gmod, it seems,that the problem is solved
(http://gmod.827538.n3.nabble.com/trouble-with-authentication-through-Apac
he-td1809681.html 
http://gmod.827538.n3.nabble.com/trouble-with-authentication-through-Apac
he-td1809681.html)
but I did not find the solution in my case.

I use LDAP authentification and Galaxy is implemented on a VM.

Could you please explain me how to solve it.  I really need help.

I modified universe file ( in galaxy-dist) and i created .htaccess file
in galaxy-dist/ :
* in universe_wsgi.ini:

use_remote_user = True
remote_user_maildomain = example.org http://example.org

* in .htaccess file in galaxy-dist/ Directory :

   1 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_1 # Define Galaxy as a
valid Proxy
   2 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_2 Proxy
http://localhost:8080
   3 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_3 Order deny,allow
   4 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_4 Allow from all
   5 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_5 /Proxy
   6 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_6 #!highlight apache
   7 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_7 Location /
   8 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_8 AuthType Basic
   9 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_9 AuthBasicProvider
*ldap*
  10 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_10 Auth*LDAP*URL
*ldap*://server:389/ou=People,dc=example,dc=org?uid?sub?(objectClass=pers
on)
  11 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_11
Authz*LDAP*Authoritative off
  12 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_12 Require valid-user
  13 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_13 # Set the
REMOTE_USER header to the contents of the *LDAP* query response's uid
attribute
  14 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_14 RequestHeader set
REMOTE_USER %{AUTHENTICATE_uid}e
  15 
http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_15 /Location

with location, AuthBasicProvider and proxy competed

But i have the error :

Access to Galaxy is denied

Galaxy is configured to authenticate users via an external method (such
as HTTP authentication in Apache), but a username was not provided by
the upstream (proxy) server. This is generally due to a misconfiguration
in the upstream server.

Thank you, and best regards

Sarah Maman


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/


___
Please keep all replies on the list

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Fields, Christopher J

We also ran into this. Our local server configuration was case-sensitive and 
the LDAP uid key was converted to upper-case, so we used:

   RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e

which fixed the problem.  We found the solution by searching for 
'AUTHENTICATE_uid', it appears to be a relatively common thing.  

Beyond that everything else was the same from the Galaxy wiki.

chris

On Jan 19, 2012, at 8:19 AM, Langhorst, Brad wrote:

 Hi Sarah:
 
 I don't know what kind of LDAP you're talking to... I'm talking to Active
 Directory and this configuration works in that situation.
 
 Order allow,deny
allow from all
 
AuthType Basic
AuthName NEB Credentials
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPBindDN ccaloo...@neb.com
AuthLDAPBindPassword password
AuthLDAPURL
 ldap://ldap.domain.com:389/dc=domain,dc=com?sAMAccountName
require valid-user
 
RewriteCond %{IS_SUBREQ} ^false$
RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
RequestHeader set REMOTE_USER
 %{AUTHENTICATE_sAMAccountName}e
 
 
 -- 
 Brad Langhorst
 New England Biolabs
 langho...@neb.com
 
 
 
 
 
 
 
 On 1/19/12 8:59 AM, Sarah Maman sarah.ma...@toulouse.inra.fr wrote:
 
 Hello,
 
 I'm trying to install galaxy on a apache server,and i have the same
 error message than David HOOVER when i tried to generate the
 authentification files. Thanks to David Hoover who gives me some helpful
 explanation.
 
 When i read the thread on gmod, it seems,that the problem is solved
 (http://gmod.827538.n3.nabble.com/trouble-with-authentication-through-Apac
 he-td1809681.html 
 http://gmod.827538.n3.nabble.com/trouble-with-authentication-through-Apac
 he-td1809681.html)
 but I did not find the solution in my case.
 
 I use LDAP authentification and Galaxy is implemented on a VM.
 
 Could you please explain me how to solve it.  I really need help.
 
 I modified universe file ( in galaxy-dist) and i created .htaccess file
 in galaxy-dist/ :
 * in universe_wsgi.ini:
 
 use_remote_user = True
 remote_user_maildomain = example.org http://example.org
 
 * in .htaccess file in galaxy-dist/ Directory :
 
  1 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_1 # Define Galaxy as a
 valid Proxy
  2 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_2 Proxy
 http://localhost:8080
  3 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_3 Order deny,allow
  4 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_4 Allow from all
  5 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_5 /Proxy
  6 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_6 #!highlight apache
  7 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_7 Location /
  8 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_8 AuthType Basic
  9 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_9 AuthBasicProvider
 *ldap*
 10 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_10 Auth*LDAP*URL
 *ldap*://server:389/ou=People,dc=example,dc=org?uid?sub?(objectClass=pers
 on)
 11 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_11
 Authz*LDAP*Authoritative off
 12 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_12 Require valid-user
 13 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_13 # Set the
 REMOTE_USER header to the contents of the *LDAP* query response's uid
 attribute
 14 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_14 RequestHeader set
 REMOTE_USER %{AUTHENTICATE_uid}e
 15 
 http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy?highlight=%28Ldap%2
 9#CA-e07a9dbbf4f816c0970d4995f4abb957a6683638_15 /Location
 
 with location, AuthBasicProvider and proxy competed
 
 But i have the error :
 
 Access to Galaxy is denied
 
 Galaxy is configured to authenticate users via an external method (such
 as HTTP authentication in Apache), but a

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Sarah Maman


Hi Brad, Hi Christopher,

Thanks a lot.
I'm talking to Open LDAP. I have changed

RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
to
RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e

But I always have the same message /(Access to Galaxy is denied
Galaxy is configured to authenticate users via an external method (such 
as HTTP authentication in Apache), but a username was not provided by 
the upstream (proxy) server. This is generally due to a misconfiguration 
in the upstream server. )/


This is my .conf file :
  Proxy http://localhost:8080
   Order deny,allow
   Allow from all
   /Proxy

   RewriteEngine on

   Location /
   AuthType Basic
   AuthName Galaxy
   AuthBasicProvider ldap
   AuthLDAPURL 
ldap://server/ou=People,ou=genopole,ou=toulouse,o=inra,c=fr?u

id?sub?(objectClass=person)
   AuthzLDAPAuthoritative off
   Require valid-user
   # Take the $REMOTE_USER environment variable and set it 
as a header in the proxy request.

   #RewriteCond %{IS_SUBREQ} ^false$
   #RewriteCond %{LA-U:REMOTE_USER} (.+)
   #RewriteRule . - [E=RU:%1]
   #RequestHeader set REMOTE_USER %{RU}e
   RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
   /Location


Thanks in advance,
Sarah




Langhorst, Brad a écrit :

Hi Sarah:

I don't know what kind of LDAP you're talking to... I'm talking to Active
Directory and this configuration works in that situation.

Order allow,deny
allow from all

AuthType Basic
AuthName NEB Credentials
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPBindDN ccaloo...@neb.com
AuthLDAPBindPassword password
AuthLDAPURL
ldap://ldap.domain.com:389/dc=domain,dc=com?sAMAccountName
require valid-user

RewriteCond %{IS_SUBREQ} ^false$

RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
RequestHeader set REMOTE_USER
%{AUTHENTICATE_sAMAccountName}e


  


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread David Hoover

Why do you need to create a proxy?  The way I have ours set up is within a 
virtual host:

 VirtualHost

  Location /
AuthName Helix Systems
AuthType Basic
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPUrl ldaps://helixdrive.nih.gov/ou=Users,ou=helix.nih.gov,o=scb?uid
AuthLDAPBindDN cn=maccount,ou=Special,o=scb
AuthLDAPBindPassword 
Require valid-user
RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
Options None
Order Allow,Deny
Allow from all
  /Location

  RewriteRule ^/static/style/(.*) 
/data/galaxy/pro/static/june_2007_style/blue/$1 [L]
  RewriteRule ^/static/scripts/(.*) /data/galaxy/pro/static/scripts/packed/$1 
[L]
  RewriteRule ^/static/(.*) /data/galaxy/pro/static/$1 [L]
  RewriteRule ^/favicon.ico /data/galaxy/pro/static/favicon.ico [L] 
  RewriteRule ^/robots.txt /data/galaxy/pro/static/robots.txt [L]
  RewriteRule ^(.*) http://helixweb4.cit.nih.gov:8080$1 [P]

 /VirtualHost

David


On Jan 19, 2012, at 10:49 AM, Sarah Maman wrote:

 Hi Brad, Hi Christopher,
 
 Thanks a lot.
 I'm talking to Open LDAP. I have changed
 
 RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
 to
 RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
 
 But I always have the same message /(Access to Galaxy is denied
 Galaxy is configured to authenticate users via an external method (such as 
 HTTP authentication in Apache), but a username was not provided by the 
 upstream (proxy) server. This is generally due to a misconfiguration in the 
 upstream server. )/
 
 This is my .conf file :
  Proxy http://localhost:8080
   Order deny,allow
   Allow from all
   /Proxy
 
   RewriteEngine on
 
   Location /
   AuthType Basic
   AuthName Galaxy
   AuthBasicProvider ldap
   AuthLDAPURL 
 ldap://server/ou=People,ou=genopole,ou=toulouse,o=inra,c=fr?u
 id?sub?(objectClass=person)
   AuthzLDAPAuthoritative off
   Require valid-user
   # Take the $REMOTE_USER environment variable and set it as a 
 header in the proxy request.
   #RewriteCond %{IS_SUBREQ} ^false$
   #RewriteCond %{LA-U:REMOTE_USER} (.+)
   #RewriteRule . - [E=RU:%1]
   #RequestHeader set REMOTE_USER %{RU}e
   RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
   /Location
 
 
 Thanks in advance,
 Sarah
 
 
 
 
 Langhorst, Brad a écrit :
 Hi Sarah:
 
 I don't know what kind of LDAP you're talking to... I'm talking to Active
 Directory and this configuration works in that situation.
 
 Order allow,deny
allow from all
 
AuthType Basic
AuthName NEB Credentials
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPBindDN ccaloo...@neb.com
AuthLDAPBindPassword password
AuthLDAPURL
 ldap://ldap.domain.com:389/dc=domain,dc=com?sAMAccountName
require valid-user
RewriteCond %{IS_SUBREQ} ^false$
RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
RequestHeader set REMOTE_USER
 %{AUTHENTICATE_sAMAccountName}e
 
 
  
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Assaf Gordon

Hello Sarah,

Sarah Maman wrote, On 01/19/2012 10:49 AM:
 I'm talking to Open LDAP. I have changed
 
 RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
 to
 RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
 
 But I always have the same message /(Access to Galaxy is denied
 Galaxy is configured to authenticate users via an external method (such as 
 HTTP authentication in Apache), but a username was not provided by the 
 upstream (proxy) server. This is generally due to a misconfiguration in the 
 upstream server. )/

If I may suggest further reading and testing:

http://galaxy-users-list-archive.2308625.n4.nabble.com/Galaxy-Apache-External-Authentication-tt4131493.html#a4131495

This post details my configuration after I finally got galaxy to work with LDAP.
I would highly recommend trying step 3 - adding two debug lines to galaxy and 
see which HTTP variables are actually transmitted from the apache server to the 
galaxy process.

-gordon

___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Fields, Christopher J

David,

This is the approach recommended on the wiki for performance:

   http://wiki.g2.bx.psu.edu/Admin/Config/Apache%20Proxy

Would be nice if your alternative approach using virtual hosts were similarly 
documented.

chris

On Jan 19, 2012, at 10:01 AM, David Hoover wrote:

 Why do you need to create a proxy?  The way I have ours set up is within a 
 virtual host:
 
 VirtualHost
 
  Location /
AuthName Helix Systems
AuthType Basic
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPUrl ldaps://helixdrive.nih.gov/ou=Users,ou=helix.nih.gov,o=scb?uid
AuthLDAPBindDN cn=maccount,ou=Special,o=scb
AuthLDAPBindPassword 
Require valid-user
RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
Options None
Order Allow,Deny
Allow from all
  /Location
 
  RewriteRule ^/static/style/(.*) 
 /data/galaxy/pro/static/june_2007_style/blue/$1 [L]
  RewriteRule ^/static/scripts/(.*) /data/galaxy/pro/static/scripts/packed/$1 
 [L]
  RewriteRule ^/static/(.*) /data/galaxy/pro/static/$1 [L]
  RewriteRule ^/favicon.ico /data/galaxy/pro/static/favicon.ico [L] 
  RewriteRule ^/robots.txt /data/galaxy/pro/static/robots.txt [L]
  RewriteRule ^(.*) http://helixweb4.cit.nih.gov:8080$1 [P]
 
 /VirtualHost
 
 David
 
 
 On Jan 19, 2012, at 10:49 AM, Sarah Maman wrote:
 
 Hi Brad, Hi Christopher,
 
 Thanks a lot.
 I'm talking to Open LDAP. I have changed
 
 RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
 to
 RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
 
 But I always have the same message /(Access to Galaxy is denied
 Galaxy is configured to authenticate users via an external method (such as 
 HTTP authentication in Apache), but a username was not provided by the 
 upstream (proxy) server. This is generally due to a misconfiguration in the 
 upstream server. )/
 
 This is my .conf file :
 Proxy http://localhost:8080
  Order deny,allow
  Allow from all
  /Proxy
 
  RewriteEngine on
 
  Location /
  AuthType Basic
  AuthName Galaxy
  AuthBasicProvider ldap
  AuthLDAPURL 
 ldap://server/ou=People,ou=genopole,ou=toulouse,o=inra,c=fr?u
 id?sub?(objectClass=person)
  AuthzLDAPAuthoritative off
  Require valid-user
  # Take the $REMOTE_USER environment variable and set it as a 
 header in the proxy request.
  #RewriteCond %{IS_SUBREQ} ^false$
  #RewriteCond %{LA-U:REMOTE_USER} (.+)
  #RewriteRule . - [E=RU:%1]
  #RequestHeader set REMOTE_USER %{RU}e
  RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
  /Location
 
 
 Thanks in advance,
 Sarah
 
 
 
 
 Langhorst, Brad a écrit :
 Hi Sarah:
 
 I don't know what kind of LDAP you're talking to... I'm talking to Active
 Directory and this configuration works in that situation.
 
 Order allow,deny
   allow from all
 
   AuthType Basic
   AuthName NEB Credentials
   AuthBasicProvider ldap
   AuthzLDAPAuthoritative off
   AuthLDAPBindDN ccaloo...@neb.com
   AuthLDAPBindPassword password
   AuthLDAPURL
 ldap://ldap.domain.com:389/dc=domain,dc=com?sAMAccountName
   require valid-user
   RewriteCond %{IS_SUBREQ} ^false$
   RewriteCond %{LA-U:REMOTE_USER} (.+)
   RewriteRule . - [E=RU:%1]
   RequestHeader set REMOTE_USER
 %{AUTHENTICATE_sAMAccountName}e
 
 
 
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/
 


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

[galaxy-dev] job quotas

2012-01-19 Thread Ivan Merelli


Hi,

is it possible to restrict the number of concurrent jobs for a single
user in a local instance of galaxy? I see that in the public site
this feature is implemented, but I don't find documentation about
how to implement this locally. I think it's not possible through
pbs because all users submit jobs as the defauls galaxy user.

Cheers,

Ivan
___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

 http://lists.bx.psu.edu/

Re: [galaxy-dev] changeset inconsistency

2012-01-19 Thread Nate Coraor

Hi Hans,

We're releasing a distribution today.  If this is still the case with that 
distribution, let me know.  There were branches that weren't showing up under 
newer versions of Mercurial.  Hopefully this is all resolved in the latest.

--nate

On Jan 17, 2012, at 4:11 AM, Hans-Rudolf Hotz wrote:

 Hi
 
 I am in the process of upgrading all our Galaxy servers to the current 
 changeset (b258de1e6cea, Nov. 18) and I have noticed an inconsistency:
 
 If I upgrade one of our old servers (which was on 720455407d1c, June 23) 
 with 'hg pull -u -r b258de1e6cea' I get the following:
 
 
 haruhotz@silicon: hg heads
 changeset:   6297:b258de1e6cea
 tag: tip
 user:jeremy goecks jeremy.goe...@emory.edu
 date:Thu Nov 17 15:08:49 2011 -0500
 summary: Pack js scripts.
 
 haruhotz@silicon:
 
 
 
 If I create a new installation with
 'hg clone http://www.bx.psu.edu/hg/galaxy galaxy-dist' I get the following:
 
 
 
 haruhotz@silicon: hg heads
 changeset:   6298:b258de1e6cea
 tag: tip
 user:jeremy goecks jeremy.goe...@emory.edu
 date:Thu Nov 17 15:08:49 2011 -0500
 summary: Pack js scripts.
 
 changeset:   5832:70b8039a2eef
 branch:  feature/ws
 parent:  5826:6206a27bd2ae
 user:Nate Coraor n...@bx.psu.edu
 date:Wed Jul 27 10:48:03 2011 -0400
 summary: Closed the feature/ws branch, changes from this branch were 
 merged in 5827:f3a1086fac91.
 
 haruhotz@silicon:
 
 
 
 I am not worry about the merged branched, I am just a little bit surprised 
 about the differences:
 
 6298:b258de1e6cea versus 6297:b258de1e6cea
 
 
 Can anybody with more mercurial experience enlighten me what went wrong? and 
 even better tell me that there is nothing to worry about...
 
 
 Regards, Hans
 
 
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Switching TorquePBSpro: qsub error 111 (cannot connect to server)

2012-01-19 Thread Nate Coraor


On Jan 16, 2012, at 5:22 AM, Louise-Amélie Schmitt wrote:

 Hello,
 
 We want to move Galaxy's jobs from our small TORQUE local install to a big 
 cluster running PBS Pro.
 
 In the universe_wsgi.ini, I changed the cluster address as follows:
 default_cluster_job_runner = pbs:///
 to:
 default_cluster_job_runner = pbs://sub-master/clng_new/
 where sub-master is the name of the machine and clng_new is the queue.
 
 However, I get an error when trying to run any job:
 
 galaxy.jobs.runners.pbs ERROR 2012-01-16 11:10:00,894 Connection to PBS 
 server for submit failed: 111: Could not find a text for this error, uhhh
 
 This corresponds to the qsub error 111 (Cannot connect to specified server 
 host) which is, for some reason, caught by pbs_python as an error of its own 
 (111 not corresponding to any pbs_python error code, hence the 
 face-plant-message).
 
 Our guess is that we might need to re-scramble the pbs_python egg with PBS 
 pro's libraries, is that correct?
 If it's the case, what do we have to set as LIBTORQUE_DIR?

Hi L-A,

pbs_python is only designed for TORQUE, I don't think it is compatible with the 
PBS Pro API.  For that, you need to use the drmaa runner, which uses the 
FedStage libdrmaa for PBS Pro.

--nate

 
 Thanks,
 L-A
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] how to set FTP quota?

2012-01-19 Thread Nate Coraor

Hi Leon,

Jen is correct.  As far as the FTP directory, we use the MaxStoreFileSize 
directive in ProFTPd to limit the size of any single file uploaded.

--nate

On Jan 17, 2012, at 10:22 PM, Jennifer Jackson wrote:

 Hi Leon,
 
 I can let you know that there is a hard limit of 50G per file allowed for any 
 dataset FTP'd and moved into a history. Datasets will exist for 3 days in the 
 user-specific holding area before they are automatically purged, if not 
 moved into a history first. While in that holding area, the data are not 
 counted toward or managed by the Galaxy quota system. For more details about 
 exactly how ProFTPd is configured (outside of Galaxy), Nate may be able to 
 jump in with more help if needed. Or maybe you just wanted to double check 
 that the Galaxy quota system was not linked into the initial FTP?
 
 When the time comes, to get started with the Galaxy quota system, you will 
 want to make some adjustments in the universe_wsgi.ini, specifically to this 
 section:
 
 # Enable Galaxy's Upload via FTP interface.  You'll need to install and
 # configure an FTP server (we've used ProFTPd since it can use Galaxy's
 # database for authentication) and set the following two options.
 
 # This should point to a directory containing subdirectories matching users'
 # email addresses, where Galaxy will look for files.
 #ftp_upload_dir = None
 
 # This should be the hostname of your FTP server, which will be provided to
 # users in the help text.
 #ftp_upload_site = None
 
 # Enable enforcement of quotas.  Quotas can be set from the Admin interface.
 #enable_quotas = False
 
 These instruction have the details for setting up quotas (data after it has 
 been imported into a history as a dataset): 
 http://wiki.g2.bx.psu.edu/Admin/Disk%20Quotas
 
 Hopefully this helps, but please let us know if you need more help.
 
 Best,
 
 Jen
 Galaxy team
 
 On 1/16/12 7:41 AM, Leon Mei wrote:
 Dear Galaxy team,
 
 We just installed ProFTPd (the same as Galaxy Main) on our Galaxy
 server. Now we would like to implement a FTP quota. I am wondering how
 Galaxy main handles this. Could you give us some hints? And does the
 disk quota counting script include the FTP directory?
 
 Thanks very much,
 Leon
 
 --
 Hailiang (Leon) Mei
 Netherlands Bioinformatics Center
 BioAssist NGS Taskforce
  - https://wiki.nbic.nl/index.php/Next_Generation_Sequencing
 Skype: leon_meiMobile: +31 6 41709231
 
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
   http://lists.bx.psu.edu/
 
 -- 
 Jennifer Jackson
 http://usegalaxy.org
 http://galaxyproject.org/wiki/Support
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] LDAP authentification

2012-01-19 Thread Nate Coraor

On Jan 19, 2012, at 10:49 AM, Sarah Maman wrote:

 Hi Brad, Hi Christopher,
 
 Thanks a lot.
 I'm talking to Open LDAP. I have changed
 
 RequestHeader set REMOTE_USER %{AUTHENTICATE_uid}e
 to
 RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
 
 But I always have the same message /(Access to Galaxy is denied
 Galaxy is configured to authenticate users via an external method (such as 
 HTTP authentication in Apache), but a username was not provided by the 
 upstream (proxy) server. This is generally due to a misconfiguration in the 
 upstream server. )/

Hi Sarah,

This configuration will need to be in your Apache config (the location of which 
varies depending on your OS and/or how Apache is installed) rather than an 
.htaccess file.

--nate

 
 This is my .conf file :
  Proxy http://localhost:8080
   Order deny,allow
   Allow from all
   /Proxy
 
   RewriteEngine on
 
   Location /
   AuthType Basic
   AuthName Galaxy
   AuthBasicProvider ldap
   AuthLDAPURL 
 ldap://server/ou=People,ou=genopole,ou=toulouse,o=inra,c=fr?u
 id?sub?(objectClass=person)
   AuthzLDAPAuthoritative off
   Require valid-user
   # Take the $REMOTE_USER environment variable and set it as a 
 header in the proxy request.
   #RewriteCond %{IS_SUBREQ} ^false$
   #RewriteCond %{LA-U:REMOTE_USER} (.+)
   #RewriteRule . - [E=RU:%1]
   #RequestHeader set REMOTE_USER %{RU}e
   RequestHeader set REMOTE_USER %{AUTHENTICATE_UID}e
   /Location
 
 
 Thanks in advance,
 Sarah
 
 
 
 
 Langhorst, Brad a écrit :
 Hi Sarah:
 
 I don't know what kind of LDAP you're talking to... I'm talking to Active
 Directory and this configuration works in that situation.
 
 Order allow,deny
allow from all
 
AuthType Basic
AuthName NEB Credentials
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPBindDN ccaloo...@neb.com
AuthLDAPBindPassword password
AuthLDAPURL
 ldap://ldap.domain.com:389/dc=domain,dc=com?sAMAccountName
require valid-user
RewriteCond %{IS_SUBREQ} ^false$
RewriteCond %{LA-U:REMOTE_USER} (.+)
RewriteRule . - [E=RU:%1]
RequestHeader set REMOTE_USER
 %{AUTHENTICATE_sAMAccountName}e
 
 
  
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
 http://lists.bx.psu.edu/
 


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Quota will not decrease with permanent delete

2012-01-19 Thread Nate Coraor

On Jan 16, 2012, at 8:31 AM, Peter Cock wrote:

 On Mon, Jan 16, 2012 at 1:20 PM, graham etherington (TSL)
 graham.ethering...@sainsbury-laboratory.ac.uk wrote:
 Hi Nate,
 We are getting quite large discrepancies between 'Size on disk' and
 'Usage' numbers, so I've run the scripts/set_user_disk_usage.py script.
 When I run it, some of the 'old usage' numbers are negative numbers, e.g.
 User1 old usage: -226417886261, new usage: 139894688577
 User2 old usage: 263698720473, new usage: 68974834617
 User3 old usage: 397108490587, new usage: 406950046043
 User4 old usage: -57441684074, new usage: 7536071906
 
 This seems a little strange (how can a user have a 'minus' usage?). Is
 there an explanation for this?
 
 Negative numbers here are probably a sign of integer overflow
 (something too large for a signed 32 bit integer would be my
 guess).

Possible, although I would guess that set_user_disk_usage.py was never run 
after the disk accounting was added (harmless, since it can be run at any 
time).  Users who already had a bunch of data then started at '0', deleted a 
bunch of data, and end up with negative usage.

--nate

 
 Peter


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] samtools version detection issue

2012-01-19 Thread Nate Coraor

On Jan 18, 2012, at 5:28 PM, Matt Shirley wrote:

 Hi all. I'm sorry if this has been answered before, but I've searched and 
 cannot find a solution other than add SAMtools to your PATH, which I 
 already have done. I can invoke samtools from BASH while logged in as the 
 galaxy user I set up according to the production installation guide. When I 
 try to add .bam files to a shared data library, I get the following message:
 
 Traceback (most recent call last):
 File /home/galaxy/galaxy-dist/tools/data_source/upload.py, line 394, in
 __main__()
 File /home/galaxy/galaxy-dist/tools/data_source/upload.py, line 386, in 
 __main__
 add_file( dataset, registry, json_file, output_path )
 File /home/galaxy/galaxy-dist/tools/data_source/upload.py, line 327, in 
 add_file
 if link_data_only == 'copy_files' and 
 datatype.dataset_content_needs_grooming( output_path ):
 File /home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 79, in 
 dataset_content_needs_grooming
 version = self._get_samtools_version()
 File /home/galaxy/galaxy-dist/lib/galaxy/datatypes/binary.py, line 63, in 
 _get_samtools_version
 output = subprocess.Popen( [ 'samtools' ], stderr=subprocess.PIPE, 
 stdout=subprocess.PIPE ).communicate()[1]
 File /usr/lib64/python2.6/subprocess.py, line 639, in __init__
 errread, errwrite)
 File /usr/lib64/python2.6/subprocess.py, line 1220, in _execute_child
 raise child_exception
 OSError: [Errno 2] No such file or directory
 
 I've taken a peek at the dataset_content_needs_grooming function and it 
 looks like there is an OS call that executes $ samtools and splits the 
 resulting string to access the version number of samtools (why couldn't the 
 developers add a --version flag?). It seems to me like samtools cannot be 
 executed, but I can't think of any reason why. Invoking samtools as user 
 galaxy (with the user's associated PATH) gives:
 
 Program: samtools (Tools for alignments in the SAM format)
 Version: 0.1.12a (r862)
 Usage: samtools command [options]
 Command: view SAM-BAM conversion
 sort sort alignment file
 pileup generate pileup output
 mpileup multi-way pileup
 faidx index/extract FASTA
 tview text alignment viewer
 index index alignment
 idxstats BAM index stats (r595 or later)
 fixmate fix mate information
 glfview print GLFv3 file
 flagstat simple stats
 calmd recalculate MD/NM tags and '=' bases
 merge merge sorted alignments
 rmdup remove PCR duplicates
 reheader replace BAM header
 
 Does anyone know where I've gone wrong?

Hi Matt,

Are you running tools on a cluster?  If running locally, are you starting 
Galaxy by hand (i.e. run.sh) from that same shell in which you can run samtools?

--nate

  
 -- 
 Matt Shirley
 Ph.D Candidate - BCMB
 Pevsner Lab
 Johns Hopkins Medicine
 
 ___
 Please keep all replies on the list by using reply all
 in your mail client.  To manage your subscriptions to this
 and other Galaxy lists, please use the interface at:
 
  http://lists.bx.psu.edu/


___
Please keep all replies on the list by using reply all
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Re: [galaxy-dev] Galaxy Hang after DrmCommunicationException

2012-01-19 Thread Shantanu Pavgi


I haven't added any such delay after DrmCommunicationException however I have 
thought about increasing it in the job monitor function itself (site specific 
change, not suggesting it in the blessed galaxy repo).

https://bitbucket.org/galaxy/galaxy-central/src/cb8d6a29ea22/lib/galaxy/jobs/runners/drmaa.py#cl-259

--
Shantanu

On Jan 18, 2012, at 5:25 PM, Edward Kirton wrote:

perhaps a 15sec sleep should also be added, since the scheduler is overwhelmed

{{{
except ( drmaa.DrmCommunicationException ), e:
  # Catch drmaa communication exception, log a warning message and
  # continue to monitor jobs.
  log.warning((%s/%s) Couldn't communicate with the cluster scheduler to check 
job status. %( galaxy_job_id, job_id ))
  # give scheduler time to catch up
  time.sleep( 15 )
  # keep old job state
  new_watched.append( drm_job_state )
  continue
}}}


On Wed, Jan 18, 2012 at 1:07 PM, Edward Kirton 
eskir...@lbl.govmailto:eskir...@lbl.gov wrote:
excellent, thanks for the correction!


On Wed, Jan 18, 2012 at 8:59 AM, Shantanu Pavgi 
pa...@uab.edumailto:pa...@uab.edu wrote:

Ed,

I think you may want to add job with DrmCommunicationException error back into 
the watched_jobs list.
{{{
except ( drmaa.DrmCommunicationException ), e:
  # Catch drmaa communication exception, log a warning message and
  # continue to monitor jobs.
  log.warning((%s/%s) Couldn't communicate with the cluster scheduler to check 
job status. %( galaxy_job_id, job_id ))
  # keep old job state
  new_watched.append( drm_job_state )
  continue
}}}

Here is a patch and related discussion on it: 
https://bitbucket.org/galaxy/galaxy-central/pull-request/23/catch-drmcommunication-exception

--
Shantanu


On Jan 13, 2012, at 2:11 PM, Edward Kirton wrote:

i had seen the job process die with this error:

if state != old_state:
UnboundLocalError: local variable 'state' referenced before assignment

since the DRM timeout is an intermittent error, i'm not absolutely positive i 
have nailed it (a bit more time will tell -- i'm keeping an eye on it with 
debug log messages), but it seems so.  i intended to share this as a patched 
clone when i became sure but when i read your email i just sent what i had 
immediately.  let us know if that seems to solve the problem for you, so we'll 
have some confirmation.

glad to help,
ed

p.s. i have another patch for output not returned from cluster errors that 
i'm also still validating, due to NFS race since we don't have inode metadata 
caching turned off as the galaxy developers suggest.

On Fri, Jan 13, 2012 at 8:06 AM, Ann Black 
annbl...@eng.uiowa.edumailto:annbl...@eng.uiowa.edu wrote:
Thanks so much! I have applied the fix to our env.

In looking over the logic, there was an existing exception block that would 
have caught the communication exception generically – but the job moved (in 
this scenario erroneously) the job into a failure workflow.  I would like to 
understand what ended up hanging galaxy – so it must be related to a valid job 
being moved into failure state ? Did you follow it down the rabbit hole by any 
chance to see what caused the hang in your env ?

Thanks again,

Ann

From: Edward Kirton eskir...@lbl.govmailto:eskir...@lbl.gov
Date: Thu, 12 Jan 2012 13:00:27 -0800
To: Ann Black annbl...@eng.uiowa.edumailto:annbl...@eng.uiowa.edu
Cc: galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu 
galaxy-dev@lists.bx.psu.edumailto:galaxy-dev@lists.bx.psu.edu
Subject: Re: [galaxy-dev] Galaxy Hang after DrmCommunicationException

sometimes the scheduler can't keep up with all the work in it's 15sec cycle, so 
it doesn't respond to some messages.  here's a fix i've been trying that seems 
to work.

in lib/galaxy/jobs/runners/drmaa.py:

def check_watched_items( self ):

Called by the monitor thread to look at each watched job and deal
with state changes.

new_watched = []
for drm_job_state in self.watched:
job_id = drm_job_state.job_id
galaxy_job_id = drm_job_state.job_wrapper.job_id
old_state = drm_job_state.old_state
try:
state = self.ds.jobStatus( job_id )
# InternalException was reported to be necessary on some DRMs, but
# this could cause failures to be detected as completion!  Please
# report if you experience problems with this.
except ( drmaa.InvalidJobException, drmaa.InternalException ), e:
# we should only get here if an orphaned job was put into the 
queue at app star
tup
log.debug((%s/%s) job left DRM queue with following message: 
%s % ( galaxy_jo
b_id, job_id, e ) )
self.work_queue.put( ( 'finish', drm_job_state ) )
continue
# BEGIN DRM TIMEOUT: Don't fail on scheduler communication error 
(probably just too busy)
except ( drmaa.DrmCommunicationException ), e:
log.warning((%s/%s) DRM Communication Exception

Re: [galaxy-dev] Error download composite dataset from history

Re: [galaxy-dev] problems with location of filter bar in trackster

Re: [galaxy-dev] Error download composite dataset from history

[galaxy-dev] LDAP authentification

Re: [galaxy-dev] LDAP authentification

Re: [galaxy-dev] LDAP authentification

Re: [galaxy-dev] LDAP authentification

Re: [galaxy-dev] LDAP authentification

Re: [galaxy-dev] LDAP authentification

Re: [galaxy-dev] LDAP authentification

[galaxy-dev] job quotas

Re: [galaxy-dev] changeset inconsistency

Re: [galaxy-dev] Switching TorquePBSpro: qsub error 111 (cannot connect to server)

Re: [galaxy-dev] how to set FTP quota?

Re: [galaxy-dev] LDAP authentification

Re: [galaxy-dev] Quota will not decrease with permanent delete

Re: [galaxy-dev] samtools version detection issue

Re: [galaxy-dev] Galaxy Hang after DrmCommunicationException

18 matches

Site Navigation

Mail list logo

Footer information