date:20150122

Re: [galaxy-dev] Manually installing rsem_datatypes package from toolshed

2015-01-22 Thread Ryan G

Yup, that fixed it. Thanks!

On Wed, Jan 21, 2015 at 6:51 PM, John Chilton  wrote:

> I believe this is an ancient bug that I has been fixed in Galaxy's
> central branch but is not in the latest release:
>
>
> https://bitbucket.org/galaxy/galaxy-central/commits/1422966f1ca88472b8685015f5a8cdd3dd0f1db9
>
> The previous workaround for this bug was that one would need to import
> the datatype at the top of registry.py (import rsem_datatypes) which
> is another option for addressing this problem instead of applying the
> patch.
>
> Hope this helps,
> -John
>
>
> On Wed, Jan 21, 2015 at 2:16 PM, Ryan G 
> wrote:
> > I'm trying to manually install rsem_datatypes package into my local
> Galaxy
> > install.  I can't use the Toolshed for various reasons.
> >
> > Anyway, I downloaded the package using
> >
> > hg clone https://toolshed.g2.bx.psu.edu/repos/jjohnson/rsem_datatypes
> >
> > I see two files:
> >
> > 1)  datatypes_conf.xml, and
> > 2) rsem.py
> >
> > I copied the  lines into my
> > config/datatypes_conf.xml file.
> > I also copied  rsem.py to lib/galaxy/datatypes/.
> >
> > When I restart Galaxy, it doesn't seem that Galaxy knows how to handle
> rsem
> > datatypes, as I get this error:
> >
> > galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing
> > datatype module galaxy.datatypes.rsem: 'module' object has no attribute
> > 'rsem'
> > Traceback (most recent call last):
> >   File "/data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line
> > 210, in load_datatypes
> > module = getattr( module, mod )
> > AttributeError: 'module' object has no attribute 'rsem'
> > galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,519 Error importing
> > datatype module galaxy.datatypes.rsem: 'module' object has no attribute
> > 'rsem'
> > Traceback (most recent call last):
> >   File "/data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line
> > 210, in load_datatypes
> > module = getattr( module, mod )
> > AttributeError: 'module' object has no attribute 'rsem'
> > galaxy.datatypes.registry ERROR 2015-01-21 14:01:47,520 Error importing
> > datatype module galaxy.datatypes.rsem: 'module' object has no attribute
> > 'rsem'
> > Traceback (most recent call last):
> >   File "/data/galaxy/galaxy-dist/lib/galaxy/datatypes/registry.py", line
> > 210, in load_datatypes
> > module = getattr( module, mod )
> > AttributeError: 'module' object has no attribute 'rsem'
> >
> > I'm obviously missing something but not sure what.  I suspect maybe the
> > rsem.py file is not in the right place?
> >
> > Ryan
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] show stopper for my galaxy installation

2015-01-22 Thread Eric Rasche

The tool to fetch data isn't being run on your submit node but instead on
the cluster, which is the problem.

It's a fairly easy to solve problem:

   - In your job_conf you'll need to define a way to run jobs locally
   (either via local runner, or via a special queue dedicated to your submit
   node)
   - Then you'll need to specify that the upload tool is forced to that job
   runner.

Something like this (untested, probably invalid xml... )


  
  


  
  



  
  


Cheers,
Eric

2015-01-22 13:51 GMT-06:00 Fernandez Edgar :

>  Hello gents,
>
>
>
> I started this new thread because I have a showstopper for my installation
> of galaxy.
>
> Here’s my situation:
>
> 1.   I have a galaxy server that has internet access.
>
> 2.   My galaxy server is also my torque server and my only torque
> submit node.
>
> 3.   I have three torque compute nodes that DOESN’T have internet
> access.
>
>
>
> Now, I’ve tried uploading a file (larger than 2Gb) via and URL and it
> *fails*.
>
> However, I’ve added my galaxy server as a compute node and re-tried to
> upload the same file and I made sure the job will run on that machine and
> it *works*.
>
>
>
> Now, my compute nodes are in a network that is completely protected from
> the outside world.
>
> So no internet access.
>
>
>
> What are my option in this case?
>
>
>
> Cordialement / Regards,
>
>
>
> *Edgar Fernandez*
>
> System Administrator (Linux)
>
> Direction Générale des Technologies de l'Information et de la Communication
>
> (  Bur. : *1-514-343-6111 poste 16568*
>
>
>
> *Université de Montréal*
>
> PAVILLON ROGER-GAUDRY, bureau X-218
>
>
>
> ___
> Please keep all replies on the list by using "reply all"
> in your mail client.  To manage your subscriptions to this
> and other Galaxy lists, please use the interface at:
>   https://lists.galaxyproject.org/
>
> To search Galaxy mailing lists use the unified search at:
>   http://galaxyproject.org/search/mailinglists/
>



-- 
Eric Rasche
Programmer II

Center for Phage Technology
Rm 312A, BioBio
Texas A&M University
College Station, TX 77843
404-692-2048
e...@tamu.edu
rasche.e...@yandex.ru
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

[galaxy-dev] show stopper for my galaxy installation

2015-01-22 Thread Fernandez Edgar

Hello gents,

I started this new thread because I have a showstopper for my installation of 
galaxy.
Here's my situation:

1.   I have a galaxy server that has internet access.

2.   My galaxy server is also my torque server and my only torque submit 
node.

3.   I have three torque compute nodes that DOESN'T have internet access.

Now, I've tried uploading a file (larger than 2Gb) via and URL and it fails.
However, I've added my galaxy server as a compute node and re-tried to upload 
the same file and I made sure the job will run on that machine and it works.

Now, my compute nodes are in a network that is completely protected from the 
outside world.
So no internet access.

What are my option in this case?

Cordialement / Regards,

Edgar Fernandez
System Administrator (Linux)
Direction Générale des Technologies de l'Information et de la Communication
*  Bur. : 1-514-343-6111 poste 16568

Université de Montréal
PAVILLON ROGER-GAUDRY, bureau X-218

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] galaxy and torque - resource allocation

2015-01-22 Thread Fernandez Edgar

Hello guys,

I want to apologies.
I didn’t see the note:
Due to browser limitations, uploading files larger than 2GB is guaranteed to 
fail. To upload large files, use the URL method (below) or FTP (if enabled by 
the site administrator).
and I forgot my file was that big.

Please accept my sincere apologies.

Cordialement / Regards,
Edgar Fernandez

De : Fernandez Edgar
Envoyé : January-22-15 12:35 PM
À : 'Nate Coraor'
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : RE: [galaxy-dev] galaxy and torque - resource allocation

Hello Nate,

I just tried the modal upload dialog and it says "File exceeds 2GB. Please use 
FTP client".
Is there a way to increase that file size limitation?

Cordialement / Regards,
Edgar Fernandez

De : Nate Coraor [mailto:n...@bx.psu.edu]
Envoyé : January-22-15 12:02 PM
À : Fernandez Edgar
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Hi Edgar,

Please make sure you have disabled the developer options `use_interactive` and 
`debug` in galaxy.ini, then restart your Galaxy server and try the upload again.

Also, please try out the modal upload dialog as I suggested, it should prevent 
a "stuck" upload in the future.

--nate



On Thu, Jan 22, 2015 at 10:25 AM, Fernandez Edgar 
mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello Nate,


Thank you for answer so quickly.
My apache logs (/var/log/httpd/galaxy-prod-access_log) looks like this
ip_addr - - [22/Jan/2015:10:17:31 -0500] "POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 37
ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 1401
ip_addr - - [22/Jan/2015:10:17:36 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:40 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:44 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:48 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235

And my galaxy logs (/home/galaxy/galaxy-prod/config/main.log):
ip_addr - - [22/Jan/2015:10:17:31 -0400] "POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
ip_addr - - [22/Jan/2015:10:17:32 -0400] "POST /galaxy-prod/tool_runner/index 
HTTP/1.1" 500 - "http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; 
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/40.0.2214.91 Safari/537.36"
Error - : signed integer is greater than 
maximum
ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
URL: http://esiservera.esi.umontreal.ca:8081/galaxy-prod/tool_runner/index
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/error.py', 
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
 line 633 in __call__
  return self.application(environ, start_response)
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 132 in 
__call__
  return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 185 in 
handle_request
  kwargs = trans.request.params.mixed()
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 900 in params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 892 in 
str_params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 818 in str_POST
File '/usr/lib64/python2.6/cgi.py', line 508 in __init__
  self.read_multi(environ, keep_blank_values, strict_parsing)
File '/usr/lib64/python2.6/cgi.py', line 635 in read_multi
  headers = rfc822.Message(self.fp)
File '/usr/lib64/python2.6/rfc822.py', line 108 in __init__
  self.readheaders()
File '/usr/lib64/python2.6/rfc822.py', line 155 in readheaders
  line = self.fp.readline()
File 
'/home/galaxy/galaxy-prod/eggs/Paste-

Re: [galaxy-dev] galaxy and torque - resource allocation

2015-01-22 Thread Fernandez Edgar

Hello Nate,

I just tried the modal upload dialog and it says "File exceeds 2GB. Please use 
FTP client".
Is there a way to increase that file size limitation?

Cordialement / Regards,
Edgar Fernandez

De : Nate Coraor [mailto:n...@bx.psu.edu]
Envoyé : January-22-15 12:02 PM
À : Fernandez Edgar
Cc : John Chilton; galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Hi Edgar,

Please make sure you have disabled the developer options `use_interactive` and 
`debug` in galaxy.ini, then restart your Galaxy server and try the upload again.

Also, please try out the modal upload dialog as I suggested, it should prevent 
a "stuck" upload in the future.

--nate



On Thu, Jan 22, 2015 at 10:25 AM, Fernandez Edgar 
mailto:edgar.fernan...@umontreal.ca>> wrote:
Hello Nate,


Thank you for answer so quickly.
My apache logs (/var/log/httpd/galaxy-prod-access_log) looks like this
ip_addr - - [22/Jan/2015:10:17:31 -0500] "POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 37
ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 1401
ip_addr - - [22/Jan/2015:10:17:36 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:40 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:44 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:48 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235

And my galaxy logs (/home/galaxy/galaxy-prod/config/main.log):
ip_addr - - [22/Jan/2015:10:17:31 -0400] "POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
ip_addr - - [22/Jan/2015:10:17:32 -0400] "POST /galaxy-prod/tool_runner/index 
HTTP/1.1" 500 - "http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; 
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/40.0.2214.91 Safari/537.36"
Error - : signed integer is greater than 
maximum
ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
URL: http://esiservera.esi.umontreal.ca:8081/galaxy-prod/tool_runner/index
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/error.py', 
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
 line 633 in __call__
  return self.application(environ, start_response)
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 132 in 
__call__
  return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 185 in 
handle_request
  kwargs = trans.request.params.mixed()
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 900 in params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 892 in 
str_params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 818 in str_POST
File '/usr/lib64/python2.6/cgi.py', line 508 in __init__
  self.read_multi(environ, keep_blank_values, strict_parsing)
File '/usr/lib64/python2.6/cgi.py', line 635 in read_multi
  headers = rfc822.Message(self.fp)
File '/usr/lib64/python2.6/rfc822.py', line 108 in __init__
  self.readheaders()
File '/usr/lib64/python2.6/rfc822.py', line 155 in readheaders
  line = self.fp.readline()
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py', 
line 482 in readline
  data = self.file.readline(max_read)
File '/usr/lib64/python2.6/socket.py', line 415 in readline
  bline = buf.readline(size)
OverflowError: signed integer is greater than maximum


CGI Variables
-
  CONTENT_LENGTH: '2625522525'
  CONTENT_TYPE: 'multipart/form-data; 
boundary=WebKitFormBoundary9c9hOVrAGAXKJtdd'
  HTTP_ACCEPT: '*/*'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.8'
  HTTP_CONNECTION: 'Keep-Alive'
  HTTP_COOKIE: 
'galaxysession=d122

Re: [galaxy-dev] galaxy and torque - resource allocation

2015-01-22 Thread Nate Coraor

Hi Edgar,

Please make sure you have disabled the developer options `use_interactive`
and `debug` in galaxy.ini, then restart your Galaxy server and try the
upload again.

Also, please try out the modal upload dialog as I suggested, it should
prevent a "stuck" upload in the future.

--nate



On Thu, Jan 22, 2015 at 10:25 AM, Fernandez Edgar <
edgar.fernan...@umontreal.ca> wrote:

>  Hello Nate,
>
>
>
>
>
> Thank you for answer so quickly.
>
> My apache logs (/var/log/httpd/galaxy-prod-access_log) looks like this
>
> *ip_addr - - [22/Jan/2015:10:17:31 -0500] "POST
> /galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 37*
>
> *ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET
> /galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235*
>
> *ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET
> /galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 1401*
>
> *ip_addr - - [22/Jan/2015:10:17:36 -0500] "GET
> /galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235*
>
> *ip_addr - - [22/Jan/2015:10:17:40 -0500] "GET
> /galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235*
>
> *ip_addr - - [22/Jan/2015:10:17:44 -0500] "GET
> /galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235*
>
> *ip_addr - - [22/Jan/2015:10:17:48 -0500] "GET
> /galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235*
>
>
>
> And my galaxy logs (/home/galaxy/galaxy-prod/config/main.log):
>
> ip_addr - - [22/Jan/2015:10:17:31 -0400] "POST
> /galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 - "
> http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0
> (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/40.0.2214.91 Safari/537.36"
>
> ip_addr - - [22/Jan/2015:10:17:32 -0400] "POST
> /galaxy-prod/tool_runner/index HTTP/1.1" 500 - "
> http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0
> (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/40.0.2214.91 Safari/537.36"
>
> Error - : signed integer is greater than
> maximum
>
> ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET
> /galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 - "
> http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0
> (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/40.0.2214.91 Safari/537.36"
>
> ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET
> /galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 - "
> http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0
> (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/40.0.2214.91 Safari/537.36"
>
> URL: http://esiservera.esi.umontreal.ca:8081/galaxy-prod/tool_runner/index
>
> File
> '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/error.py',
> line 149 in __call__
>
>   app_iter = self.application(environ, sr_checker)
>
> File
> '/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py',
> line 84 in __call__
>
>   return self.application(environ, start_response)
>
> File
> '/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
> line 633 in __call__
>
>   return self.application(environ, start_response)
>
> File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 132
> in __call__
>
>   return self.handle_request( environ, start_response )
>
> File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 185
> in handle_request
>
>   kwargs = trans.request.params.mixed()
>
> File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 900 in
> params
>
> File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 892 in
> str_params
>
> File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 818 in
> str_POST
>
> File '/usr/lib64/python2.6/cgi.py', line 508 in __init__
>
>   self.read_multi(environ, keep_blank_values, strict_parsing)
>
> File '/usr/lib64/python2.6/cgi.py', line 635 in read_multi
>
>   headers = rfc822.Message(self.fp)
>
> File '/usr/lib64/python2.6/rfc822.py', line 108 in __init__
>
>   self.readheaders()
>
> File '/usr/lib64/python2.6/rfc822.py', line 155 in readheaders
>
>   line = self.fp.readline()
>
> File
> '/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py',
> line 482 in readline
>
>   data = self.file.readline(max_read)
>
> File '/usr/lib64/python2.6/socket.py', line 415 in readline
>
>   bline = buf.readline(size)
>
> OverflowError: signed integer is greater than maximum
>
>
>
>
>
> CGI Variables
>
> -
>
>   CONTENT_LENGTH: '2625522525'
>
>   CONTENT_TYPE: 'multipart/form-data;
> boundary=WebKitFormBoundary9c9hOVrAGAXKJtdd'
>
>   HTTP_ACCEPT: '*/*'
>
>   HTTP_ACCEPT_ENCODING: 'gzip, deflate'
>
>   HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.8'
>
>   HTTP_CONNECTION: 'Keep-Alive'
>
>   HTTP_COOKIE:
> 'galaxysession=d122f4f53e5da72f16add67ded507460da12cda8c8b0781476c9ddbea84bd40a1e299b3d9b7a9e3a;
> galaxysession=d122f4f53e5da72f439b77b098b6a77c8f

Re: [galaxy-dev] galaxy and torque - resource allocation

2015-01-22 Thread Fernandez Edgar

Hello Nate,


Thank you for answer so quickly.
My apache logs (/var/log/httpd/galaxy-prod-access_log) looks like this
ip_addr - - [22/Jan/2015:10:17:31 -0500] "POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 37
ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:32 -0500] "GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 1401
ip_addr - - [22/Jan/2015:10:17:36 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:40 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:44 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235
ip_addr - - [22/Jan/2015:10:17:48 -0500] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 235

And my galaxy logs (/home/galaxy/galaxy-prod/config/main.log):
ip_addr - - [22/Jan/2015:10:17:31 -0400] "POST 
/galaxy-prod/tool_runner/upload_async_create HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
ip_addr - - [22/Jan/2015:10:17:32 -0400] "POST /galaxy-prod/tool_runner/index 
HTTP/1.1" 500 - "http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; 
"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) 
Chrome/40.0.2214.91 Safari/537.36"
Error - : signed integer is greater than 
maximum
ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET 
/galaxy-prod/api/histories/5a0831c911c29227/contents HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
ip_addr - - [22/Jan/2015:10:17:32 -0400] "GET 
/galaxy-prod/tool_runner/upload_async_message HTTP/1.1" 200 - 
"http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root"; "Mozilla/5.0 (X11; 
Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.91 
Safari/537.36"
URL: http://esiservera.esi.umontreal.ca:8081/galaxy-prod/tool_runner/index
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/middleware/error.py', 
line 149 in __call__
  app_iter = self.application(environ, sr_checker)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/recursive.py', 
line 84 in __call__
  return self.application(environ, start_response)
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpexceptions.py',
 line 633 in __call__
  return self.application(environ, start_response)
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 132 in 
__call__
  return self.handle_request( environ, start_response )
File '/home/galaxy/galaxy-prod/lib/galaxy/web/framework/base.py', line 185 in 
handle_request
  kwargs = trans.request.params.mixed()
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 900 in params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 892 in 
str_params
File 'build/bdist.linux-x86_64-ucs4/egg/webob/__init__.py', line 818 in str_POST
File '/usr/lib64/python2.6/cgi.py', line 508 in __init__
  self.read_multi(environ, keep_blank_values, strict_parsing)
File '/usr/lib64/python2.6/cgi.py', line 635 in read_multi
  headers = rfc822.Message(self.fp)
File '/usr/lib64/python2.6/rfc822.py', line 108 in __init__
  self.readheaders()
File '/usr/lib64/python2.6/rfc822.py', line 155 in readheaders
  line = self.fp.readline()
File 
'/home/galaxy/galaxy-prod/eggs/Paste-1.7.5.1-py2.6.egg/paste/httpserver.py', 
line 482 in readline
  data = self.file.readline(max_read)
File '/usr/lib64/python2.6/socket.py', line 415 in readline
  bline = buf.readline(size)
OverflowError: signed integer is greater than maximum


CGI Variables
-
  CONTENT_LENGTH: '2625522525'
  CONTENT_TYPE: 'multipart/form-data; 
boundary=WebKitFormBoundary9c9hOVrAGAXKJtdd'
  HTTP_ACCEPT: '*/*'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.8'
  HTTP_CONNECTION: 'Keep-Alive'
  HTTP_COOKIE: 
'galaxysession=d122f4f53e5da72f16add67ded507460da12cda8c8b0781476c9ddbea84bd40a1e299b3d9b7a9e3a;
 
galaxysession=d122f4f53e5da72f439b77b098b6a77c8fe4c3cee6d2a1f587a8445290a468f305b3911dd27efa9e'
  HTTP_HOST: 'esiservera.esi.umontreal.ca:8081'
  HTTP_ORIGIN: 'http://esiservera.esi.umontreal.ca:8081'
  HTTP_REFERER: 'http://esiservera.esi.umontreal.ca:8081/galaxy-prod/root'
  HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, 
like Gecko) Chrome/40.0.2214.91 Safari/537.36'
  HTTP_X_REQUESTED_WITH: 'XMLHttpRequest'
  PATH_INFO: '/tool_runner/index'
  REMOTE_ADDR: 'ip_addr'
  REQUEST_METHOD: 'POST'
  SCRIPT_NAME: '/galaxy-prod'
  SERVER_NAME: 'esiservera.esi.umontreal.ca'
  SERVER_PORT: '7112'
  SERVER_PROTOCOL: 'HTTP/1.1'


WSGI Variables
--
  application: 
  is_api_re

Re: [galaxy-dev] galaxy and torque - resource allocation

2015-01-22 Thread Nate Coraor

Hi Edgar,

I'd suggest checking your Apache logs to see if the upload was denied by
the server configuration (possibly for being too large).

Also, please try using the modal upload method to see whether the upload is
actually completing. This can be found by clicking the icon of a box with
an up arrow at the top of the tool panel.

There are multiple steps to an upload - the first is sending the data to
the server. After that, Galaxy creates a job and detects metadata about the
uploaded dataset. Prior to the modal upload, a (purple "uploading") history
item would be created as soon as the upload started, but if the transfer to
the server was interrupted the history item may remain "stuck" in that
state without providing feedback to the user that the upload was no longer
progressing. The modal provides an upload progress bar, and only once all
the data has been sent to the server does it create a history item (in the
gray "queued") state.


On Thu, Jan 22, 2015 at 8:50 AM, Fernandez Edgar <
edgar.fernan...@umontreal.ca> wrote:

> Hello John,
>
> This is totally excellent!
> This gave me a great idea for my little torque installation.
>
> I hope you don't mind but I have an unrelated issue with torque that I do
> not seem to grasp.
> My issue is that it take a rediculious amount of time to upload a 2Gb file.
> I tried many different configurations of apache to upload that 2Gb file
> without success.
> Galaxy keeps showing that arrow going up constantly (over 12h).
> And this is an upload from a file located on the server running galaxy.
>
> Here is some information from my installation.
> 1. Server redhat6 with galaxy behind an Apache HTTP Server proxy like
> described on your website.
> 2. I have Apache HTTP Server running as the galaxy user with
> a. Serving Galaxy at a sub directory
> b. Compression and caching
> c. Sending files using Apache
>
> You will see in attachment my apache's galaxy, deflate, expire and
> xsendfile configuration file.
> Furthermore, I've changed the following in galaxy.ini:
> apache_xsendfile = True
> upstream_gzip = False
>
> Any suggestions?
> Any information I missed?
> PLEASE HELP!!!
>
> Cordialement / Regards,
> Edgar Fernandez
>
>
> -Message d'origine-
> De : John Chilton [mailto:jmchil...@gmail.com]
> Envoyé : January-21-15 9:34 PM
> À : Fernandez Edgar
> Cc : galaxy-...@bx.psu.edu
> Objet : Re: [galaxy-dev] galaxy and torque - resource allocation
>
> Was hoping someone with an actual torque cluster would respond. I think
> the way you would configure this might be for instance might be setting
> native_specification like "-l nodes=1:ppn=n". Depending on how things are
> configured or simply because I am ignorant - this may be wrong - but I
> think what you really want should look a lot like your arguments to qsub.
> So let say you have defined a job runner plugin called pbs_drmaa at the top
> of your job_conf.xml file. Then you could default everything to a single
> core by default and send "big" jobs to a destination with 8 cores with
> destinations and tool sections that look something like this...
>
> ...
>
>   
> 
>   -l nodes=1:ppn=1
> 
> 
>   -l nodes=1:ppn=8
> 
>   
>
> and ...
>
>   
> 
> 
> 
> 
> 
> 
> 
>   
>
> Again - that native specification could be wrong - probably best to test
> it out locally (or maybe Nate or JJ will step in and correct me).
>
> Hope this helps,
> -John
>
>
> On Mon, Jan 19, 2015 at 8:05 AM, Fernandez Edgar <
> edgar.fernan...@umontreal.ca> wrote:
> > Hello gents,
> >
> > Once again I would like to convey my most sincere appreciation for your
> help!!!
> > And yes I would like to hear your elaboration on my DRMAA runner which
> is what I'm using.
> > So my installation looks like: galaxy --> pbs_drmaa --> torque
> >
> > Cordialement / Regards,
> > Edgar Fernandez
> >
> > -Message d'origine-
> > De : John Chilton [mailto:jmchil...@gmail.com] Envoyé : January-18-15
> > 8:59 PM À : Fernandez Edgar Cc : galaxy-...@bx.psu.edu Objet : Re:
> > [galaxy-dev] galaxy and torque - resource allocation
> >
> > Galaxy generally defers to the DRM (torque in your case) for dealing
> with these things. In your job_conf.xml you can specify limits for memory
> or CPUs and Galaxy will pass these along to the DRM at which point it is up
> to the DRM to enforce these - details depend on if you are using the PBS
> runner or the DRMAA runner (let me know which and I can try to elobrate if
> that would be useful).
> >
> > In your particular case - I don't believe torque "schedules" RAM so
> > things will generally only be... throttled... by CPUs counts. This is
> > what I was told by the admins at MSI when I worked there anyway. If
> > you want to place hard limits on RAM I think you need to "upgrade" to
> > the MOAB scheduler or switch over to a different DRM entirely like
> > SLURM. Even for DRMs that deal more directly with memor

Re: [galaxy-dev] galaxy and torque - resource allocation

2015-01-22 Thread Fernandez Edgar

Hello John,

This is totally excellent!
This gave me a great idea for my little torque installation.

I hope you don't mind but I have an unrelated issue with torque that I do not 
seem to grasp.
My issue is that it take a rediculious amount of time to upload a 2Gb file.
I tried many different configurations of apache to upload that 2Gb file without 
success.
Galaxy keeps showing that arrow going up constantly (over 12h).
And this is an upload from a file located on the server running galaxy.

Here is some information from my installation.
1. Server redhat6 with galaxy behind an Apache HTTP Server proxy like described 
on your website.
2. I have Apache HTTP Server running as the galaxy user with
a. Serving Galaxy at a sub directory
b. Compression and caching
c. Sending files using Apache

You will see in attachment my apache's galaxy, deflate, expire and xsendfile 
configuration file.
Furthermore, I've changed the following in galaxy.ini:
apache_xsendfile = True
upstream_gzip = False

Any suggestions?
Any information I missed?
PLEASE HELP!!!

Cordialement / Regards, 
Edgar Fernandez

-Message d'origine-
De : John Chilton [mailto:jmchil...@gmail.com] 
Envoyé : January-21-15 9:34 PM
À : Fernandez Edgar
Cc : galaxy-...@bx.psu.edu
Objet : Re: [galaxy-dev] galaxy and torque - resource allocation

Was hoping someone with an actual torque cluster would respond. I think the way 
you would configure this might be for instance might be setting 
native_specification like "-l nodes=1:ppn=n". Depending on how things are 
configured or simply because I am ignorant - this may be wrong - but I think 
what you really want should look a lot like your arguments to qsub. So let say 
you have defined a job runner plugin called pbs_drmaa at the top of your 
job_conf.xml file. Then you could default everything to a single core by 
default and send "big" jobs to a destination with 8 cores with destinations and 
tool sections that look something like this...

...

  -l nodes=1:ppn=1

  -l nodes=1:ppn=8

and ...

Again - that native specification could be wrong - probably best to test it out 
locally (or maybe Nate or JJ will step in and correct me).

Hope this helps,
-John

On Mon, Jan 19, 2015 at 8:05 AM, Fernandez Edgar  
wrote:
> Hello gents,
>
> Once again I would like to convey my most sincere appreciation for your 
> help!!!
> And yes I would like to hear your elaboration on my DRMAA runner which is 
> what I'm using.
> So my installation looks like: galaxy --> pbs_drmaa --> torque
>
> Cordialement / Regards,
> Edgar Fernandez
>
> -Message d'origine-
> De : John Chilton [mailto:jmchil...@gmail.com] Envoyé : January-18-15 
> 8:59 PM À : Fernandez Edgar Cc : galaxy-...@bx.psu.edu Objet : Re: 
> [galaxy-dev] galaxy and torque - resource allocation
>
> Galaxy generally defers to the DRM (torque in your case) for dealing with 
> these things. In your job_conf.xml you can specify limits for memory or CPUs 
> and Galaxy will pass these along to the DRM at which point it is up to the 
> DRM to enforce these - details depend on if you are using the PBS runner or 
> the DRMAA runner (let me know which and I can try to elobrate if that would 
> be useful).
>
> In your particular case - I don't believe torque "schedules" RAM so 
> things will generally only be... throttled... by CPUs counts. This is 
> what I was told by the admins at MSI when I worked there anyway. If 
> you want to place hard limits on RAM I think you need to "upgrade" to 
> the MOAB scheduler or switch over to a different DRM entirely like 
> SLURM. Even for DRMs that deal more directly with memory - Galaxy 
> doesn't provide a general mechanism for passing this along to the tool
> (https://trello.com/c/3RkTDnIn) - so it would be up to the tool to interact 
> with the environment variables your DRM sets .
>
> This sounds really terrible in the abstract - but it reality it usually isn't 
> an issue - most of Galaxy's multi-core mappers say have relatively low memory 
> per CPU usage - and for things like assemblers where this is more important - 
> one can usually just assign them to their own CPU or something like that to 
> ensure they get all the memory available.
>
> Unlike memory - Galaxy will attempt to pass the number of slots allocated to 
> a job to the tools by setting the GALAXY_SLOTS environment variable. All of 
> the multiple core devteam Galaxy tools at this point use this and so should 
> work - as should probably most multi-core tools from the tool shed - at least 
> the most popular ones.
>
> Hope this helps,
>
> -John
>
>
> On Tue, Jan 13, 2015 at 11:52 AM, Fernandez Edgar 
>  wrote:
>> Hello gents,
>>
>>
>>
>> Hope everyone had a great holiday break!
>>
>> Wish you guys all the best for 2015!
>>
>>
>>
>> I have a couple of questions about how resources (CPU and memory) is 
>> allocated when you

[galaxy-dev] GALAXY_DATA_INDEX_DIR problem

2015-01-22 Thread Lukasse, Pieter

Hi,

In the documentation page 
https://wiki.galaxyproject.org/Admin/Tools/ToolConfigSyntax it mentions that

"the reserved parameter name GALAXY_DATA_INDEX_DIR - it points to the 
~/tool-data 
directory."

However, when we use this in a tool that is installed via toolshed we get a 
different interpretation of the variable: it seems to point to something like 
this at runtime:

./database/tmp/tmp-toolshed-gmfcrIr59Sq


Ideas?


Pieter Lukasse
Wageningen UR, Plant Research International
Department of Bioinformatics (Bioscience)
Wageningen Campus, Building 107, Droevendaalsesteeg 1, 6708 PB,
Wageningen, the Netherlands
T: +31-317481122;
M: +31-628189540;
skype: pieter.lukasse.wur
http://www.pri.wur.nl

___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Tool development - Selecting a single item from input dataset.

2015-01-22 Thread Vimalkumar Velayudhan

Thanks Peter. I see how this feature would be useful, but the program I'm
writing a wrapper for has an argument with values corresponding to the
input files. I am using a  tag to maintain this order. With the
multi-run option, files are selected in a random manner and added to the
job queue. It is best not to display the multi-run option in this case.

I see there is a TODO on this already:
https://bitbucket.org/galaxy/galaxy-dist/src/a2308bdc93b897af974766b190abe019ade49e9a/lib/galaxy/tools/parameters/basic.py?at=default#cl-2084

For now, I have set allow=False but I believe this is best set at the param
tag level:




Vimal

On Wed, Jan 21, 2015 at 2:26 AM, Peter Cock 
wrote:

> I think this is the (relatively new) Galaxy ability to automatically
> run N copies of your tool given N input files, making N outputs
> and is related to the collections work.
>
> (This is possible if your tool takes a single input file)
>
> Peter
>
> On Tue, Jan 20, 2015 at 6:17 PM, Vimalkumar Velayudhan
>  wrote:
> > Hi all,
> >
> > I am trying to create a select box with the possibility of selecting
> only a
> > single item from the input dataset (figure 1). This works fine but the
> > option for selecting multiple files is still visible (figure 2). The
> > multiple="false" attribute has no effect.
> >
> > Figure: http://i.imgur.com/oJVFCoF.png
> >
> > I have the following in my XML.
> >
> >  >label="Select Ribo-Seq alignment file" multiple="false" >
> > 
> >
> > Any suggestions?
> >
> > galaxy-dist revision 5f4c13d622b8
> >
> >
> > Regards,
> > Vimalkumar Velayudhan
> >
> > ___
> > Please keep all replies on the list by using "reply all"
> > in your mail client.  To manage your subscriptions to this
> > and other Galaxy lists, please use the interface at:
> >   https://lists.galaxyproject.org/
> >
> > To search Galaxy mailing lists use the unified search at:
> >   http://galaxyproject.org/search/mailinglists/
>
___
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:
  https://lists.galaxyproject.org/

To search Galaxy mailing lists use the unified search at:
  http://galaxyproject.org/search/mailinglists/

Re: [galaxy-dev] Manually installing rsem_datatypes package from toolshed

Re: [galaxy-dev] show stopper for my galaxy installation

[galaxy-dev] show stopper for my galaxy installation

Re: [galaxy-dev] galaxy and torque - resource allocation

Re: [galaxy-dev] galaxy and torque - resource allocation

Re: [galaxy-dev] galaxy and torque - resource allocation

Re: [galaxy-dev] galaxy and torque - resource allocation

Re: [galaxy-dev] galaxy and torque - resource allocation

Re: [galaxy-dev] galaxy and torque - resource allocation

[galaxy-dev] GALAXY_DATA_INDEX_DIR problem

Re: [galaxy-dev] Tool development - Selecting a single item from input dataset.

11 matches

Site Navigation

Mail list logo

Footer information