LimitRequestBody has a max value of 2147483647 (2GB).  You probably want
to set it to zero to disable the limit:
http://httpd.apache.org/docs/current/mod/core.html#limitrequestbody

Also, you may be running into memory limits on the server, especially if
it's running a 32-bit OS (where each process is limited to about 2GB of
memory):
http://stackoverflow.com/questions/1922414/file-upload-limit-in-http


Ilya


On 11/29/12 12:57 PM, "greg" <margeem...@gmail.com> wrote:

>Hmm, no luck on the large uploads.
>
>I tried using a recent version of Google Chrome.
>
>I added these settings to /etc/httpd/conf/httpd.conf
>
>Timeout 6000
>LimitRequestBody 32147483647
>
>And restarted apache.
>
>But when I try to upload a 3.7GB file I get this error in Galaxy:
>
>"Bad Gateway
>
>The proxy server received an invalid response from an upstream server."
>
>
>Now I do see this error in the Galaxy log, so I'm thinking I'm not
>hitting an Apache issue, and I'm not hitting a browser issue.  Perhaps
>it's a paste issue?
>
>
>Exception happened during processing of request from ('127.0.0.1', 52128)
>Traceback (most recent call last):
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.
>py",
>line 1053, in process_request_in_thread
>    self.finish_request(request, client_address)
>  File "/usr/local/python27/lib/python2.7/SocketServer.py", line 323,
>in finish_request
>    self.RequestHandlerClass(request, client_address, self)
>  File "/usr/local/python27/lib/python2.7/SocketServer.py", line 639,
>in __init__
>    self.handle()
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.
>py",
>line 432, in handle
>    BaseHTTPRequestHandler.handle(self)
>  File "/usr/local/python27/lib/python2.7/BaseHTTPServer.py", line
>337, in handle
>    self.handle_one_request()
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.
>py",
>line 427, in handle_one_request
>    self.wsgi_execute()
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.
>py",
>line 287, in wsgi_execute
>    self.wsgi_start_response)
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/urlmap.py",
>line 202, in __call__
>    return app(environ, start_response)
>  File 
>"/misc/local/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/xforwa
>rdedhost.py",
>line 21, in __call__
>    return self.app( environ, start_response )
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/PasteDeploy-1.3.3-py2.7.egg/paste/dep
>loy/config.py",
>line 164, in __call__
>    app_iter = self.application(environ, start_response)
>  File 
>"/misc/local/galaxy/galaxy-dist/lib/galaxy/web/framework/middleware/transl
>ogger.py",
>line 68, in __call__
>    return self.application(environ, replacement_start_response)
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/eval
>exception/middleware.py",
>line 226, in __call__
>    return self.respond(environ, start_response)
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/WebError-0.8a-py2.7.egg/weberror/eval
>exception/middleware.py",
>line 399, in respond
>    if self.xmlhttp_key in req.params:
>  File "build/bdist.linux-x86_64/egg/webob/__init__.py", line 900, in
>params
>    params = self.str_params
>  File "build/bdist.linux-x86_64/egg/webob/__init__.py", line 892, in
>str_params
>    return NestedMultiDict(self.str_GET, self.str_POST)
>  File "build/bdist.linux-x86_64/egg/webob/__init__.py", line 818, in
>str_POST
>    keep_blank_values=True)
>  File "/usr/local/python27/lib/python2.7/cgi.py", line 508, in __init__
>    self.read_multi(environ, keep_blank_values, strict_parsing)
>  File "/usr/local/python27/lib/python2.7/cgi.py", line 632, in read_multi
>    environ, keep_blank_values, strict_parsing)
>  File "/usr/local/python27/lib/python2.7/cgi.py", line 510, in __init__
>    self.read_single()
>  File "/usr/local/python27/lib/python2.7/cgi.py", line 647, in
>read_single
>    self.read_lines()
>  File "/misc/local/galaxy/galaxy-dist/lib/galaxy/web/framework/base.py",
>line 268, in read_lines
>    self.read_lines_to_outerboundary()
>  File "/usr/local/python27/lib/python2.7/cgi.py", line 697, in
>read_lines_to_outerboundary
>    line = self.fp.readline(1<<16)
>  File 
>"/misc/local/galaxy/galaxy-dist/eggs/Paste-1.6-py2.7.egg/paste/httpserver.
>py",
>line 467, in readline
>    data = self.file.readline(self.length - self._consumed)
>  File "/usr/local/python27/lib/python2.7/socket.py", line 412, in
>readline
>    bline = buf.readline(size)
>OverflowError: signed integer is greater than maximum
>
>
>
>
>
>
>On Thu, Nov 29, 2012 at 11:26 AM, greg <margeem...@gmail.com> wrote:
>> This says Google Chrome can support uploads larger than 4 GB.
>> http://www.motobit.com/help/scptutl/pa98.htm
>>
>> Maybe I just need to configure Apache?
>>
>> I'll let you guys know what I find.
>>
>> -Greg
>>
>> On Thu, Nov 29, 2012 at 11:22 AM, greg <margeem...@gmail.com> wrote:
>>> Follow up question.  What's the deal with the 2GB limit it mentions?
>>>
>>> I think it would be pretty convienent to allow larger uploads.  We're
>>> on a fast local network so even 10 or 20 GB uploads pretty fast.
>>>
>>> Can I increase the limit in Apache?  Which browsers are limited?  This
>>> is an internal install so I can tell users to only use certain
>>> browsers for uploads.
>>>
>>> -Greg
>>>
>>> On Thu, Nov 29, 2012 at 11:01 AM, Nate Coraor <n...@bx.psu.edu> wrote:
>>>> On Nov 29, 2012, at 10:54 AM, greg wrote:
>>>>
>>>>> Thanks.  That seemed to have worked.  Can you explain what happened?
>>>>> (I'm curious)
>>>>
>>>> It's due to a bug that we've done quite a bit of work to investigate,
>>>>but haven't yet found  the cause.  It seems to have cropped up more
>>>>frequently in the past few weeks, so we'll be working on it in depth
>>>>in the next few days.
>>>>
>>>> --nate
>>>>
>>>>>
>>>>> -Greg
>>>>>
>>>>> On Thu, Nov 29, 2012 at 8:43 AM, Nate Coraor <n...@bx.psu.edu> wrote:
>>>>>> On Nov 28, 2012, at 1:23 PM, greg wrote:
>>>>>>
>>>>>>> I just finished setting up a local install on our cluster but I'm
>>>>>>> having trouble uploading files.
>>>>>>>
>>>>>>> I was able to (successfully?) upload one 160MB fasta file.  But any
>>>>>>> further uploads I try, they just hang.
>>>>>>
>>>>>> Hi Greg,
>>>>>>
>>>>>> Could you try the following:
>>>>>>
>>>>>> Open tools/data_source/upload.xml
>>>>>>
>>>>>> In the following line, set 'ajax-upload="false"':
>>>>>>
>>>>>> <param name="file_data" type="file" size="30" label="File"
>>>>>>ajax-upload="false" help="TIP: Due to browser limitations, uploading
>>>>>>files larger than 2GB is guaranteed to fail.  To upload large files,
>>>>>>use the URL method (below) or FTP (if enabled by the site
>>>>>>administrator).">
>>>>>>
>>>>>> Then restart Galaxy and retry the upload and let us know if it
>>>>>>succeeds.
>>>>>>
>>>>>> --nate
>>>>>>
>>>>>>>
>>>>>>> Here's the information from the log file:
>>>>>>>
>>>>>>> paste.httpserver.ThreadPool INFO 2012-11-28 12:15:18,190
>>>>>>> kill_hung_threads status: 10 threads (0 working, 10 idle, 0
>>>>>>>starting)
>>>>>>> ave time N/A, max time 0.00sec, killed 0 workers
>>>>>>> galaxy.tools.actions.upload_common INFO 2012-11-28 12:20:36,597
>>>>>>>tool
>>>>>>> upload1 created job id 4
>>>>>>> galaxy.jobs.handler INFO 2012-11-28 12:20:38,386 (4) Job dispatched
>>>>>>> galaxy.jobs.runners.drmaa INFO 2012-11-28 12:20:38,930 (4) queued
>>>>>>>as 3718641
>>>>>>> galaxy.objectstore CRITICAL 2012-11-28 12:21:03,071
>>>>>>> /home/galaxy/job_working_directory/000/4 delete error [Errno 39]
>>>>>>> Directory not empty: '/home/galaxy/job_working_directory/000/4'
>>>>>>> paste.httpserver.ThreadPool INFO 2012-11-28 12:21:55,763
>>>>>>> kill_hung_threads status: 10 threads (0 working, 10 idle, 0
>>>>>>>starting)
>>>>>>> ave time N/A, max time 0.00sec, killed 0 workers
>>>>>>> paste.httpserver.ThreadPool INFO 2012-11-28 12:28:26,466
>>>>>>> kill_hung_threads status: 10 threads (0 working, 10 idle, 0
>>>>>>>starting)
>>>>>>> ave time N/A, max time 0.00sec, killed 0 workers
>>>>>>> paste.httpserver.ThreadPool INFO 2012-11-28 12:35:09,662
>>>>>>> kill_hung_threads status: 10 threads (0 working, 10 idle, 0
>>>>>>>starting)
>>>>>>> ave time N/A, max time 0.00sec, killed 0 workers
>>>>>>> paste.httpserver.ThreadPool INFO 2012-11-28 12:46:35,565
>>>>>>> kill_hung_threads status: 10 threads (0 working, 10 idle, 0
>>>>>>>starting)
>>>>>>> ave time N/A, max time 0.00sec, killed 0 workers
>>>>>>>
>>>>>>>
>>>>>>> The first upload that worked was around 12:20 (above), and the
>>>>>>>second
>>>>>>> failed one was at 12:40.  The log file doesn't seem to even mention
>>>>>>> it? Could that error have killed the server somehow?
>>>>>>>
>>>>>>> Let me know if I can provide more info.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Greg
>>>>>>> ___________________________________________________________
>>>>>>> Please keep all replies on the list by using "reply all"
>>>>>>> in your mail client.  To manage your subscriptions to this
>>>>>>> and other Galaxy lists, please use the interface at:
>>>>>>>
>>>>>>> http://lists.bx.psu.edu/
>>>>>>
>>>>>
>>>>
>
>___________________________________________________________
>Please keep all replies on the list by using "reply all"
>in your mail client.  To manage your subscriptions to this
>and other Galaxy lists, please use the interface at:
>
>  http://lists.bx.psu.edu/


___________________________________________________________
Please keep all replies on the list by using "reply all"
in your mail client.  To manage your subscriptions to this
and other Galaxy lists, please use the interface at:

  http://lists.bx.psu.edu/

Reply via email to