Ian and Jakub,
I also noticed a different large file problem, on Linux: 2.6.16,
Python: 2.5, Pylons: 0.9.6. When a large file upload failed, the
server thread handling the request would hang. I was able to
mitigate this problem with the following patch to Paste. How should
this be fixed? Note, I didn't see this problem on OS X.
Thanks,
Casey
Index: cascade.py
===================================================================
--- cascade.py (revision 7229)
+++ cascade.py (working copy)
@@ -91,6 +91,8 @@
copy_len = length
while copy_len > 0:
chunk = environ['wsgi.input'].read(min
(copy_len, 4096))
+ if len(chunk) == 0:
+ raise IOError("Fileobject read
returned zero bytes.")
f.write(chunk)
copy_len -= len(chunk)
f.seek(0)
On Feb 12, 2008, at 9:02 AM, Ian Bicking wrote:
>
> Thanks, that looks right to me. Applied in trunk.
>
> Jakub Stolarski wrote:
>> On 11 Lut, 12:41, Jakub Stolarski <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> I have to upload huge files (up to 200MB) through Pylons and save it
>>> on disk. When I read request.params.keys memory consumption
>>> raises up
>>> to 250MB with 100MB uploaded file (then it generally falls). Is
>>> there
>>> any method to keep memory usage low?
>>>
>>> Python 2.4/2.5
>>> Pylons 0.9.6.1
>>> Run with egg:Paste#http
>>>
>>> Thanks in advance
>>
>> I think I have found a problem. In paste.httpserver
>> LimitedLengthFile.readline does not use 'size' parameter, which is
>> used in cgi.FieldStorage. Here is a patch (aginst trunk). It would be
>> nice if someone could confirm that bug.
>>
>>
>> --- httpserver.py.old Mon Feb 11 16:18:42 2008
>> +++ httpserver.py Mon Feb 11 16:19:37 2008
>> @@ -464,7 +464,10 @@
>> return data
>>
>> def readline(self, *args):
>> - data = self.file.readline(self.length - self._consumed)
>> + max_read = self.length - self._consumed
>> + if len(args):
>> + max_read = min(args[0], max_read
>> + data = self.file.readline(max_read)
>> self._consumed += len(data)
>> return data
>>
>>>
>
>
> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google
> Groups "pylons-discuss" group.
> To post to this group, send email to [email protected]
> To unsubscribe from this group, send email to pylons-discuss-
> [EMAIL PROTECTED]
> For more options, visit this group at http:
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---
