I don't have any kind of answer, but I've had a similar thing come
up, so I'll pass it on, in case it helps narrow the focus:
- In my dev environment (Webware 0.8.1 / Apache 2 / Mac OS X 10.4),
when I do large file uploads, it freezes the entire webware process
Works fine in the production environment, so I've been lazy about
tracking it down, but it's a particularly nasty freeze. I have to
kill -9 the process to get it to die. Which does have the flavor of
some sort of GIL issue.
"Large" seems to be "large enough that the browser doesn't send it as
a single POST" (which it seems to do for small files)
Also, you're not going to be able to get a meaningful progress check
the way you're trying to. The file upload lifecycle works as follows:
- Apache gets the beginning of the huge request
- It starts forwarding it to the Webware process
- The webware proc uses the cgi library to parse out the values
- The cgi library creates a temp file for the upload and fills it
as the rest of the request is pulled from Apache
- Once it's done getting *all* the data from Apache, Webware then
hands control over to your servlet
- At that point, when you do things like file.value, it gets read
out of that temp file
The slow part is happening in the cgi.py library, before your servlet
is called. Your servlet is just copying from the temp file, which is
fast.
You *can* subclass cgi.py and play some games to get a progress check
as the temp file is filled (cgi.py is designed to let you override
the temp file creation, so you can instrument that, basically).
-Dan
On Sep 12, 2006, at 2:06 PM, Gary Perez wrote:
>
> On Sep 12, 2006, at 12:35 PM, Chuck Esterbrook wrote:
>
>> On 9/12/06, Gary Perez <[EMAIL PROTECTED]> wrote:
>>> On Sep 8, 2006, at 8:34 AM, sophana wrote:
>>>
>>>> Looking at the error messages, it seems that it is webware that
>>>> don't
>>>> accept the connection.
>>>> I don't know why, and don't even know if I have the same problem.
>>>> Are you sure that your second request is not blocked by the first
>>>> one in
>>>> webware?
>>>
>>> How would I be able to determine whether the second request is being
>>> blocked? If the simple answer is "because the second request doesn't
>>> get served", then yeah, it's being blocked... but why, and is
>>> there a
>>> way around this?
>>>
>>> I admit I don't know a lot about threads/threading, but I was under
>>> the impression that the AppServer (as config'd below) could handle
>>> multiple, simultaneous requests.
>>>
>>> Is there something completely obvious that I'm overlooking?
>>
>> Does anyone think this could be Python's GIL kicking in? Perhaps the
>> first thread has acquired the GIL and is not letting it go?
>
> GIL (global interpreter lock) - I'm *very* unqualified to address
> this.
>
>> Is the first thread using any extension modules (Python modules
>> written in C instead of Python)?
>
> No sir. From the first thread (form processor):
>
> from Template import Template
> import vtools, os.path
>
> ... where "vtools" is an external .py module that's written entirely
> in Python (no C).
>
> The 2nd request is also a python-only module that simply does a
> glob.glob('*') on the pwd.
>
>> What does it do with all the form
>> data being uploaded?
>> -Chuck
>
>
> At the moment, it simply does a bit of form checking (e.g., did the
> user select a file to upload), then writes the file.value from the
> form data, as such:
>
> filename, contents = file.filename, file.value
> open(os.path.join(PROJDIR + pdir, filename), 'wb').write(contents)
>
> As I previously stated, I was going to try to figure out a way to
> write the file data in chunks (?) so I could provide some type of
> status display to the user doing the uploading, but that's a problem
> for a later time...
>
> -Gary
>
> ----------------------------------------------------------------------
> ---
> Using Tomcat but need to do more? Need to support web services,
> security?
> Get stuff done quickly with pre-integrated technology to make your
> job easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> http://sel.as-us.falkag.net/sel?
> cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/webware-discuss
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Webware-discuss mailing list
Webware-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/webware-discuss
