On Mar 6, 11:53 am, Graham Dumpleton <[email protected]>
wrote:
> 2009/3/6 Brian Sutherland <[email protected]>:
>
> > I've managed to make a minimal wsgi application that shows off a
> > problem we encountered in production. It looks like mod_wsgi is
> > leaking the temporary files we give to it. Perhaps we are doing
> > something wrong, but I couldn't find any docs against passing
> > tempfile.mkstemp files to mod_wsgi. Curiously, creating the file with
> > tempfile.TemporaryFile rather than tempfile.mkstemp has allowed us to
> > work around the issue.
>
> > Basically running ab against this application:
>
> > #!/usr/bin/python
> > import tempfile
> > import os
>
> > def application(environ, start_response):
> > status = '200 OK'
> > output = 'Hello World!'
>
> > response_headers = [('Content-type', 'text/plain'),
> > ('Content-Length', str(len(output)))]
>
> > start_response(status, response_headers)
>
> > # Make a temporary file and write data to it
> > _, filename = tempfile.mkstemp()
>
> This returns a file descriptor which you are assigning to '_' variable
> but you aren't closing it. This means that when it is closed is at the
> mercy of the Python garbage collector. They thus aren't being closed
> promptly and you are running out of file descriptors.
The problem is that they are never closed, after hours, even while
other requests succeed. To try discount your idea I changed my test
script to this, and got the same result, after ~1000 requests it
starts failing:
#!/usr/bin/python
import tempfile
import os
import gc
def application(environ, start_response):
status = '200 OK'
output = 'Hello World!'
response_headers = [('Content-type', 'text/plain'),
('Content-Length', str(len(output)))]
start_response(status, response_headers)
# Make a temporary file and write data to it
filedesc, filename = tempfile.mkstemp()
del filedesc
gc.collect()
f = open(filename, 'wb')
f.write(output)
f.close()
# Make a deleted read-only file to pass to mod_wsgi
readf = open(filename, 'rb')
os.remove(filename)
wrapper = environ.get('wsgi.file_wrapper')
return wrapper(readf)
> > f = open(filename, 'wb')
> > f.write(output)
> > f.close()
>
> > # Make a deleted read-only file to pass to mod_wsgi
> > readf = open(filename, 'rb')
> > os.remove(filename)
>
> You probably want to use something like:
>
> f, filename = tempfile.mkstemp()
> f.write(output)
> f.flush()
> f.seek(0)
>
> Then use 'f' as argument to file wrapper.
Doesn't seem to work, f is an "OS-level handle to an open file (as
would be returned by os.open())":
>>> import tempfile
>>> f,filename = tempfile.mkstemp()
>>> f.write('bobo')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'int' object has no attribute 'write'
>
> So, my questions about MPM and mode aren't relevant.
>
> Graham
>
> > wrapper = environ.get('wsgi.file_wrapper')
> > return wrapper(readf)
>
> > Causes these errors:
>
> > [Fri Mar 06 09:50:16 2009] [error] [client 62.57.1.12] mod_wsgi
> > (pid=18665): Exception occurred processing WSGI script '/etc/apache2/
> > wsgi/wsgitest.wsgi'.
> > [Fri Mar 06 09:50:16 2009] [error] Traceback (most recent call last):
> > [Fri Mar 06 09:50:16 2009] [error] File "/etc/apache2/wsgi/
> > wsgitest.wsgi", line 15, in application
> > [Fri Mar 06 09:50:16 2009] [error] File "tempfile.py", line 302, in
> > mkstemp
> > [Fri Mar 06 09:50:16 2009] [error] File "tempfile.py", line 236, in
> > _mkstemp_inner
> > [Fri Mar 06 09:50:16 2009] [error] OSError: [Errno 24] Too many open
> > files: '/tmp/tmp3zONgs'
>
> > Because you very quickly get to the per-process file limit:
>
> > load:/etc/apache2/wsgi# ls /proc/18665/fd | wc -l
> > 1023
>
> > We're running python 2.4.4, apache 2.2.3-4+etch6 on Debian Etch and
> > have seen this with mod_wsgi 2.1 and 2.3.
>
> > Many Thanks,
> > Brian
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en
-~----------~----~----~----~------~----~------~--~---
