Hmmmm, I think I know why.
It is very interesting that no one has noticed or commented on this
behaviour before, although I also suspect I know why.
In mod_wsgi at the C code level, a new thread state object is created
for each request, with it being destroyed at the end of the request.
The threading.local() object is actually attached to this C thread
state object, but because the thread state object is destroyed at the
end of the request, so also is the threading.local() object.
Thus, for all intents and purposes, and as far as threading.local() is
concerned, to the application it looks like a brand new Python thread
instance is used for every request, even though the same foreign
thread, created outside of Python, can be used for multiple requests.
The reason no one has probably noticed is that normally people would
only want a threading.local() instance to exist for the life time of a
request. Frameworks should even deliberately delete attributes added
to the threading.local() object at the end of the request to ensure it
is cleared of anything.
FWIW, the way mod_wsgi works in this respect is actually the same as
mod_python. One would hope that no application is dependent on this
quirk of mod_python/mod_wsgi and they do always cleanup
threading.local() attributes at the end of the request. If they don't,
then when hosting on native Python web server, or fastcgi etc, the
code wouldn't necessarily work as threading.local() instance could be
polluted with attributes from previous request.
Anyway, this behaviour would be an issue for code which is trying to
use threading.local() as a shortcut to avoid having to do their own
thead mutex locking etc.
The thing is though, although mod_wsgi behaviour could be changed,
will still occur in mod_python. Also, there would be no guarantee in
any WSGI hosting mechanism that threads would be reused time and time
again to handle requests, so use of threading.local() in the way you
want may not be reliable anyway. I believe for example that CherryPy
will keep a pool of threads and may at its discretion create
additional threads if it thinks it needs more and then later destroy
threads if it thinks it has more than it needs. Thus in CherryPy as
well, although threads may get reused for a while, I think they also
may disappear and be replaced with new threads.
In summary, if one looks across different WSGI hosting mechanism,
given that the WSGI application object is the entry point for a
request and one cannot know how that will come to be called, I don't
think you could rely on data keyed to thread instance being preserved
across requests, with threading.local(). You also wouldn't want to key
itself based on the thread ID either as this may not be a fixed set
and if there is no way to cleanup when a thread is destroyed data from
anything keyed on thread ID, your application memory would just keep
growing with junk.
That all said, what are other peoples opinions on this? Yes I could
attempt to change code to remember thread state objects for Apache
foreign threads, since the number of threads is always constant and
they aren't deleted until process being killed, but would making this
change such that threading.local() persists across requests cause
problems for code out there? Leaving the behaviour as is, may actually
be better as it may protect better against applications which aren't
written to clear per thread state properly.
Comments?
Graham
2008/10/3 Graham Dumpleton <[EMAIL PROTECTED]>:
> One possibility is that although you think you are using daemon mode,
> you aren't, and that Apache compiled with worker MPM. This can occur
> if you don't have WSGIProcessGroup directive set properly to refer to
> daemon process group setup using WSGIDaemonProcess.
>
> Post the mod_wsgi bits of the Apache configuration, include any
> VirtualHost, Directory, Location context they are defined in.
>
> In your per request debug also print out:
>
> print >> sys.stderr, environ.get('mod_wsgi.process_group')
> print >> sys.stderr, environ.get('mod_wsgi.application_group')
>
> print >> sys.stderr, environ.get('wsgi.multithread')
> print >> sys.stderr, environ.get('wsgi.multiprocess')
>
> If mod_wsgi.process_group prints out as empty string, you are running
> in embedded mode. If multithread is true shows that running in
> multithreaded process and not single threaded process as you expect
> and thus also why result may be different each time.
>
> You could also print out:
>
> print >> sys.stderr, threading.currentThread()
>
> to show what Python thinks the thread is each time.
>
> Graham
>
>
>
> 2008/10/3 William Dode <[EMAIL PROTECTED]>:
>>
>> Maybe more clear :
>>
>> #!/usr/bin/python
>> import cgi
>> import threading
>> import sys
>> import os
>>
>> class MyLoc(threading.local):
>> def __init__(self):
>> print >> sys.stderr , 'init'
>>
>> myloc = MyLoc()
>>
>> def application(environ, start_response):
>> status = '200 OK'
>> myloc.i = 0
>> print >> sys.stderr, 'pid: %s id: %s id.__dict__: %s' % (os.getpid(),
>> id(myloc), id(myloc.__dict__))
>> output = ''
>> response_headers = [('Content-type', 'text/html'), ]
>> start_response(status, response_headers)
>>
>> return [cgi.escape(output)]
>>
>> [Fri Oct 03 11:37:04 2008] [error] init
>> [Fri Oct 03 11:37:04 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>> 3067884244
>> [Fri Oct 03 11:37:04 2008] [error] init
>> [Fri Oct 03 11:37:04 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>> 3067885740
>> [Fri Oct 03 11:37:05 2008] [error] init
>> [Fri Oct 03 11:37:05 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>> 3067884244
>> [Fri Oct 03 11:37:05 2008] [error] init
>> [Fri Oct 03 11:37:05 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>> 3067885740
>>
>> init should not be called and __dict__ should not be different
>>
>> The same with wsgiref :
>> from wsgiref.simple_server import make_server
>> make_server('',8080,application).serve_forever()
>>
>> init
>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>>
>> I'll look if i can reproduce this with the code of _threading_local
>>
>>
>> --
>> William Dodé - http://flibuste.net
>> Informaticien Indépendant
>>
>>
>> >>
>>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en
-~----------~----~----~----~------~----~------~--~---
