BTW, if I am right, you would see the behaviour you expect to see if you use:
WSGIApplicationGroup %{GLOBAL}
That is, force your application to run in main Python interpreter
instance within process.
This is because when using main interpreter mod_wsgi uses simplified
API for GIL so as to be compatible with third party C extension
modules that do the same. In that case Python internally manages the
thread state objects and so they get reused rather than fresh one
created each time.
Graham
2008/10/3 Graham Dumpleton <[EMAIL PROTECTED]>:
> Hmmmm, I think I know why.
>
> It is very interesting that no one has noticed or commented on this
> behaviour before, although I also suspect I know why.
>
> In mod_wsgi at the C code level, a new thread state object is created
> for each request, with it being destroyed at the end of the request.
> The threading.local() object is actually attached to this C thread
> state object, but because the thread state object is destroyed at the
> end of the request, so also is the threading.local() object.
>
> Thus, for all intents and purposes, and as far as threading.local() is
> concerned, to the application it looks like a brand new Python thread
> instance is used for every request, even though the same foreign
> thread, created outside of Python, can be used for multiple requests.
>
> The reason no one has probably noticed is that normally people would
> only want a threading.local() instance to exist for the life time of a
> request. Frameworks should even deliberately delete attributes added
> to the threading.local() object at the end of the request to ensure it
> is cleared of anything.
>
> FWIW, the way mod_wsgi works in this respect is actually the same as
> mod_python. One would hope that no application is dependent on this
> quirk of mod_python/mod_wsgi and they do always cleanup
> threading.local() attributes at the end of the request. If they don't,
> then when hosting on native Python web server, or fastcgi etc, the
> code wouldn't necessarily work as threading.local() instance could be
> polluted with attributes from previous request.
>
> Anyway, this behaviour would be an issue for code which is trying to
> use threading.local() as a shortcut to avoid having to do their own
> thead mutex locking etc.
>
> The thing is though, although mod_wsgi behaviour could be changed,
> will still occur in mod_python. Also, there would be no guarantee in
> any WSGI hosting mechanism that threads would be reused time and time
> again to handle requests, so use of threading.local() in the way you
> want may not be reliable anyway. I believe for example that CherryPy
> will keep a pool of threads and may at its discretion create
> additional threads if it thinks it needs more and then later destroy
> threads if it thinks it has more than it needs. Thus in CherryPy as
> well, although threads may get reused for a while, I think they also
> may disappear and be replaced with new threads.
>
> In summary, if one looks across different WSGI hosting mechanism,
> given that the WSGI application object is the entry point for a
> request and one cannot know how that will come to be called, I don't
> think you could rely on data keyed to thread instance being preserved
> across requests, with threading.local(). You also wouldn't want to key
> itself based on the thread ID either as this may not be a fixed set
> and if there is no way to cleanup when a thread is destroyed data from
> anything keyed on thread ID, your application memory would just keep
> growing with junk.
>
> That all said, what are other peoples opinions on this? Yes I could
> attempt to change code to remember thread state objects for Apache
> foreign threads, since the number of threads is always constant and
> they aren't deleted until process being killed, but would making this
> change such that threading.local() persists across requests cause
> problems for code out there? Leaving the behaviour as is, may actually
> be better as it may protect better against applications which aren't
> written to clear per thread state properly.
>
> Comments?
>
> Graham
>
> 2008/10/3 Graham Dumpleton <[EMAIL PROTECTED]>:
>> One possibility is that although you think you are using daemon mode,
>> you aren't, and that Apache compiled with worker MPM. This can occur
>> if you don't have WSGIProcessGroup directive set properly to refer to
>> daemon process group setup using WSGIDaemonProcess.
>>
>> Post the mod_wsgi bits of the Apache configuration, include any
>> VirtualHost, Directory, Location context they are defined in.
>>
>> In your per request debug also print out:
>>
>> print >> sys.stderr, environ.get('mod_wsgi.process_group')
>> print >> sys.stderr, environ.get('mod_wsgi.application_group')
>>
>> print >> sys.stderr, environ.get('wsgi.multithread')
>> print >> sys.stderr, environ.get('wsgi.multiprocess')
>>
>> If mod_wsgi.process_group prints out as empty string, you are running
>> in embedded mode. If multithread is true shows that running in
>> multithreaded process and not single threaded process as you expect
>> and thus also why result may be different each time.
>>
>> You could also print out:
>>
>> print >> sys.stderr, threading.currentThread()
>>
>> to show what Python thinks the thread is each time.
>>
>> Graham
>>
>>
>>
>> 2008/10/3 William Dode <[EMAIL PROTECTED]>:
>>>
>>> Maybe more clear :
>>>
>>> #!/usr/bin/python
>>> import cgi
>>> import threading
>>> import sys
>>> import os
>>>
>>> class MyLoc(threading.local):
>>> def __init__(self):
>>> print >> sys.stderr , 'init'
>>>
>>> myloc = MyLoc()
>>>
>>> def application(environ, start_response):
>>> status = '200 OK'
>>> myloc.i = 0
>>> print >> sys.stderr, 'pid: %s id: %s id.__dict__: %s' % (os.getpid(),
>>> id(myloc), id(myloc.__dict__))
>>> output = ''
>>> response_headers = [('Content-type', 'text/html'), ]
>>> start_response(status, response_headers)
>>>
>>> return [cgi.escape(output)]
>>>
>>> [Fri Oct 03 11:37:04 2008] [error] init
>>> [Fri Oct 03 11:37:04 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>>> 3067884244
>>> [Fri Oct 03 11:37:04 2008] [error] init
>>> [Fri Oct 03 11:37:04 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>>> 3067885740
>>> [Fri Oct 03 11:37:05 2008] [error] init
>>> [Fri Oct 03 11:37:05 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>>> 3067884244
>>> [Fri Oct 03 11:37:05 2008] [error] init
>>> [Fri Oct 03 11:37:05 2008] [error] pid: 1836 id: 3067827220 id.__dict__:
>>> 3067885740
>>>
>>> init should not be called and __dict__ should not be different
>>>
>>> The same with wsgiref :
>>> from wsgiref.simple_server import make_server
>>> make_server('',8080,application).serve_forever()
>>>
>>> init
>>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>>> pid: 1868 id: 3082036284 id.__dict__: 3082022260
>>>
>>> I'll look if i can reproduce this with the code of _threading_local
>>>
>>>
>>> --
>>> William Dodé - http://flibuste.net
>>> Informaticien Indépendant
>>>
>>>
>>> >>>
>>>
>>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en
-~----------~----~----~----~------~----~------~--~---
