On Mon, Dec 7, 2009 at 5:35 PM, Mark Hammond <mhamm...@skippinet.com.au> wrote: > On 8/12/2009 9:45 AM, Preston Landers wrote: >> >> Someone had mentioned needing to restrict the worker pool to a single >> thread but indicated they didn't know how to do that. I searched >> around MSDN and couldn't find anything relevant. I also checked the >> PyWin32 ISAPI code and didn't see anything that appeared to start a >> new thread. Does anyone know how to force single threading in this >> scenario? > > Finding out where the thread was created should be your first goal. You are > correct that pywin32's ISAPI support doesn't create threads except when > establishing that thread-pool - which only happens when you explicitly > sub-class that threaded implementation.
As far as I can tell the threads are created directly by the ISAPI framework without reference to any Python/ISAPI code. They just make a new C thread then call the extension function without further ado. I tried to dig into it a little deeper by compiling a debug version of Python/PyWin32 (x64) but ran into some problems. Everything worked great with my debug build from the command line, including PyWin32 stuff and installing the ISAPI vdir. But when it came to actually running the code through IIS I got mysterious 500 server errors. They appeared very similar to the errors I got when I tried to use a 32 bit version of Python/ISAPI with 64 bit IIS despite setting the 32 bit app pool flag. (By the way, it might be good to note somewhere in the docs that you can't do that, if it doesn't already.) I think it's a generic "DLL could not load" type error but even with the IIS error tracing feature it wasn't very clear. I thought that maybe you also need a debug build of Windows/IIS itself for that to work, but that doesn't sound right. It's possible that since I'm running that debug build right out of the source dir I failed to register it properly or something. Anyway, even if I could identify where the thread was getting created and insert some code there I'm not even sure what it would do. I think forcing each thread to have a different Python interpreter might help. Some other Python web server packages like mod_python seems to do this. But for now I've opted for a simple lock in my WSGI handler function to bypass the whole issue. But I'm a little curious about the purpose of having a WSGI thread pool implementation in light of those facts. If ISAPI will create threads for you anyway, how does a pool help? Is it purely to avoid thread startup overhead? If a thread is being created anyway, you're just adding it to a queue to be handled by a separate thread from the pool. Seems like more overhead, not less. But I'm probably missing something there. Nor have I done a hands on performance test. > > The other option is just to use a lock - create a global lock object and > have every request acquire it and release it when done. You should then > find only 1 thread is ever actually running in your code while the rest are > blocked waiting for the lock. As noted above though, this may severely > impact performance... > Yeah I went ahead and used a lock and it does fully resolve the issue. I'm hoping to avoid or at least mitigate any performance impact by having multiple worker processes configured automatically. IIS does seem to distribute the load pretty evenly in most cases. It will be a while before I have time to run some serious performance / stress tests, but when I do I will be testing a number of different scenarios. Even with the lock, so far performance seems far better than the prior solution we have, which is a byzantine combination of "Classic ASP" Python calling out to external app servers over a proprietary FastCGI-like socket protocol. Anyway, thanks for your input on this issue and all of your other excellent work. It's much appreciated. -Preston _______________________________________________ python-win32 mailing list python-win32@python.org http://mail.python.org/mailman/listinfo/python-win32
