RE: [Zope] Multithreading sessions
Dieter, Thanks for this input. Your comments here turned out to be dead on. The problem is the GIL. We are currently running Sybase databases using the python Sybase-0.37 module for data access. This module will set the GIL. So if the query takes any time at all, every other session will freeze. It does release it when the connection is closed, but if the query takes awhile it effectively denies service to every other session. I am just learning about this, but my initial inquiries suggest that the only way to achieve true concurrency using a language like python is to launch multiple interpreters. I don't yet have the solution to my problem, but at least now I know what the problem is, and that is half the battle. Thanks again, and thanks to everyone's input, it has helped immensely. Jon Emmons -Original Message- From: Dieter Maurer [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 28, 2008 2:27 PM To: Jon Emmons Cc: zope@zope.org Subject: Re: [Zope] Multithreading sessions Jon Emmons wrote at 2008-5-23 08:58 -0400: ... I am running zope 2.9.4 and have observed that it will not simultaneously serve pages to my users. Usually, it does. I have seen database adapter packages (an old psycopg version, to be precise) that forgot to release the GIL for some operations. Then Zope freezes while these operations are executed. A surprising report of a similar kind was: while Zope's embedded profiler is in use, Zope effectively runs in single thread mode (such that other threads do not distort the timing results). Probably not your problem ... but who knows. Further potential reasons (for almost surely not responsible for your current problem: Expensive operations implemented in C (such as operations on huge strings) Creating excessive amounts of objects (causing lots of generation 2 garbage collections which hold the GIL) -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: RE: [Zope] Multithreading sessions
We are using Plone and Zope against a legacy Ingres database, and I found a similar problem using the adapter provided. I found that the C library modules (open source) actually contained calls to make the adapter work as we wanted, much like our other web interfaces which maintain the db and user permissions info as a connection, but close the actual query after each. I had to modify the Python Zope product to set autocommit and call the desired close in order to get the behaviour we were looking for -- keeping the connection, but releasing each session so that the DB backend does not register an active session between actual queries. That's open source. Code terminology for re-write it. - Original Message - From: Jon Emmons [EMAIL PROTECTED] Date: Wednesday, June 4, 2008 8:50 Subject: RE: [Zope] Multithreading sessions To: 'Dieter Maurer' [EMAIL PROTECTED] Cc: zope@zope.org Dieter, Thanks for this input. Your comments here turned out to be dead on. The problem is the GIL. We are currently running Sybase databases using the python Sybase-0.37 module for data access. This module will set the GIL. So if the query takes any time at all, every other session will freeze. It does release it when the connection is closed, but if the query takes awhile it effectively denies service to every other session. I am just learning about this, but my initial inquiries suggest that the only way to achieve true concurrency using a language like python is to launch multiple interpreters. I don't yet have the solution to my problem, but at least now I know what the problem is, and that is half the battle. Thanks again, and thanks to everyone's input, it has helped immensely. Jon Emmons -Original Message- From: Dieter Maurer [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 28, 2008 2:27 PM To: Jon Emmons Cc: zope@zope.org Subject: Re: [Zope] Multithreading sessions Jon Emmons wrote at 2008-5-23 08:58 -0400: ... I am running zope 2.9.4 and have observed that it will not simultaneouslyserve pages to my users. Usually, it does. I have seen database adapter packages (an old psycopg version, to be precise) that forgot to release the GIL for some operations. Then Zope freezes while these operations are executed. A surprising report of a similar kind was: while Zope's embedded profiler is in use, Zope effectively runs in single thread mode (such that other threads do not distort the timing results). Probably not your problem ... but who knows. Further potential reasons (for almost surely not responsible for your current problem: Expensive operations implemented in C (such as operations on huge strings) Creating excessive amounts of objects (causing lots of generation 2 garbage collections which hold the GIL) -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: RE: [Zope] Multithreading sessions
Carol, As a quick fix, we have created a stand alone program that does the database access. So from the python external method (zope) that retrieves data, instead of making the connections to the DB directly it now calls os.popen(python getdata.py). This forces the launch of a separate python interpreter for the duration of the query. Unfortunately, we will suffer the overhead of doing this for each DB call, but that is far better than denying service to all other sessions. We are already in the process of moving to MySQL for our databases, and the MySQL adapter does not appear to exhibit this behavior. Jon Emmons _ From: Carol Ludwig [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 04, 2008 10:58 AM To: Jon Emmons Cc: 'Dieter Maurer'; zope@zope.org Subject: Re: RE: [Zope] Multithreading sessions We are using Plone and Zope against a legacy Ingres database, and I found a similar problem using the adapter provided. I found that the C library modules (open source) actually contained calls to make the adapter work as we wanted, much like our other web interfaces which maintain the db and user permissions info as a connection, but close the actual query after each. I had to modify the Python Zope product to set autocommit and call the desired close in order to get the behaviour we were looking for -- keeping the connection, but releasing each session so that the DB backend does not register an active session between actual queries. That's open source. Code terminology for re-write it. - Original Message - From: Jon Emmons [EMAIL PROTECTED] Date: Wednesday, June 4, 2008 8:50 Subject: RE: [Zope] Multithreading sessions To: 'Dieter Maurer' [EMAIL PROTECTED] Cc: zope@zope.org Dieter, Thanks for this input. Your comments here turned out to be dead on. The problem is the GIL. We are currently running Sybase databases using the python Sybase-0.37 module for data access. This module will set the GIL. So if the query takes any time at all, every other session will freeze. It does release it when the connection is closed, but if the query takes awhile it effectively denies service to every other session. I am just learning about this, but my initial inquiries suggest that the only way to achieve true concurrency using a language like python is to launch multiple interpreters. I don't yet have the solution to my problem, but at least now I know what the problem is, and that is half the battle. Thanks again, and thanks to everyone's input, it has helped immensely. Jon Emmons -Original Message- From: Dieter Maurer [mailto:[EMAIL PROTECTED] Sent: Wednesday, May 28, 2008 2:27 PM To: Jon Emmons Cc: zope@zope.org Subject: Re: [Zope] Multithreading sessions Jon Emmons wrote at 2008-5-23 08:58 -0400: ... I am running zope 2.9.4 and have observed that it will not simultaneouslyserve pages to my users. Usually, it does. I have seen database adapter packages (an old psycopg version, to be precise) that forgot to release the GIL for some operations. Then Zope freezes while these operations are executed. A surprising report of a similar kind was: while Zope's embedded profiler is in use, Zope effectively runs in single thread mode (such that other threads do not distort the timing results). Probably not your problem ... but who knows. Further potential reasons (for almost surely not responsible for your current problem: Expensive operations implemented in C (such as operations on huge strings) Creating excessive amounts of objects (causing lots of generation 2 garbage collections which hold the GIL) -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Jon Emmons wrote at 2008-6-4 08:50 -0400: ... I am just learning about this, but my initial inquiries suggest that the only way to achieve true concurrency using a language like python is to launch multiple interpreters. This is true when you mean by true concurrency can keep a multi-CPU systems busy. One a single CPU system, you get almost the concurrency which is possible (modulo bugs such as holding the GIL during non Python related possible expensive operations). I don't yet have the solution to my problem, but at least now I know what the problem is, and that is half the battle. One option is to fix the Sybase access module, another one to use SQLRelay, a third one to access the Sybase database via ODBC. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Dieter, A fourth option that we came up with this morning is to make the data access procedure a stand alone program. We spawn a process at the OS level to query the data and pipe the results back into the zope external method. This launches a new python interpreter and completely sidesteps thread locking issues. Is this a graceful solution? No. Was it very quick to implement and required minimal changes to our code base? Yes. Initial testing has shown that we no longer have sessions freezing up if another session runs a long query. Thanks again, Jon Emmons -Original Message- From: Dieter Maurer [mailto:[EMAIL PROTECTED] Sent: Wednesday, June 04, 2008 2:09 PM To: Jon Emmons Cc: zope@zope.org Subject: RE: [Zope] Multithreading sessions Jon Emmons wrote at 2008-6-4 08:50 -0400: ... I am just learning about this, but my initial inquiries suggest that the only way to achieve true concurrency using a language like python is to launch multiple interpreters. This is true when you mean by true concurrency can keep a multi-CPU systems busy. One a single CPU system, you get almost the concurrency which is possible (modulo bugs such as holding the GIL during non Python related possible expensive operations). I don't yet have the solution to my problem, but at least now I know what the problem is, and that is half the battle. One option is to fix the Sybase access module, another one to use SQLRelay, a third one to access the Sybase database via ODBC. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
- Original Message - From: Dieter Maurer [EMAIL PROTECTED] Jon Emmons wrote at 2008-6-4 08:50 -0400: ... I am just learning about this, but my initial inquiries suggest that the only way to achieve true concurrency using a language like python is to launch multiple interpreters. This is true when you mean by true concurrency can keep a multi-CPU systems busy. One a single CPU system, you get almost the concurrency which is possible (modulo bugs such as holding the GIL during non Python related possible expensive operations). I don't yet have the solution to my problem, but at least now I know what the problem is, and that is half the battle. One option is to fix the Sybase access module, another one to use SQLRelay, a third one to access the Sybase database via ODBC. 4-th option is to use MSSQL DA with threads, it use sybase as database server (and file interfaces as redirector to other MSSQL/sybase DB). It works very well. It is quite old, but it works with 2.9.4 too (newer Zope server I does not use). http://zope.org/Members/thinmanj74/Z%20MS%20SQL%20Database%20Adapterhttp://www.object-craft.com.au/projects/mssql/ http://mail.zope.org/pipermail/zope-db/2004-May/003067.html Regards, JL. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
Jens, Thanks for your input. Unfortunately, I am out of the office this week. When I return I will be trying to resolve this issue. I don't believe it has squat to do with my Database at all, so I will be setting up some scenarios where I am making simultaneous requests, requiring no DB access. I will echo the data to my screen, hopefully I will see an interwoven jumble of characters as my web server is sending text simultaneously to multiple clients. If I don't see an interwoven jumble of stuff representing the various threads, then my problem is more fundamental than DB contention. I appreciate everyone's help. Jon Emmons On Fri, May 23, 2008 at 1:55 PM, Andreas Jung [EMAIL PROTECTED] wrote: --On 23. Mai 2008 12:31:50 -0500 Jens Vagelpohl [EMAIL PROTECTED] wrote: On May 23, 2008, at 11:49 , Jon Emmons wrote: Another thought I had, is that Zope has only one python interpreter running, pretty much forcing all processing to be serial by default given what you've just said. Sorry, that's all baloney. Just like blindly hiking the number of threads or the ZODB database connection pool size. Zope can handle more than one concurrent request. I would take a close look at the Zope database adapter for your relational database you are using. Some are not thread-enabled and will serialize database access. Not much to be added. Either your backend server blocks further requests (as indicated earlier by a database lock) or some extension module is holding the GIL. We have seen situation where Python did not perform a thread switch while working on a complex regular expression operation in one thread (preventing a thread switch over ten seconds and longer). Andreas ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
Jon Emmons wrote at 2008-5-23 08:58 -0400: ... I am running zope 2.9.4 and have observed that it will not simultaneously serve pages to my users. Usually, it does. I have seen database adapter packages (an old psycopg version, to be precise) that forgot to release the GIL for some operations. Then Zope freezes while these operations are executed. A surprising report of a similar kind was: while Zope's embedded profiler is in use, Zope effectively runs in single thread mode (such that other threads do not distort the timing results). Probably not your problem ... but who knows. Further potential reasons (for almost surely not responsible for your current problem: Expensive operations implemented in C (such as operations on huge strings) Creating excessive amounts of objects (causing lots of generation 2 garbage collections which hold the GIL) -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
--On 23. Mai 2008 08:58:25 -0400 Jon Emmons [EMAIL PROTECTED] wrote: Is this typical Zope behavior? No, and it is unlikely that Zope is in charge for this behavior. You might encounter a situation where your database locks a table or something like that and therefore blocking other requests. -aj pgpmkB6ozMtpq.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Thanks Jaroslav, I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:10 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions You need to increase number of server threads and/or number of database connections (it mean ZoDB connections). - Original Message - From: Jon Emmons User 1 requests data that takes the MySQL server 30 seconds to retrieve Meanwhile. User 2 wants to log on User 2 will not be served the logon page until user 1's query has finished. Is this typical Zope behavior? How do I get Zope to serve pages, run python scripts, etc. to multiple users simultaneously? Thanks, Jon Emmons ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
You need to increase number of server threads and/or number of database connections (it mean ZoDB connections). - Original Message - From: Jon Emmons User 1 requests data that takes the MySQL server 30 seconds to retrieve Meanwhile. User 2 wants to log on User 2 will not be served the logon page until user 1's query has finished. Is this typical Zope behavior? How do I get Zope to serve pages, run python scripts, etc. to multiple users simultaneously? Thanks, Jon Emmons ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Thanks Andreas, And for a very long time that is what I thought. We have 5 database servers. They are each multisession servers. If a query on any one of them is taking any time, every zope user is locked up until that query finishes, despite the server they are accessing. I don't think it has a single thing to do with the database access, but I have somehow gotten my zope installation in such a mode that each user request is served serially and there is zero parallel processing. It's not like I've seen a lot of zope installations, and I don't know how they behave. Typically the pages are served so rapidly that this is not a noticeable behavior, but our user base and databases have grown to the point that it has become critical. Thanks for your help. Jon Emmons -Original Message- From: Andreas Jung [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:19 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions --On 23. Mai 2008 08:58:25 -0400 Jon Emmons [EMAIL PROTECTED] wrote: Is this typical Zope behavior? No, and it is unlikely that Zope is in charge for this behavior. You might encounter a situation where your database locks a table or something like that and therefore blocking other requests. -aj ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Jaroslav, OK, that looks real! Thanks, I am going to try it on my development server and run some tests. Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:37 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Jaroslav and everyone, I have made the changes Jaroslav suggested and I cannot get Zope to serve even 2 sessions simultaneously. In our testing, one client will start a query to the data servers. Another will try to simply get the logon page to be served. Zope will simply sit there and wait until the DB returns its results for the first user before doing anything at all for the second. The 2nd user is not hitting the DB server. The python script accessing the data is waiting for the results for the first user. Could python be the problem? I am running python 2.4.4. Is it possible that I missed a flag or something when I build the Zope? Thanks for your suggestions so far, but I think I have missed a fundamental setup step that allows zope to multithread sessions. If I were echoing what was being served to the clients, shouldn't I see it interwoven? I don't, it is always (client 1's results) then (client 2's results)... very atomic. Thanks again, Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:37 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
What it sounds like is that Python is using a sequential threading model. I don't know about Python in particular, but some languages implement a sequential model internally so that multithreaded programs will run correctly (if not effeciently) in the absense of whatever thread libraries the language uses for normal threading. As an aside, something I learned the hard way is that Python doesn't thread very well. The documentation talks a lot about threading, but in most instances only one thread can run at a given time (it should still swap between threads, though). I never saw any benefit to increasing z-server-threads beyond 4 -- all the extra threads are going to do is consume memory. If you need get real concurrency (for example to make use of additional processors) you need to run multiple zope processes. Kevin -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon Emmons Sent: Friday, May 23, 2008 8:10 AM To: 'Jaroslav Lukesh'; zope@zope.org Subject: RE: [Zope] Multithreading sessions Jaroslav and everyone, I have made the changes Jaroslav suggested and I cannot get Zope to serve even 2 sessions simultaneously. In our testing, one client will start a query to the data servers. Another will try to simply get the logon page to be served. Zope will simply sit there and wait until the DB returns its results for the first user before doing anything at all for the second. The 2nd user is not hitting the DB server. The python script accessing the data is waiting for the results for the first user. Could python be the problem? I am running python 2.4.4. Is it possible that I missed a flag or something when I build the Zope? Thanks for your suggestions so far, but I think I have missed a fundamental setup step that allows zope to multithread sessions. If I were echoing what was being served to the clients, shouldn't I see it interwoven? I don't, it is always (client 1's results) then (client 2's results)... very atomic. Thanks again, Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:37 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
On Friday 23 May 2008 17:09:49 Jon Emmons wrote: Jaroslav and everyone, I have made the changes Jaroslav suggested and I cannot get Zope to serve even 2 sessions simultaneously. In our testing, one client will start a query to the data servers. Another will try to simply get the logon page to be served. Zope will simply sit there and wait until the DB returns its results for the first user before doing anything at all for the second. The 2nd user is not hitting the DB server. The python script accessing the data is waiting for the results for the first user. Could python be the problem? I very much doubt it. Zope (and Python) can easily serve many concurrent sessions. You'll need to find out what exactly makes Zope hang and where it hangs. One option to do this is to attach with gdb to the Python process and figure it out from the backtrace. http://www.upfrontsystems.co.za/Members/jean/zope-notes/debug-spinning-zope talks about that a bit Another option is to use the Deadlock Debugger product (I haven't used that personally though): http://www.zope.org/Members/nuxeo/Products/DeadlockDebugger hth, peter. I am running python 2.4.4. Is it possible that I missed a flag or something when I build the Zope? Thanks for your suggestions so far, but I think I have missed a fundamental setup step that allows zope to multithread sessions. If I were echoing what was being served to the clients, shouldn't I see it interwoven? I don't, it is always (client 1's results) then (client 2's results)... very atomic. Thanks again, Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:37 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) signature.asc Description: This is a digitally signed message part. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This is relevant: http://docs.python.org/api/threads.html http://www.pyzine.com/Issue001/Section_Articles/article_ThreadingGlobalInterpreter.html But notice some C extensions do allow concurrency by releasing the GIL. Most I/O operations in the standard library do that. And probably the network library used by python to connect to MySQL does that too. However, if I got everything Jon explained, the problems happens even with several DB servers. :? Best regards, Manuel. Sours, Kevin wrote: What it sounds like is that Python is using a sequential threading model. I don't know about Python in particular, but some languages implement a sequential model internally so that multithreaded programs will run correctly (if not effeciently) in the absense of whatever thread libraries the language uses for normal threading. As an aside, something I learned the hard way is that Python doesn't thread very well. The documentation talks a lot about threading, but in most instances only one thread can run at a given time (it should still swap between threads, though). I never saw any benefit to increasing z-server-threads beyond 4 -- all the extra threads are going to do is consume memory. If you need get real concurrency (for example to make use of additional processors) you need to run multiple zope processes. Kevin -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon Emmons Sent: Friday, May 23, 2008 8:10 AM To: 'Jaroslav Lukesh'; zope@zope.org Subject: RE: [Zope] Multithreading sessions Jaroslav and everyone, I have made the changes Jaroslav suggested and I cannot get Zope to serve even 2 sessions simultaneously. In our testing, one client will start a query to the data servers. Another will try to simply get the logon page to be served. Zope will simply sit there and wait until the DB returns its results for the first user before doing anything at all for the second. The 2nd user is not hitting the DB server. The python script accessing the data is waiting for the results for the first user. Could python be the problem? I am running python 2.4.4. Is it possible that I missed a flag or something when I build the Zope? Thanks for your suggestions so far, but I think I have missed a fundamental setup step that allows zope to multithread sessions. If I were echoing what was being served to the clients, shouldn't I see it interwoven? I don't, it is always (client 1's results) then (client 2's results)... very atomic. Thanks again, Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:37 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFINvPgI2zpkmcEAhgRAgdwAKCAFmjuU93kwK/N9YG1rNFIsWF0vwCffP/D jii2wbhTczH9PhkD1VvdGYA= =vovu -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
Thanks Kevin, Another thought I had, is that Zope has only one python interpreter running, pretty much forcing all processing to be serial by default given what you've just said. In the zope.conf there is a setting: Python-check-interval Which defaults to 500 (I assume milliseconds, no documentation to confirm) It is supposed to check the various threads at that interval. If the multi-threading is sequential then I may be out of luck. However, given what you said, I should see even 1 cpu give each session thread time slices. Just a thought; I am still trying to find a definitive answer to what is happening, or a clear path to true multithreaded session handling. Jon Emmons -Original Message- From: Sours, Kevin [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 11:48 AM To: Jon Emmons; zope@zope.org Subject: RE: [Zope] Multithreading sessions What it sounds like is that Python is using a sequential threading model. I don't know about Python in particular, but some languages implement a sequential model internally so that multithreaded programs will run correctly (if not effeciently) in the absense of whatever thread libraries the language uses for normal threading. As an aside, something I learned the hard way is that Python doesn't thread very well. The documentation talks a lot about threading, but in most instances only one thread can run at a given time (it should still swap between threads, though). I never saw any benefit to increasing z-server-threads beyond 4 -- all the extra threads are going to do is consume memory. If you need get real concurrency (for example to make use of additional processors) you need to run multiple zope processes. Kevin -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jon Emmons Sent: Friday, May 23, 2008 8:10 AM To: 'Jaroslav Lukesh'; zope@zope.org Subject: RE: [Zope] Multithreading sessions Jaroslav and everyone, I have made the changes Jaroslav suggested and I cannot get Zope to serve even 2 sessions simultaneously. In our testing, one client will start a query to the data servers. Another will try to simply get the logon page to be served. Zope will simply sit there and wait until the DB returns its results for the first user before doing anything at all for the second. The 2nd user is not hitting the DB server. The python script accessing the data is waiting for the results for the first user. Could python be the problem? I am running python 2.4.4. Is it possible that I missed a flag or something when I build the Zope? Thanks for your suggestions so far, but I think I have missed a fundamental setup step that allows zope to multithread sessions. If I were echoing what was being served to the clients, shouldn't I see it interwoven? I don't, it is always (client 1's results) then (client 2's results)... very atomic. Thanks again, Jon Emmons -Original Message- From: Jaroslav Lukesh [mailto:[EMAIL PROTECTED] Sent: Friday, May 23, 2008 9:37 AM To: Jon Emmons; zope@zope.org Subject: Re: [Zope] Multithreading sessions Hi Jon, I use 24 zserver-threads and I have here defined number of ZoDB connections (pool-size): in your instance etc/zope.conf look for that section: zodb_db main # Main FileStorage database filestorage path $INSTANCE/var/Data.fs /filestorage mount-point / cache-size 1000 pool-size 24 /zodb_db Please pay attention, if you have bunch of cache, your RAM shoul be exhausted. Regards, JL. - Original Message - From: Jon Emmons [EMAIL PROTECTED] I had already gone into zope.conf and added the line: zserver-threads 10 This seemed to have no effect. Is that what you mean? You think I should bump it higher? 100 maybe? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
On May 23, 2008, at 11:49 , Jon Emmons wrote: Another thought I had, is that Zope has only one python interpreter running, pretty much forcing all processing to be serial by default given what you've just said. Sorry, that's all baloney. Just like blindly hiking the number of threads or the ZODB database connection pool size. Zope can handle more than one concurrent request. I would take a close look at the Zope database adapter for your relational database you are using. Some are not thread-enabled and will serialize database access. jens ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] Multithreading sessions
This is relevant: http://docs.python.org/api/threads.html http://www.pyzine.com/Issue001/Section_Articles/article_ThreadingGlobalI nterpreter.html But notice some C extensions do allow concurrency by releasing the GIL. Most I/O operations in the standard library do that. And probably the network library used by python to connect to MySQL does that too. My experience with Zope is that this is a fairly minor effect. In my case a multi processor server running single zope instance would max out a single processor an (very) lightly load a second (pretty much no matter how many threads I ran). Kevin ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Multithreading sessions
--On 23. Mai 2008 12:31:50 -0500 Jens Vagelpohl [EMAIL PROTECTED] wrote: On May 23, 2008, at 11:49 , Jon Emmons wrote: Another thought I had, is that Zope has only one python interpreter running, pretty much forcing all processing to be serial by default given what you've just said. Sorry, that's all baloney. Just like blindly hiking the number of threads or the ZODB database connection pool size. Zope can handle more than one concurrent request. I would take a close look at the Zope database adapter for your relational database you are using. Some are not thread-enabled and will serialize database access. Not much to be added. Either your backend server blocks further requests (as indicated earlier by a database lock) or some extension module is holding the GIL. We have seen situation where Python did not perform a thread switch while working on a complex regular expression operation in one thread (preventing a thread switch over ten seconds and longer). Andreas pgpesVG7z7wN2.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
