[Zope] Re: Zope hangs when database takes too long to respond

2006-09-07 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Sinang, Danny wrote:
 
 It is a known limitation of Zope, which has a set number of worker
 threads available to service user requests.  If a worker thread blocks
 (e.g., waiting for MySQL), it is not able to do any work for other
 pending requests.
 
 You can tweak the number of threads in your zope.conf file, via the
 'zserver-threads' directive (the default value is 4).  If you do
 increase this value, you also need to increase the number of database
 connections avialble (via 'pool-size') which will increase the RAM
 required by your appserver.
 
 
 Thanks Tres.
 
 I found 'zserver-threads' in zope.conf, but couldn't find 'pool-size'.
 Am using Zope 2.7.6.
 
 Is this something available on higher versions of Zope ?

'pool-size' is an attribute of the database configuration stanza, e.g.:

 zodb_db main
  filestorage
path $INSTANCE_HOME/var/Data.fs
  /filestorage
  mount-point/
  cache-size 5000
  pool-size  7
  version-pool-size  3
  version-cache-size 100
 /zodb_db



Tres.
- --
===
Tres Seaver  +1 202-558-7113  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFFAFhB+gerLs4ltQ4RAvAwAKDbZ3An6fQlX5v4BHrLhOBfqwru9QCeLT7l
IpJ42XCh6NBh6auRjMNh7F0=
=rmp6
-END PGP SIGNATURE-

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


[Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Sinang, Danny wrote:
 Sorry for the typo error. 
  
 The subject should read hangs, not hands .
 
 
 
 From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of
 Sinang, Danny
 Sent: Wednesday, September 06, 2006 10:56 PM
 To: zope@zope.org
 Subject: [Zope] Zope hands when database takes too long to respond
 
 
 Hello,
  
 We wrote an application (on Zope v2.7.6-final) that queries a MySQL
 database via ZMySQLDA v2.0.8.
  
 Whenever the MySQL server experiences heavy load, Zope seems to hang,
 such that we can't even access the ZMI.
  
 Is this a bug in version 2.7.6 or ZMySQLDA ?

It is a known limitation of Zope, which has a set number of worker
threads available to service user requests.  If a worker thread blocks
(e.g., waiting for MySQL), it is not able to do any work for other
pending requests.

You can tweak the number of threads in your zope.conf file, via the
'zserver-threads' directive (the default value is 4).  If you do
increase this value, you also need to increase the number of database
connections avialble (via 'pool-size') which will increase the RAM
required by your appserver.

 Or are there some settings I can tweak to adjust some database
 connection timeout value ?

Not that I know of.


Tres.
- --
===
Tres Seaver  +1 202-558-7113  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFE/zql+gerLs4ltQ4RAioIAJ98L+XTswgN8cRE24qAf7/kGckA+ACg1ZKp
nms3D4/jbMw/KjPVTp+AC5k=
=hmb+
-END PGP SIGNATURE-

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Jonathan


- Original Message - 
From: Tres Seaver [EMAIL PROTECTED]

To: zope@zope.org
Sent: Wednesday, September 06, 2006 5:16 PM
Subject: [Zope] Re: Zope hangs when database takes too long to respond



-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Sinang, Danny wrote:
Sorry for the typo error. 
 
The subject should read hangs, not hands .




From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of
Sinang, Danny
Sent: Wednesday, September 06, 2006 10:56 PM
To: zope@zope.org
Subject: [Zope] Zope hands when database takes too long to respond


Hello,
 
We wrote an application (on Zope v2.7.6-final) that queries a MySQL

database via ZMySQLDA v2.0.8.
 
Whenever the MySQL server experiences heavy load, Zope seems to hang,

such that we can't even access the ZMI.
 
Is this a bug in version 2.7.6 or ZMySQLDA ?


It is a known limitation of Zope, which has a set number of worker
threads available to service user requests.  If a worker thread blocks
(e.g., waiting for MySQL), it is not able to do any work for other
pending requests.

You can tweak the number of threads in your zope.conf file, via the
'zserver-threads' directive (the default value is 4).  If you do
increase this value, you also need to increase the number of database
connections avialble (via 'pool-size') which will increase the RAM
required by your appserver.


Or are there some settings I can tweak to adjust some database
connection timeout value ?


Not that I know of.




Another possible solution is to have a look at twisted:

http://twistedmatrix.com/projects/core/documentation/howto/faq.html


Jonathan



___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Chris McDonough

On Sep 6, 2006, at 5:30 PM, Jonathan wrote:



Another possible solution is to have a look at twisted:

http://twistedmatrix.com/projects/core/documentation/howto/faq.html


Unless he wants to rewrite his application, Twisted likely won't  
help, right?


The OP may also want to see why his MySQL queries are taking so  
long by using the MySQL query log and its optimizer.


One potential solution to the immediate cant access the ZMI when I  
have all threads tied up in long-running code problem is Zope zodb  
connection policies, which is an idea that I proposed several years  
ago but never got around to implementing fully.  Essentially, you'd  
be able reserve a ZODB connection for management requests.


- C

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Jonathan


- Original Message - 
From: Chris McDonough [EMAIL PROTECTED]

To: Jonathan [EMAIL PROTECTED]
Cc: zope@zope.org
Sent: Wednesday, September 06, 2006 5:40 PM
Subject: Re: [Zope] Re: Zope hangs when database takes too long to respond



On Sep 6, 2006, at 5:30 PM, Jonathan wrote:



Another possible solution is to have a look at twisted:

http://twistedmatrix.com/projects/core/documentation/howto/faq.html


Unless he wants to rewrite his application, Twisted likely won't  help, 
right?


Twisted is, as i said Another possible solution - yes it will require work 
to integrate, but it is still a potential solution.  I don't like to tell 
people what to do, i just try to provide alternatives.



Jonathan



___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Chris McDonough


On Sep 6, 2006, at 5:47 PM, Jonathan wrote:
Unless he wants to rewrite his application, Twisted likely won't   
help, right?


Twisted is, as i said Another possible solution - yes it will  
require work to integrate, but it is still a potential solution.  I  
don't like to tell people what to do, i just try to provide  
alternatives.


Well, by those standards, I propose he look at Delphi too... ;-)

- C

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Jonathan


- Original Message - 
From: Chris McDonough [EMAIL PROTECTED]

To: Jonathan [EMAIL PROTECTED]
Cc: zope@zope.org
Sent: Wednesday, September 06, 2006 5:50 PM
Subject: Re: [Zope] Re: Zope hangs when database takes too long to respond




On Sep 6, 2006, at 5:47 PM, Jonathan wrote:
Unless he wants to rewrite his application, Twisted likely won't   
help, right?


Twisted is, as i said Another possible solution - yes it will  
require work to integrate, but it is still a potential solution.  I  
don't like to tell people what to do, i just try to provide  
alternatives.


Well, by those standards, I propose he look at Delphi too... ;-)


Do you mean he should consult the Oracle?   ;-)


Jonathan

P.S.  I really was trying to be helpful

___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Chris McDonough


On Sep 6, 2006, at 5:54 PM, Jonathan wrote:

Well, by those standards, I propose he look at Delphi too... ;-)


Do you mean he should consult the Oracle?   ;-)


Sure, why not, while he's at it? ;-)  We can probably even haul in  
Interbase.




P.S.  I really was trying to be helpful


Sure, I just wanted to provide a counterpoint so he didn't go off and  
waste time researching Twisted with the belief that there was some  
way he could use twisted without essentially abandoning his current  
codebase; not that he shouldn't use Twisted or whatever, it's just  
that the problem can likely be solved within Zope without abandoning  
what he has already.


- C


___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


Re: [Zope] Re: Zope hangs when database takes too long to respond

2006-09-06 Thread Jonathan


- Original Message - 
From: Chris McDonough [EMAIL PROTECTED]

To: Jonathan [EMAIL PROTECTED]
Cc: zope@zope.org
Sent: Wednesday, September 06, 2006 6:01 PM
Subject: Re: [Zope] Re: Zope hangs when database takes too long to respond




On Sep 6, 2006, at 5:54 PM, Jonathan wrote:

Well, by those standards, I propose he look at Delphi too... ;-)


Do you mean he should consult the Oracle?   ;-)


Sure, why not, while he's at it? ;-)  We can probably even haul in 
Interbase.


Actually, I was going for the Oracle at Delphi reference (oracle knows 
all, sees all, tells all - for a price), but the reference was possibly a 
bit obtuse...






___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope-dev )


[Zope] RE: Zope hangs when database takes too long to respond

2006-09-06 Thread Sinang, Danny


 It is a known limitation of Zope, which has a set number of worker
threads available to service user requests.  If a worker thread blocks
(e.g., waiting for MySQL), it is not able to do any work for other
pending requests.

 You can tweak the number of threads in your zope.conf file, via the
'zserver-threads' directive (the default value is 4).  If you do
increase this value, you also need to increase the number of database
connections avialble (via 'pool-size') which will increase the RAM
required by your appserver.


Thanks Tres.

I found 'zserver-threads' in zope.conf, but couldn't find 'pool-size'.
Am using Zope 2.7.6.

Is this something available on higher versions of Zope ?

Regards,
Danny


___
Zope maillist  -  Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
**   No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope-dev )