Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-08-08 Thread Chris Withers
Dieter Maurer wrote: Chris Withers wrote at 2004-7-23 10:58 +0100: Dieter Maurer wrote: All DA's I saw up to now, do a reconnect. ZOracleDA didn't... Really? I saw it reconnecting... It had some pretty whacky heuristic crap for when to reconnect, you must have been very lucky :-) Raising an

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-26 Thread Brad Clements
On 25 Jul 2004 at 23:22, Dieter Maurer wrote: If a connection is dropped due to inactivity it should not affect any transactions going to occur in the future as a reconnect is issued before submitted new transactions. However, connections can be lost for other reasons than inactivity... I

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-25 Thread Dieter Maurer
Brad Clements wrote at 2004-7-23 16:12 -0400: On 23 Jul 2004 at 20:08, Dieter Maurer wrote: The bad sequence can look as follows: * Zope starts a request (and thereby a transaction) * The request sends a modifying request to a relational database * The connection is lost; the former

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-25 Thread Dieter Maurer
sathya wrote at 2004-7-23 18:00 -0500: I dont see a data loss problem either (unless there are hardware failures) If a connection is dropped due to inactivity it should not affect any transactions going to occur in the future as a reconnect is issued before submitted new transactions. However,

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-23 Thread Chris Withers
Dieter Maurer wrote: All DA's I saw up to now, do a reconnect. ZOracleDA didn't... But this is *WRONG -- as part of a transaction may have been lost. After reconnecting, they should raise an exception derived from ConflictError and let the complete request retry. ...but it now does this :-) Chris

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-23 Thread Brad Clements
On 23 Jul 2004 at 10:58, Chris Withers wrote: Dieter Maurer wrote: All DA's I saw up to now, do a reconnect. ZOracleDA didn't... But this is *WRONG -- as part of a transaction may have been lost. After reconnecting, they should raise an exception derived from ConflictError and

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-23 Thread Dieter Maurer
Brad Clements wrote at 2004-7-23 09:53 -0400: On 23 Jul 2004 at 10:58, Chris Withers wrote: Dieter Maurer wrote: All DA's I saw up to now, do a reconnect. ZOracleDA didn't... But this is *WRONG -- as part of a transaction may have been lost. After reconnecting, they should raise an

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-23 Thread Dieter Maurer
Chris Withers wrote at 2004-7-23 10:58 +0100: Dieter Maurer wrote: All DA's I saw up to now, do a reconnect. ZOracleDA didn't... Really? I saw it reconnecting... But this is *WRONG -- as part of a transaction may have been lost. After reconnecting, they should raise an exception derived

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-23 Thread Brad Clements
On 23 Jul 2004 at 20:08, Dieter Maurer wrote: The bad sequence can look as follows: * Zope starts a request (and thereby a transaction) * The request sends a modifying request to a relational database * The connection is lost; the former modification is discarded as the database

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-23 Thread sathya
I dont see a data loss problem either (unless there are hardware failures) If a connection is dropped due to inactivity it should not affect any transactions going to occur in the future as a reconnect is issued before submitted new transactions. However connections getting dropped due to

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-22 Thread Dieter Maurer
Chris Withers wrote at 2004-7-21 16:44 +0100: Brad Clements wrote: I have set the SAPDB timeout to 32400 seconds. Unless you have an extremely heavily loaded Zope, that's too low, and may be causing you problems... That is the maximum allowed value in dbmgui. My point is that unless you

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-21 Thread Chris Withers
Brad Clements wrote: I have set the SAPDB timeout to 32400 seconds. Unless you have an extremely heavily loaded Zope, that's too low, and may be causing you problems... That is the maximum allowed value in dbmgui. My point is that unless you implement a connection pool model independent of Zope's

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-21 Thread Brad Clements
On 21 Jul 2004 at 16:44, Chris Withers wrote: My point is that unless you implement a connection pool model independent of Zope's threading, you WILL run into problems if you have ANY kind of timeout where the DA isn't smnart enough to reconnect connections that have closed... I understand.

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-19 Thread Chris Withers
Brad Clements wrote: What's the actual problem you're experiencing? I have set the SAPDB timeout to 32400 seconds. Unless you have an extremely heavily loaded Zope, that's too low, and may be causing you problems... Chris -- Simplistix - Content Management, Zope Python Consulting -

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-19 Thread Chris Withers
Brad Clements wrote: That does suck, but it doesn't seem like an insurmountable problem for a DA. You just need a per-DA instance mutex which you acquire before opening a connection release afterwards. Someone else said that every thread has it's own copy of ZODB, hence their own copies of each

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-19 Thread Brad Clements
On 19 Jul 2004 at 7:44, Chris Withers wrote: Brad Clements wrote: What's the actual problem you're experiencing? I have set the SAPDB timeout to 32400 seconds. Unless you have an extremely heavily loaded Zope, that's too low, and may be causing you problems... That is the maximum

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-16 Thread Chris Withers
Brad Clements wrote: On 9 Jul 2004 at 21:17, Dieter Maurer wrote: Check whether the connection is maintained in a _v_ attribute (in this case, it is per thread). Otherwise, the connection would need to be maintained in a module global registry indexed with a DA id (e.g. its ZODB path). This is

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-16 Thread Brad Clements
On 16 Jul 2004 at 8:57, Chris Withers wrote: This often means that many more database connections are opened than are actually necessary. And if you start timing out inactive database connectiosn on the database side, you're in for a world of pain and suffering. I branched ZOracleDA some

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-16 Thread Andy Dustman
On Fri, 16 Jul 2004 10:36:27 -0400, Brad Clements [EMAIL PROTECTED] wrote: The SAP folks say their adapter is multi-thread capable, but I've come to believe that only means that it's ok to open a connection in one thread and use it in another, but NOT to have 2 threads make requests on

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-16 Thread sathya
sqlrelay seems to hold some promise for connection pooling although I am not sure if theres support for sapdb Brad Clements wrote: On 16 Jul 2004 at 8:57, Chris Withers wrote: This often means that many more database connections are opened than are actually necessary. And if you start timing

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-16 Thread Brad Clements
On 16 Jul 2004 at 11:38, Andy Dustman wrote: Also, two threads can' t open a connection at the same time because their connection table management isn't thread safe. That does suck, but it doesn't seem like an insurmountable problem for a DA. You just need a per-DA instance mutex which

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-16 Thread Ganesha Shanmuganathan
Hi, On Fri, 2004-07-16 at 09:36, Brad Clements wrote: On 16 Jul 2004 at 8:57, Chris Withers wrote: This often means that many more database connections are opened than are actually necessary. And if you start timing out inactive database connectiosn on the database side, you're in for a

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-10 Thread Roché Compaan
* Brad Clements [EMAIL PROTECTED] [2004-07-09 22:36]: On 9 Jul 2004 at 21:17, Dieter Maurer wrote: Check whether the connection is maintained in a _v_ attribute (in this case, it is per thread). Otherwise, the connection would need to be maintained in a module global registry indexed

[Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-09 Thread Brad Clements
I am using ZsapdbDA with SAPDB (now MaxDB) and have run into problems when Zope has more than one thread. The folks at SAP suggest that there should be one database connection per thread, rather than sharing one connection across threads. I haven't yet looked at the Zsapdb source, but I think

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-09 Thread sathya
sathya wrote: we have been looking at psycopg for postgres. zope opens atleast one connection per zope thread. I have obeserved the same for mysql as well. Are you creating your own threads within zope ? in which case you have to do the locking yourself Brad Clements wrote: I am using

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-09 Thread Dieter Maurer
Brad Clements wrote at 2004-7-9 09:35 -0400: I am using ZsapdbDA with SAPDB (now MaxDB) and have run into problems when Zope has more than one thread. The folks at SAP suggest that there should be one database connection per thread, rather than sharing one connection across threads. This is