[ZODB-Dev] Re: RelStorage now in Subversion
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Shane Hathaway wrote: Hi all, RelStorage now exists in the Zope subversion repository here: http://svn.zope.org/relstorage/trunk/ I have also created a wiki page: http://wiki.zope.org/ZODB/RelStorage Testing help and database-specific advice would be appreciated! With both PostgreSQL and Oracle, all relevant ZODB tests pass and I can create a Plone 3 site, but I haven't tested other Zope applications, more than one machine at a time, or Windows. Furthermore, I wonder what suggestions any Oracle gurus might have. According to charts I've made, Oracle appears to be leading already in read performance (faster than PostgreSQL and FileStorage+ZEO), but the write performance is plain dismal. Is it because I'm using two-phase commit? I don't know. Anyway, have fun with the code! A release is planned for the end of February. I've got a question for you on the removal of the LISTEN / NOTIFY stuff in the Postgresql adapter: it looks to me as though your code was not doing the async check-for-notifies as shown in the 'notify.py' in the psycopg2 examples: http://www.initd.org/tracker/psycopg/browser/psycopg2/trunk/examples/notify.py So, any notifies you got were only capture on the basis of syncrhonous activity (a cursor doing some other work). Could that explain the unreliability which caused you to remove it? The symptom would be like the time that the ZEO async stuff got dropped in Zope3 during the Twisted integration[1]: a busy client would stay up to date, but an idle one would serve stale objects from cache. BTW, thanks again to Jarn for making this possible. Yay, Jarn. [1] http://thread.gmane.org/gmane.comp.web.zope.zope3/15286/focus=15353 Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHogi8+gerLs4ltQ4RAgJ5AJ9f/628xemOrEDYkpvGbaVymB3tTQCfUA8f sE9qtZwzT77dCStOUkB07JQ= =VhOU -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: RelStorage now in Subversion
Tres Seaver wrote: I've got a question for you on the removal of the LISTEN / NOTIFY stuff in the Postgresql adapter: it looks to me as though your code was not doing the async check-for-notifies as shown in the 'notify.py' in the psycopg2 examples: http://www.initd.org/tracker/psycopg/browser/psycopg2/trunk/examples/notify.py So, any notifies you got were only capture on the basis of syncrhonous activity (a cursor doing some other work). Could that explain the unreliability which caused you to remove it? The symptom would be like the time that the ZEO async stuff got dropped in Zope3 during the Twisted integration[1]: a busy client would stay up to date, but an idle one would serve stale objects from cache. It does appear that a short select() loop is required. However, I tried that and learned that notifications have no timeliness guarantees. The current model requires strict timeliness of change notification in order to maintain ZODB consistency. However, I think we can still take advantage of listen and notify in a slightly different way. Currently, the Postgres adapter rolls back the load transaction on every connection close, but if it instead leaves the session idle with a transaction open, it would be safe for RelStorage to sometimes ignore new changes and only roll back and poll after an async notification arrives. I could see scalability going way up that way, as long as Postgres doesn't mind all the idle sessions with an open transaction. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: RelStorage now in Subversion
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Shane Hathaway wrote: Tres Seaver wrote: I've got a question for you on the removal of the LISTEN / NOTIFY stuff in the Postgresql adapter: it looks to me as though your code was not doing the async check-for-notifies as shown in the 'notify.py' in the psycopg2 examples: http://www.initd.org/tracker/psycopg/browser/psycopg2/trunk/examples/notify.py So, any notifies you got were only capture on the basis of syncrhonous activity (a cursor doing some other work). Could that explain the unreliability which caused you to remove it? The symptom would be like the time that the ZEO async stuff got dropped in Zope3 during the Twisted integration[1]: a busy client would stay up to date, but an idle one would serve stale objects from cache. It does appear that a short select() loop is required. However, I tried that and learned that notifications have no timeliness guarantees. The current model requires strict timeliness of change notification in order to maintain ZODB consistency. However, I think we can still take advantage of listen and notify in a slightly different way. Currently, the Postgres adapter rolls back the load transaction on every connection close, but if it instead leaves the session idle with a transaction open, it would be safe for RelStorage to sometimes ignore new changes and only roll back and poll after an async notification arrives. I could see scalability going way up that way, as long as Postgres doesn't mind all the idle sessions with an open transaction. Hmm, I think I read that NOTIFY events are only delivered between transactions, which would make that strategy unworkable. It might satisfy the timeliness / ordering problem you mention above, though. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHojzk+gerLs4ltQ4RAmHXAKC3DWecX3L+EbH92yNsZpeqoxDx2QCfcIEy Hwg0wQPK3UoaMV9YxUmd3UQ= =/vNj -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
