Re: [ZODB-Dev] Supporting a DataManager without Two Phase Commit

[Christian Theune]

Am Samstag, den 05.05.2007, 02:46 +0200 schrieb Andreas Jung:
> 
> --On 4. Mai 2007 22:33:05 +0100 Laurence Rowe <[EMAIL PROTECTED]> wrote:
> > Following a discussion with several of the sqlalchemy integration authors
> > on #plone today we came up with the following hack to implement this:
> > http://dev.plone.org/collective/browser/collective.lead/trunk/collective/
> > lead/tx.py
> >
> 
> I think z3c.zalchemy is doing it right:

Nope, it's not.

The commit, which is executed in tpc_finish() might very well fail, e.g.
for serialization issues. That's not an uncommon thing and won't be
handled correctly.

I think the proposal from Laurence does a better job.

Christian

-- 
gocept gmbh & co. kg - forsterstraße 29 - 06112 halle/saale - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development


signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Supporting a DataManager without Two Phase Commit

[Andreas Jung]

--On 5. Mai 2007 02:46:58 +0200 Andreas Jung <[EMAIL PROTECTED]> wrote:



If commit fails or another DataManager fails, data is not commited to the
database.



However I am not sure if this is completely true. Although this approach 
ensures that all SQL statements were execute properly it does not take into 
account that a commit operation itself might fail even at that point for 
some reason. Even the Python DB API itself does not support the TPC and 
most don't driver don't support TPC through an API. So that's basically a 
flaw in the Python DB-API level. Even up2date drivers like psycopg2 for 
Postgres don't support something like PREPARE TRANSACTION or COMMIT 
PREPARED.


See also



Looking back at your hack..it seems to introduce some more safety for a
particular usecase (perhaps general for people dealing only with one 
database at a time).


But thanks for pointing out this problem. I'll change my code in 
z3c.sqlalchemy in a similar way to gain at least a bit more safety.


Andreas

pgpDNDiDzggFQ.pgp
Description: PGP signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Supporting a DataManager without Two Phase Commit

[Andreas Jung]

--On 4. Mai 2007 22:33:05 +0100 Laurence Rowe <[EMAIL PROTECTED]> wrote:

Following a discussion with several of the sqlalchemy integration authors
on #plone today we came up with the following hack to implement this:
http://dev.plone.org/collective/browser/collective.lead/trunk/collective/
lead/tx.py



I think z3c.zalchemy is doing it right:

"""
Changed how the two phase commit works.

This is how the two phase commit is used in zope.

   1. tpc_begin(txn)
   2. commit(txn)
   3. tpc_vote(txn)
   4. tpc_finish(txn)

What zalchemy is doing:

 - commit does a session.flush() which actually executes all sql 
statements.

 - tpc_finish() does a transaction.commit() in the sqlalchemy transaction
 - tpc_abort() does a transaction.rollback() in the sqlalchemy transaction

If commit fails or another DataManager fails, data is not commited to the
database.

Thanks to Michael Bayer (the author of sqlalchemy) who gave me the hint to
integrate sqlalchemy correctly into the two phase commit.

"""

Andreas

pgpTh1fCqZm2X.pgp
Description: PGP signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Supporting a DataManager without Two Phase Commit

[Andreas Jung]

--On 4. Mai 2007 22:33:05 +0100 Laurence Rowe <[EMAIL PROTECTED]> wrote:


Following a discussion with several of the sqlalchemy integration authors
on #plone today we came up with the following hack to implement this:
http://dev.plone.org/collective/browser/collective.lead/trunk/collective/
lead/tx.py

The DataManager is given a high sortKey to ensure that it is considered
last, and commits in tpc_vote, before the other (2PC) DataManagers'
tpc_finish methods are called.

The hack obviously relies on only one DataManager making use of the
trick. It would be nice to make this was supported directly so that an a
error could be thrown when more than one 1PC DataManager joined a
transaction.


Well, the hack is really only a hack. As soon as more than one data manager 
is involved you'll run into the same problems. I have some apps with 
multiple SQL connections so this hack would give me nothing.




This could be implemented by changing the signature of
transaction._transaction.Transaction.join to have an optional
single_phase argument (default would be False). The 1PC resource would
then be registered seperately to the 2PC resources and _commitResources
would call commit on the 1PC resource between tpc_vote and tpc_finish.

If you think this would be helpful I'll try and supply a patch (need to
look into the detail of how failed transactions are cleaned up).


You could resolve this issue only if all subsystems participating in 
transaction could rollback a commit happened with this transaction in case 
of a data manager failing during its commit.


Andreas


pgp2PTDD5Or7O.pgp
Description: PGP signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev