Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions,Retries,OhMy....
On Sat, 31 May 2003, Dieter Maurer wrote: > Shane Hathaway wrote at 2003-5-30 14:52 -0400: > > ... > > The ideas weren't as clear in my head then as they are now. :-) See my > > reply to Chris. > > What do you do with "ReadConflictError"s? They are worse than > (write) "ConflictError"s as they are not affected by conflict > resolution? The Shield prototype dealt with read conflict errors just fine. It just performed a rollback/replay operation early, if I recall correctly, although that's potentially a lot of extra work. MVCC will help here. > When you replay (without rereading and applying the application > logic), you can loose changes. Actually, the idea is to replay the application logic, not merge pickles. This is a lot easier than it sounds: the application just invokes any operation that changes the database through an interface that logs that operation. The log is either consulted for replay or discarded at the end of the transaction. The prototype proves it can work. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions,Retries,OhMy....
Shane Hathaway wrote at 2003-5-30 14:52 -0400: > ... > The ideas weren't as clear in my head then as they are now. :-) See my > reply to Chris. What do you do with "ReadConflictError"s? They are worse than (write) "ConflictError"s as they are not affected by conflict resolution? When you replay (without rereading and applying the application logic), you can loose changes. This might not be a big problem with the session objects themselves, but the session administrative data surely will not good with this approach. And many of the current ConflictErrors come from access to the administrative data. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions,Retries,OhMy....
Leonardo Rochael Almeida wrote: What are these rules? what happens if an application doesn't follow them? Do we get Conflicts just like before or are we suddenly bound to make the application follow the rules? The important rule, which only has to be followed by the portion of the application running inside the special connection, is that changes to objects should be made through the replayable interface provided by the connection. If the connection detects changes made outside the replayable interface, it will probably complain in a log message and let conflict errors propagate to the application until the next transaction. I read the Shield/README.txt and the ArmoredCatalog proposal, but I'm still a bit lost. The ideas weren't as clear in my head then as they are now. :-) See my reply to Chris. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions,Retries,OhMy....
On Fri, 2003-05-30 at 11:06, Shane Hathaway wrote: > I hear only crickets. I said we can avoid nearly all conflicts in > sessions. Does this not interest anyone? It does interest me, even though I'm not currently suffering from Conflicts. But the discussion seemed a little over my head so I didn't comment. I have some questions though > > Shane Hathaway wrote: > > A while ago I experimented with ways to prevent conflict errors from > > reaching the application. ZODB has matured since then and it should now > > be possible to make a Connection class that resolves conflicts > > internally rather than propagate them to the application--as long as the > > application follows certain rules. Should we consider finishing that > > effort? What are these rules? what happens if an application doesn't follow them? Do we get Conflicts just like before or are we suddenly bound to make the application follow the rules? > > > > Relevant URLs: > > > > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/zodbex/zodbex/Shield > > http://dev.zope.org/Wikis/DevSite/Proposals/ArmoredCatalog I read the Shield/README.txt and the ArmoredCatalog proposal, but I'm still a bit lost. Cheers, Leo -- Ideas don't stay in some minds very long because they don't like solitary confinement. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors,Transactions, Retries,OhMy....
On Fri, 2003-05-30 at 11:01, Shane Hathaway wrote: > The basic idea is that you track changes to session data in a replayable > way. If a conflict error happens, you roll back the session data, bring > it in sync with the current state of the database, and replay the > changes, all during transaction commit. If a conflict happens again, > you repeat the process, up to some user-defined limit. Right, that sounds really good. > To do this, you have to manage the session data as a single > transactional entity. This was hard to do until I convinced Jeremy to > slip in a new register() method on Connection objects. By overriding > register(), you can get the Connection to manage all changes to its > objects, and the Connection becomes that single transactional entity. So what make up the responsibilities of the session data object itself? > > (We either need an internal project to fail miserably due to conflict > > errors, or I need to stop playing MechAssault on the Xbox so much after > > work and get back to coding after work. ;-) > > Have you tried the commercial version of Tux Racer? I quite enjoy that > simple game. I got a 3D card just so I could play it. ;-) I haven't, because I don't have a PC that is powerful enough to play any games. But if anyone else has an Xbox out there, you can find me via my, uh, gamertag "mcdonc". ;-) - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions, Retries,OhMy....
Chris McDonough wrote: It interests me! ;-) I haven't had time to look at the resources you posted, though... Oops, I didn't intend for you to analyze those links deeply. I just wanted to see if there was interest in the idea. The basic idea is that you track changes to session data in a replayable way. If a conflict error happens, you roll back the session data, bring it in sync with the current state of the database, and replay the changes, all during transaction commit. If a conflict happens again, you repeat the process, up to some user-defined limit. To do this, you have to manage the session data as a single transactional entity. This was hard to do until I convinced Jeremy to slip in a new register() method on Connection objects. By overriding register(), you can get the Connection to manage all changes to its objects, and the Connection becomes that single transactional entity. The "Shield" experiment proved this can work and doesn't require a lot of code, but it also uncovered some complexity you have to be aware of, like subtransactions. (We either need an internal project to fail miserably due to conflict errors, or I need to stop playing MechAssault on the Xbox so much after work and get back to coding after work. ;-) Have you tried the commercial version of Tux Racer? I quite enjoy that simple game. I got a 3D card just so I could play it. ;-) Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions,Retries,OhMy....
It interests me! ;-) I haven't had time to look at the resources you posted, though... Dieter also sent me a different transient object container implementation before which has better conflict avoidance. I have not been able to look at that either. :-( Guh. (We either need an internal project to fail miserably due to conflict errors, or I need to stop playing MechAssault on the Xbox so much after work and get back to coding after work. ;-) - C On Fri, 2003-05-30 at 10:06, Shane Hathaway wrote: > I hear only crickets. I said we can avoid nearly all conflicts in > sessions. Does this not interest anyone? > > Shane Hathaway wrote: > > A while ago I experimented with ways to prevent conflict errors from > > reaching the application. ZODB has matured since then and it should now > > be possible to make a Connection class that resolves conflicts > > internally rather than propagate them to the application--as long as the > > application follows certain rules. Should we consider finishing that > > effort? > > > > Relevant URLs: > > > > http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/zodbex/zodbex/Shield > > http://dev.zope.org/Wikis/DevSite/Proposals/ArmoredCatalog > ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions, Retries,OhMy....
I hear only crickets. I said we can avoid nearly all conflicts in sessions. Does this not interest anyone? Shane Hathaway wrote: A while ago I experimented with ways to prevent conflict errors from reaching the application. ZODB has matured since then and it should now be possible to make a Connection class that resolves conflicts internally rather than propagate them to the application--as long as the application follows certain rules. Should we consider finishing that effort? Relevant URLs: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/zodbex/zodbex/Shield http://dev.zope.org/Wikis/DevSite/Proposals/ArmoredCatalog ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [CRITICAL] Conflict Errors, Transactions, Retries,OhMy....
Chris McDonough wrote: On Wed, 2003-05-28 at 21:33, Jeffrey P Shell wrote: I know there are some fixes likely to be in Zope 2.6.2 that may help with the situation, but I'd like to put extra protections around this code regardless of what may be coming in the future. It will only get worse with 2.6.2: the number of conflict errors cause by the sessioning machinery in 2.6.2 is going to go up as compared to 2.6.1 and below. This is because the strategy used to reduce the number of conflict errors used currently causes data desynchronization problems. A while ago I experimented with ways to prevent conflict errors from reaching the application. ZODB has matured since then and it should now be possible to make a Connection class that resolves conflicts internally rather than propagate them to the application--as long as the application follows certain rules. Should we consider finishing that effort? Relevant URLs: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/zodbex/zodbex/Shield http://dev.zope.org/Wikis/DevSite/Proposals/ArmoredCatalog Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
