Re: [ZODB-Dev] undo (and storage interface) brokenness
I agree that it's handy, and that people would be upset if it disappeared. But I also agree that people trust it far more than they should, and that it's really only dependable for developers to roll back transactions -- however, for that, I think it's invaluable (i.e. very valuable). So I think *this* undo should not be in Plone's or other user-facing UI. If there is an undo facility, it should be implemented by way of application-level versioning as Hanno says. -- jean . .. //\\\oo///\\ ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On 24/12/09 03:09, Martin Aspeli wrote: Hanno Schlichting wrote: Unfortunately, I think the people most likely to have used it in a pinch are less likely to be reading this list. If the current API is broken, we should fix it. If that means fixing some of the UI in Zope to use a new API, that's OK too. Few people will have customised this or relied on the old API. If we had to say you only get step-by-step undo from the most recent transaction, then that'd probably be OK too. In this case, we should have a UI that lets you specify a number (undo N transactions) or a limit (undo up to and including transaction X). I think people understand that undoing a single transaction that's not the most recent one is error-prone anyway. Martin Some of my customers are low-budget customers. They rather have me restore some content with a few minutes of work even at the cost (for them) of undoing later transactions. IOW, I'll support Martin's statement : step-by-step undo should be kept if possible. -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On Wednesday 23 December 2009 20:26:13 Jim Fulton wrote: Undo is broken in a number of ways. Does anyone care? Does anyone use undo? First, a disclaimer: what I state below is from Zope 2.8-ish experiences, with its original ZODB version. I hope my mail is still on topic nevertheless. In my company, it is gradually becoming a production-server-setup step to disable (remove via monkey patch) the undo tab from Zope's ZMI, because it happens quite often for someone to click on it on a multi-gigabyte FileStorage via Zeo, causing the whole cluster to go down until someone restarts Zope Zeo or a few hours pass. This is due to the backward seeking done when listing transactions to undo (disk throughput goes down to a crawl) filtered by context path (depending on the path, the given limit of transactions to display will never be reached) and with storage lock taken, effectively bringing the cluster down. So I believe that this performance issue, although being less serious than what you present as technical breakage, is a showstopper by itself in practise (unless it becomes impossible to filter transactions to undo, making the action end in a roughly constant time for a given transaction number). Maybe this would be an argument toward offline-only undo support. Finally, I believe that undo feature is miss-represented in Zope (keep above disclaimer in mind: I don't know about most recent Zopes), because most people I know who used that undo tab are actually interested in undoing transactions modifying context, not transaction done on context (so they actually want to use the efficient history tab, the one using FileStorage object version back-pointers). -- Vincent Pelletier ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] undo (and storage interface) brokenness
I'm looking at undo today because it is complicating ZEO transaction management. Undo is broken in a number of ways. Does anyone care? Does anyone use undo? Undo is broken in the following ways: - The TransactionalUndo resource manager invalidates objects too early. - If undo is mixed with stores, store invalidations may not be handled properly. This is because, with undo, the two-phase commit calls are made on a storage multiple times. This was OK a long long time ago, but not since tpc_vote started returning important information. Of course, mixing undo and stores is problematic in a number of ways, but there's nothing to prevent it now. - Because undo returns affected oids immediately, it's impossible to delay getting the storage lock until tpc_vote is called. This is a performance issue and is less serious than the problems mentioned above. Undo can be fixed, but not in a way that preserves the existing API. I'm tempted to remove undo support, at least from ZEO. So, I repeat: is anyone actually using undo these days? BTW, this analysis reveals a flaw in the storage APIs that I'd like to fix. You're allowed to call tpc_begin multiple times with the same transaction. Extra calls are ignored. Also, calling tpc_finish with a wrong transaction (a transaction other than the one the storage is dealing with or when a storage isn't committing) is ignored. This feels like a bug magnet to me, but it's there, afaict, to support the current broken undo implementation. Changing existing standard storages to be more restrictive (and fixing tests that specifically tested the permissive behavior) doesn't break anything but the current undo implementation. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
We use undo from time to time in a zeo setup. When it works it can be a life saver! :) If it goes away I guess it wouldn't be the end of the world though. -EAD Erik A. Dahl Co-Founder and CTO, Zenoss Inc. phone: 443-837-2597 http://www.zenoss.com On Dec 23, 2009, at 3:26 PM, Jim Fulton wrote: I'm looking at undo today because it is complicating ZEO transaction management. Undo is broken in a number of ways. Does anyone care? Does anyone use undo? Undo is broken in the following ways: - The TransactionalUndo resource manager invalidates objects too early. - If undo is mixed with stores, store invalidations may not be handled properly. This is because, with undo, the two-phase commit calls are made on a storage multiple times. This was OK a long long time ago, but not since tpc_vote started returning important information. Of course, mixing undo and stores is problematic in a number of ways, but there's nothing to prevent it now. - Because undo returns affected oids immediately, it's impossible to delay getting the storage lock until tpc_vote is called. This is a performance issue and is less serious than the problems mentioned above. Undo can be fixed, but not in a way that preserves the existing API. I'm tempted to remove undo support, at least from ZEO. So, I repeat: is anyone actually using undo these days? BTW, this analysis reveals a flaw in the storage APIs that I'd like to fix. You're allowed to call tpc_begin multiple times with the same transaction. Extra calls are ignored. Also, calling tpc_finish with a wrong transaction (a transaction other than the one the storage is dealing with or when a storage isn't committing) is ignored. This feels like a bug magnet to me, but it's there, afaict, to support the current broken undo implementation. Changing existing standard storages to be more restrictive (and fixing tests that specifically tested the permissive behavior) doesn't break anything but the current undo implementation. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On Wed, Dec 23, 2009 at 3:38 PM, Erik Dahl ed...@zenoss.com wrote: We use undo from time to time in a zeo setup. When it works it can be a life saver! :) If it goes away I guess it wouldn't be the end of the world though. Do you use it via Zope 2 or do you call ZODB undo apis directly? Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
Zope 2. -EAD Erik A. Dahl Co-Founder and CTO, Zenoss Inc. phone: 443-837-2597 http://www.zenoss.com On Dec 23, 2009, at 3:40 PM, Jim Fulton wrote: On Wed, Dec 23, 2009 at 3:38 PM, Erik Dahl ed...@zenoss.com wrote: We use undo from time to time in a zeo setup. When it works it can be a life saver! :) If it goes away I guess it wouldn't be the end of the world though. Do you use it via Zope 2 or do you call ZODB undo apis directly? Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On Wed, Dec 23, 2009 at 9:26 PM, Jim Fulton j...@zope.com wrote: Undo is broken in a number of ways. Does anyone care? Does anyone use undo? Speaking from the Zope2/Plone crowd: I'm using it during development with a local file storage at times. In a real ZEO production setup, undo in Plone most of the time only works for a quick: revert the last transaction. For any transaction that happened a while ago our catalog-overuse will cause some persistent object inside the catalog to have changed in a later transaction. Throwing away all changes done in the meantime is usually not practical. For recovery purposes of older data, we have found a bit of manual work and zc.beforestorage to be the better approach. So if undo is gone from ZEO it wouldn't be tragic. If the remove last transaction could be retained, that would be nice. Hanno ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On Dec 23, 2009, at 2:26 PM, Jim Fulton wrote: On Wed, Dec 23, 2009 at 4:20 PM, Jeff Shell j...@bottlerocket.net wrote: Are you asking at the ZODB level, or at a higher level, such as Zope? I'm asking if applications are using undo these days. We use it in our Zope 3 based CMS. We usually only expose a single-level undo and have not exposed any interface to undo more. What do you mean by single undo? Undoing just one transaction? Undoing just one transaction. When doing like an action like 'delete' inside of a folder, we add a link to the end of the message that is displayed on the next page to undo: Deleted 3 items (undo). If 'undo' is clicked, then the next page has a message to redo (undo the undo). It's always meant to undo a single, recent (but maybe not the most recent) transaction, there as a quick oops! recovery mechanism. Once they move on to the next page, the link to 'undo' is gone. Thanks, Jeff Shell j...@bottlerocket.net ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
Jim Fulton wrote: So, I repeat: is anyone actually using undo these days? I haven't used undo (not the APIs nor via any Zope UI) in years. - C ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
Jim Fulton wrote: I'm looking at undo today because it is complicating ZEO transaction management. Undo is broken in a number of ways. Does anyone care? Does anyone use undo? Plone users tend to use Undo quite a lot. If you accidentally delete something, say, it's pretty much your only way out save for a full restore from backup (which would likely be at least a few hours out of date). Undo can be fixed, but not in a way that preserves the existing API. I'm tempted to remove undo support, at least from ZEO. Please, please don't. It'll upset a lot of people by removing an important (if imperfect) safety mechanism. It'll also make stronger the ZODB is too opaque, I can't trust it FUD. I think we can work to make Zope (2?) use a new API for undo is necessary. But *please* don't remove the feature. So, I repeat: is anyone actually using undo these days? I'd say thousands of Plone users, yes. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
Hanno Schlichting wrote: On Wed, Dec 23, 2009 at 9:26 PM, Jim Fultonj...@zope.com wrote: Undo is broken in a number of ways. Does anyone care? Does anyone use undo? Speaking from the Zope2/Plone crowd: I'm using it during development with a local file storage at times. In a real ZEO production setup, undo in Plone most of the time only works for a quick: revert the last transaction. For any transaction that happened a while ago our catalog-overuse will cause some persistent object inside the catalog to have changed in a later transaction. Throwing away all changes done in the meantime is usually not practical. ... although sometimes it is preferable to a full backup restore or living with whatever it is you want to restore (usually a delete). For recovery purposes of older data, we have found a bit of manual work and zc.beforestorage to be the better approach. What is zc.beforestorage? So if undo is gone from ZEO it wouldn't be tragic. If the remove last transaction could be retained, that would be nice. I actually think it would be tragic. Or at least pretty miserable. People who manage high availability setups can probably find other ways (like very frequent backups and good restore routines). For a lot of average installations, it can be a lifesaver. It was a lifesaver for one of my clients about a week ago, for example. The late night phonecall kind of lifesaver. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On Thu, Dec 24, 2009 at 1:34 AM, Martin Aspeli optilude+li...@gmail.com wrote: Hanno Schlichting wrote: Throwing away all changes done in the meantime is usually not practical. ... although sometimes it is preferable to a full backup restore or living with whatever it is you want to restore (usually a delete). Right. That's why we use beforestorage. For recovery purposes of older data, we have found a bit of manual work and zc.beforestorage to be the better approach. What is zc.beforestorage? Look at its PyPi page http://pypi.python.org/pypi/zc.beforestorage Basically it's a convenient way to open a database as it was at a certain point in time. So for Plone in practice you get a separate copy of the live database or from a backup, open it via beforestorage at the time you want and export the content you are looking for via a zexp. Put that zexp into the import of your live server and import the content. You usually have to reindex the content, as you cannot take the catalog entries with you. This works nicely for restoring deleted content and we have done so a number of times. If there's some unwanted changes made to individual content items, you can revert those via CMFEditions / application level versioning. So if undo is gone from ZEO it wouldn't be tragic. If the remove last transaction could be retained, that would be nice. I actually think it would be tragic. Or at least pretty miserable. In practice it hardly ever works. Relying on it as a substitution for backup or tested restore procedures is dangerous. People who manage high availability setups can probably find other ways (like very frequent backups and good restore routines). For a lot of average installations, it can be a lifesaver. It was a lifesaver for one of my clients about a week ago, for example. The late night phonecall kind of lifesaver. Frequent backups and good restore routines have nothing to do with high availability setups. They are a basic requirement for any type of setup. But even if you don't do frequent backups, as long as you have not packed your database, your data is still there and can be retrieved without too much hassle. But doing partial database recovery is something that requires a good deal of knowledge of the underlying application and database. Hanno ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
Hanno Schlichting wrote: What is zc.beforestorage? Look at its PyPi page http://pypi.python.org/pypi/zc.beforestorage Thanks for the pointer! In practice it hardly ever works. Relying on it as a substitution for backup or tested restore procedures is dangerous. I know that a procedure based on backups and something like zc.beforestorage is going to be better. It's also an order of magnitude more difficult for people to set up and recover from. I think we'd see a pretty big backlash if we just removed the Undo feature, imperfect though it is. Unfortunately, I think the people most likely to have used it in a pinch are less likely to be reading this list. If the current API is broken, we should fix it. If that means fixing some of the UI in Zope to use a new API, that's OK too. Few people will have customised this or relied on the old API. If we had to say you only get step-by-step undo from the most recent transaction, then that'd probably be OK too. In this case, we should have a UI that lets you specify a number (undo N transactions) or a limit (undo up to and including transaction X). I think people understand that undoing a single transaction that's not the most recent one is error-prone anyway. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Aspeli wrote: Hanno Schlichting wrote: What is zc.beforestorage? Look at its PyPi page http://pypi.python.org/pypi/zc.beforestorage Thanks for the pointer! There is also dm.historical when you need to deal with earlier versions of objects: http://pypi.python.org/pypi/dm.historical/1.1 Andreas -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAksy4iYACgkQCJIWIbr9KYxbFgCfQYanCFqC7dNmlU4P3M9esurK Y9IAn39geL7s4MltJZd5pmtyir9pbeEf =ib3E -END PGP SIGNATURE- attachment: lists.vcf___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
I'd like to add my vote to retaining Undo, I think it's a crucial feature. Changing the API is not a problem, and changing the UI could probably be an improvement. I use it all the time, especially when fiddling with customer datatabases for upgrades or bugfixes. -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
