Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters
Christian Theune wrote: >> Traceback (most recent call last): >> File "ZODB3-3.9.3-py2.6-linux-i686.egg/ZEO/zrpc/connection.py", line >> 581, in handle_request >> ret = meth(*args) >> File "gocept.zeoraid-1.0b6-py2.6.egg/gocept/zeoraid/storage.py", line >> 219, in history >> assert version is '' >> >> There is no ZEOStorage instance involved in this as far as I can tell. >> What am I missing? > > Check the "setup_delegation" method: the history method is patched > through directly so your traceback doesn't see the ZEOStorage anymore. OK, I've committed tests and patches to deal with this issue and the similar issue in ZEOStorage308Adapter on the 3.9 branch. Should I merge these changes to the trunk too? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] 3.9.4 release?
Hi Jim, It would be great if the two recent fixes I committed could make it into a 3.9.4 release some time soon... What can I do to help this happen? Assuming I do anything necessary, when could you do this release? (or give Christian permission to do it :-) ) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
On Thu, Nov 19, 2009 at 7:01 PM, Chris Withers wrote: > Jim Fulton wrote: >> >> There's nothing official or supported about a backup solution without >> automated tests. >> >> So I guess there isn't one. > > Right, so what does Zope Corp use? We use ZRS, of course. I'd prefer that there be a file-storage backup solution out of the box. repozo is the logical choice. It sounds like it needs some love though. This isn't something I'd likely get to soon. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
Jim Fulton wrote: > On Thu, Nov 19, 2009 at 7:01 PM, Chris Withers wrote: >> Jim Fulton wrote: >>> There's nothing official or supported about a backup solution without >>> automated tests. >>> >>> So I guess there isn't one. >> Right, so what does Zope Corp use? > > We use ZRS, of course. Well, ZRS solves the HA challenge the same way as zeoraid, if I understand correctly, but what about offsite backups and the like? The project I'm currently working on uses repozo to create backups that: - get hoovered by the hosting provider's backup mechanisms and rotated offsite daily - get sprayed by rsync over ssh to a DR site on another continent How would ZRS solve these problems? > I'd prefer that there be a file-storage backup solution out of the box. > repozo is the logical choice. It sounds like it needs some love though. > This isn't something I'd likely get to soon. I'm not sure how much love repozo needs. It works, and it won't need changing until FileStorage's format changes, which I don't see happening any time soon. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
On Fri, Nov 20, 2009 at 5:49 AM, Chris Withers wrote: > Hi Jim, > > It would be great if the two recent fixes I committed could make it into a > 3.9.4 release some time soon... > > What can I do to help this happen? > Assuming I do anything necessary, when could you do this release? > (or give Christian permission to do it :-) ) I need to review the changes before the release. I'll probably reject the repozo change without an automated test. In the future, please don't check changes directly into trunk or a release branch. Check them into a development branch. I'll review and merge. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
2009/11/20 Chris Withers : > Jim Fulton wrote: >> On Thu, Nov 19, 2009 at 7:01 PM, Chris Withers >> wrote: >>> Jim Fulton wrote: There's nothing official or supported about a backup solution without automated tests. So I guess there isn't one. >>> Right, so what does Zope Corp use? >> >> We use ZRS, of course. > > Well, ZRS solves the HA challenge the same way as zeoraid, if I > understand correctly, but what about offsite backups and the like? > > The project I'm currently working on uses repozo to create backups that: > > - get hoovered by the hosting provider's backup mechanisms and rotated > offsite daily > > - get sprayed by rsync over ssh to a DR site on another continent > > How would ZRS solve these problems? > >> I'd prefer that there be a file-storage backup solution out of the box. >> repozo is the logical choice. It sounds like it needs some love though. >> This isn't something I'd likely get to soon. > > I'm not sure how much love repozo needs. It works, and it won't need > changing until FileStorage's format changes, which I don't see happening > any time soon. Maybe this test I added for analyze.py could be a helpful template. http://zope3.pov.lt/trac/changeset/100422 Laurence ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
Jim Fulton wrote: > I need to review the changes before the release. I'll probably reject the > repozo change without an automated test. Are you serious? You'd rather have a broken tool than one that isn't broken on the basis that the existing tests aren't part of the test suite that gets run by default?! > In the future, please don't check changes directly into trunk or a > release branch. > Check them into a development branch. I'll review and merge. Sure, although these are both miniscule changes... I've learned my lesson, I won't try and contribute to ZODB development in future... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
On Fri, Nov 20, 2009 at 9:32 AM, Chris Withers wrote: > Jim Fulton wrote: >> >> On Thu, Nov 19, 2009 at 7:01 PM, Chris Withers >> wrote: >>> >>> Jim Fulton wrote: There's nothing official or supported about a backup solution without automated tests. So I guess there isn't one. >>> >>> Right, so what does Zope Corp use? >> >> We use ZRS, of course. > > Well, ZRS solves the HA challenge the same way as zeoraid, if I understand > correctly, but what about offsite backups and the like? We replicate to off-site secondaries. > The project I'm currently working on uses repozo to create backups that: > > - get hoovered by the hosting provider's backup mechanisms and rotated > offsite daily > > - get sprayed by rsync over ssh to a DR site on another continent > > How would ZRS solve these problems? I don't know what "these" refers to. ZRS secondaries are hot backups. They can be on- or off-site. >> I'd prefer that there be a file-storage backup solution out of the box. >> repozo is the logical choice. It sounds like it needs some love though. >> This isn't something I'd likely get to soon. > > I'm not sure how much love repozo needs. It works, and it won't need > changing until FileStorage's format changes, which I don't see happening any > time soon. It needs automated tests, at least. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
On Fri, Nov 20, 2009 at 9:39 AM, Chris Withers wrote: > Jim Fulton wrote: >> >> I need to review the changes before the release. I'll probably reject the >> repozo change without an automated test. > > Are you serious? You'd rather have a broken tool than one that isn't broken > on the basis that the existing tests aren't part of the test suite that gets > run by default?! Yes. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
[Jim Fulton] >> I need to review the changes before the release. I'll probably reject the >> repozo change without an automated test. [Chris Withers]\ > Are you serious? You'd rather have a broken tool than one that isn't > broken on the basis that the existing tests aren't part of the test > suite that gets run by default?! That's what he said -- and you made him repeat it several times by now. >> In the future, please don't check changes directly into trunk or a >> release branch. >> Check them into a development branch. I'll review and merge. > Sure, although these are both miniscule changes... > > I've learned my lesson, I won't try and contribute to ZODB development > in future... I think that's the wrong lesson to take. repozo had several fatal bugs when I inherited it, and part of the reason is that there were no tests of any kind. As the checkin comment said when I added testrepozo.py: Better Than Nothing -- which is what we had before. There wasn't time then to finish the job (i.e., to automate the testing and dope out some way to make failure output more /helpful/ than just "guts don't match"). What I checked in then was essential, though, to verify the slew of bugfixes that went in around the same time. Alas, 5 1/2 years later, repozo testing apparently remains just barely "Better Than Nothing", but that's really not good enough for a supported approach. So the right lessons are: (1) to do development on a development branch; and, (2) to finish the job I started if repozo is to be treated as more than just another random piece of flotsam in the ZODB ocean. If nobody is interested in doing that work, fine by me ;-) ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
On Nov 20, 2009, at 7:32 AM, Chris Withers wrote: > Jim Fulton wrote: >> I'd prefer that there be a file-storage backup solution out of the box. >> repozo is the logical choice. It sounds like it needs some love though. >> This isn't something I'd likely get to soon. > > I'm not sure how much love repozo needs. It works, and it won't need > changing until FileStorage's format changes, which I don't see happening > any time soon. For what it's worth, I've dealt with broken / unloved / mildly-out-of-date ZODB scripts by making a new package that my little company uses internally. It allowed me to get some useful tools available again, and was independent from any ZODB release cycle. Now I have additional scripts that get generated by my ZODB-related buildouts with names like 'fsdumpx'. You could always explore this option, and even do it better than me by sharing with the community. The ZODB 'scripts' are a wild handful of relics, some useful, some not, some just woefully out of date. It would be helpful to have an independent project that provided new or fixed tools that were maintained, were better command-line citizens, had consistent command line options, etc. There's no reason for such a project to be tied to the longer tail of the ZODB release cycle. So if 'repozo' needs love, even if it's simple love, I would recommend that someone make a 'z3c.zodbtools' or 'z3c.repozo' or 'z3c.fsbackup' project. Hell, I'll look at the package I've made and see if I can do something along those lines as a starting/example point. (I believe my zodbtools package has some dependencies on a couple of internal packages that can not be shared, but I could scrub those uses out). — Jeff Shell j...@bottlerocket.net ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
On Fri, Nov 20, 2009 at 9:32 AM, Chris Withers wrote: ... > I'm not sure how much love repozo needs. It works, and it won't need > changing until FileStorage's format changes, which I don't see happening any > time soon. It just occurred to me that repozo doesn't support blobs. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
Tim Peters wrote: > That's what he said -- and you made him repeat it several times by now. Yes, I find it hard to believe that someone would deliberately break something that someone else has taken the trouble to fix (and run the tests for!)... >> I've learned my lesson, I won't try and contribute to ZODB development >> in future... > > I think that's the wrong lesson to take. repozo had several fatal > bugs when I inherited it, and part of the reason is that there were no > tests of any kind. As the checkin comment said when I added > testrepozo.py: > > Better Than Nothing -- which is what we had before. > > There wasn't time then to finish the job (i.e., to automate the > testing and dope out some way to make failure output more /helpful/ > than just "guts don't match"). What I checked in then was essential, > though, to verify the slew of bugfixes that went in around the same > time. > > Alas, 5 1/2 years later, repozo testing apparently remains just barely > "Better Than Nothing", but that's really not good enough for a > supported approach. I'm not arguing with the above, and I'm not asking for anything more supported than already exists. However, requiring someone to completely rewrite a test suite for software that they're never needed to understand on the basis that they corrected some imports to make them compatible with a newer version of python seems unreasonable. > So the right lessons are: (1) to do development on a development > branch; Where is the development process that requires this documented? > and, (2) to finish the job I started if repozo is to be > treated as more than just another random piece of flotsam in the ZODB > ocean. I'm fine with the status quo... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
Jim Fulton wrote: > On Fri, Nov 20, 2009 at 9:32 AM, Chris Withers wrote: > ... >> I'm not sure how much love repozo needs. It works, and it won't need >> changing until FileStorage's format changes, which I don't see happening any >> time soon. > > It just occurred to me that repozo doesn't support blobs. Indeed, I've never been comfortable with blobs due to their spotty support in older tools and the need to make a filesystem HA for them to be used in a HA environment. As a result, I don't use them, so I'm okay with repozo not suppporting them. Yes, I'd love it if repozo *did* support them, but I wouldn't have the faintest idea of how to do that, even if I did have time to work on it... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
Jeff Shell wrote: > For what it's worth, I've dealt with broken / unloved / > mildly-out-of-date ZODB scripts by making a new package that my > little company uses internally. I don't agree with this because it's fragmenting a tiny development group even further, no to mention creating private forks of tools that mean everyone loses in terms of eyeballing... > It allowed me to get some useful > tools available again, Such as? And what stopped you working on them back in the ZODB source tree? > and was independent from any ZODB release > cycle. I have no problems with the ZODB release cycle... > You could always explore this option, and even do it better than me > by sharing with the community. This just feels wrong to me. If I *was* changing a script's behaviour such that it needed new tests, I'd make sure those were automatically run. The changes to repozo didn't require any new tests, the bug was provoked just by running the existing tests with python 2.6. > So if 'repozo' needs love, even if it's simple love, I would > recommend that someone make a 'z3c.zodbtools' or 'z3c.repozo' or > 'z3c.fsbackup' project. I couldn't agree less. This kind of fragmentation helps no-one. It's something I'd only consider as a last resort, and then the most likely action is just to roll my own sdist with a funky version number and stick it on the customer's index server. > Hell, I'll look at the package I've made and > see if I can do something along those lines as a starting/example > point. I'd much prefer to see your fixes worked back into the 3.9 branch and trunk of ZODB... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] repozo, neither official nor supported, apparently...
2009/11/20 Jim Fulton : > On Fri, Nov 20, 2009 at 9:32 AM, Chris Withers wrote: > ... >> I'm not sure how much love repozo needs. It works, and it won't need >> changing until FileStorage's format changes, which I don't see happening any >> time soon. > > It just occurred to me that repozo doesn't support blobs. This was touched on in a thread "Backing up Data.fs and blob directory": https://mail.zope.org/pipermail/zodb-dev/2008-September/012094.html While there is no direct support in repozo, the approach of first taking a repozo backup followed by a blob directory backup works so long as you do not pack between the repozo and blob backups. (Blobs newer than the repozo backup are safely ignored.) Laurence ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jim Fulton wrote: > On Fri, Nov 20, 2009 at 5:49 AM, Chris Withers wrote: >> Hi Jim, >> >> It would be great if the two recent fixes I committed could make it into a >> 3.9.4 release some time soon... >> >> What can I do to help this happen? >> Assuming I do anything necessary, when could you do this release? >> (or give Christian permission to do it :-) ) > > I need to review the changes before the release. I'll probably reject the > repozo change without an automated test. - -sys.maxint. In what possible universe are we better off not being able to run repozo under Python 2.6? That fix is trivial to inspect for correctness, changes no behavior, and is like dozens of other Python 2.6 compatiblitity changes made in our codebase. Repozo is *the* documented way of backing up filestorages: with Zope 2.12 and later now running on Python 2.6, we should just tell people "Sorry, you can't do backups"? Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksGw0cACgkQ+gerLs4ltQ4uHwCgrxg3tCLKKJahyRCgE70AgW1Z wt8An01/eLljzLCtcSDlVd5vfe8jgAhl =DVF7 -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tres Seaver wrote: > Jim Fulton wrote: >> On Fri, Nov 20, 2009 at 5:49 AM, Chris Withers >> wrote: >>> Hi Jim, >>> >>> It would be great if the two recent fixes I committed could make it into a >>> 3.9.4 release some time soon... >>> >>> What can I do to help this happen? >>> Assuming I do anything necessary, when could you do this release? >>> (or give Christian permission to do it :-) ) >> I need to review the changes before the release. I'll probably reject the >> repozo change without an automated test. > > -sys.maxint. In what possible universe are we better off not being able > to run repozo under Python 2.6? That fix is trivial to inspect for > correctness, changes no behavior, and is like dozens of other Python 2.6 > compatiblitity changes made in our codebase. > > Repozo is *the* documented way of backing up filestorages: with Zope > 2.12 and later now running on Python 2.6, we should just tell people > "Sorry, you can't do backups"? I just checked in a stupid test to ensure that repozo can be imported. - --- src/ZODB/scripts/tests.py (revision 105913) +++ src/ZODB/scripts/tests.py (working copy) @@ -26,6 +26,11 @@ (re.compile('hash=[0-9a-f]{40}'), 'hash=b16422d09fabdb45d4e4325e4b42d7d6f021d3c3')]) +class RepozoTests(unittest.TestCase): + +def test_importability(self): +from ZODB.scripts import repozo + def test_suite(): return unittest.TestSuite(( doctest.DocFileSuite( Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksGyKkACgkQ+gerLs4ltQ5NDACfXI8srJ2V1K/g3iwB8OOtv4Qb /8kAoJ6ELmc1vQzePnh4/w71Xw/TJy8O =6+vL -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
On Fri, Nov 20, 2009 at 11:49 AM, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Tres Seaver wrote: >> Jim Fulton wrote: >>> On Fri, Nov 20, 2009 at 5:49 AM, Chris Withers >>> wrote: Hi Jim, It would be great if the two recent fixes I committed could make it into a 3.9.4 release some time soon... What can I do to help this happen? Assuming I do anything necessary, when could you do this release? (or give Christian permission to do it :-) ) >>> I need to review the changes before the release. I'll probably reject the >>> repozo change without an automated test. >> >> -sys.maxint. In what possible universe are we better off not being able >> to run repozo under Python 2.6? You can run repozo under Python 2.6 now. You'll get a deprecation warning. >> That fix is trivial to inspect for >> correctness, changes no behavior, and is like dozens of other Python 2.6 >> compatiblitity changes made in our codebase. >> >> Repozo is *the* documented way of backing up filestorages: Where is that? >> with Zope >> 2.12 and later now running on Python 2.6, we should just tell people >> "Sorry, you can't do backups"? No, we should tell them they should use a mostly untested tool to do backups that doesn't work with blobs. That should inspire confidence. Does Zope 2.12 use Blobs? > I just checked in a stupid test to ensure that repozo can be imported. > > - --- src/ZODB/scripts/tests.py (revision 105913) > +++ src/ZODB/scripts/tests.py (working copy) > @@ -26,6 +26,11 @@ > (re.compile('hash=[0-9a-f]{40}'), > 'hash=b16422d09fabdb45d4e4325e4b42d7d6f021d3c3')]) > > +class RepozoTests(unittest.TestCase): > + > + def test_importability(self): > + from ZODB.scripts import repozo > + I appreciate your dedication to quality. That test passes under ZODB 3.9.3 with Python 2.6. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
Jim Fulton wrote: >> +class RepozoTests(unittest.TestCase): >> + >> +def test_importability(self): >> +from ZODB.scripts import repozo >> + > > I appreciate your dedication to quality. > > That test passes under ZODB 3.9.3 with Python 2.6. ...but it will emit a deprecation warning, which I thought we considered to be a test failure? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Zope 2.12 and blobs
Jim Fulton wrote: > No, we should tell them they should use a mostly untested tool to do backups > that doesn't work with blobs. That should inspire confidence. > > Does Zope 2.12 use Blobs? Not by default, no. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jim Fulton wrote: > On Fri, Nov 20, 2009 at 11:49 AM, Tres Seaver wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> Tres Seaver wrote: >>> Jim Fulton wrote: On Fri, Nov 20, 2009 at 5:49 AM, Chris Withers wrote: > Hi Jim, > > It would be great if the two recent fixes I committed could make it into a > 3.9.4 release some time soon... > > What can I do to help this happen? > Assuming I do anything necessary, when could you do this release? > (or give Christian permission to do it :-) ) I need to review the changes before the release. I'll probably reject the repozo change without an automated test. >>> -sys.maxint. In what possible universe are we better off not being able >>> to run repozo under Python 2.6? > > You can run repozo under Python 2.6 now. You'll get a deprecation warning. So we have an annoyance rather than a catastrophe: that helps some. It will still break on 2.7 without the trivial fix. >>> That fix is trivial to inspect for >>> correctness, changes no behavior, and is like dozens of other Python 2.6 >>> compatiblitity changes made in our codebase. >>> >>> Repozo is *the* documented way of backing up filestorages: > > Where is that? http://wiki.zope.org/ZODB/FileStorageBackup >>> with Zope >>> 2.12 and later now running on Python 2.6, we should just tell people >>> "Sorry, you can't do backups"? > > No, we should tell them they should use a mostly untested tool to do backups > that doesn't work with blobs. That should inspire confidence. We can use rsync to back up blobs safely / losslessly without restarting the ZEO server: we can't do that with filestorages. > Does Zope 2.12 use Blobs? If configured, sure. >> I just checked in a stupid test to ensure that repozo can be imported. >> >> - --- src/ZODB/scripts/tests.py (revision 105913) >> +++ src/ZODB/scripts/tests.py (working copy) >> @@ -26,6 +26,11 @@ >> (re.compile('hash=[0-9a-f]{40}'), >> 'hash=b16422d09fabdb45d4e4325e4b42d7d6f021d3c3')]) >> >> +class RepozoTests(unittest.TestCase): >> + >> +def test_importability(self): >> +from ZODB.scripts import repozo >> + > > I appreciate your dedication to quality. > > That test passes under ZODB 3.9.3 with Python 2.6. It emits the same DeprecationWarning (the "bug" Withers fixed). I did have to munge the 'test_suite' dinosaur as well. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAksG4bIACgkQ+gerLs4ltQ7czQCgzAjQX0TUEJKSNo92+uv8UWOw VmEAoJ9Xo7WRamCQyuwmY7Kacrqk/Krz =/5ad -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
On Fri, Nov 20, 2009 at 1:36 PM, Tres Seaver wrote: ... with Zope 2.12 and later now running on Python 2.6, we should just tell people "Sorry, you can't do backups"? >> >> No, we should tell them they should use a mostly untested tool to do backups >> that doesn't work with blobs. That should inspire confidence. > > We can use rsync to back up blobs safely / losslessly without restarting > the ZEO server: we can't do that with filestorages. Depending on backup schedules, I would not rely on the rsync backed up blobs and repozo backups remaining consistent. Packing complicates this quite a bit. >> Does Zope 2.12 use Blobs? > > If configured, sure. I would not trust repozo + blob backups. > >>> I just checked in a stupid test to ensure that repozo can be imported. >>> >>> - --- src/ZODB/scripts/tests.py (revision 105913) >>> +++ src/ZODB/scripts/tests.py (working copy) >>> @@ -26,6 +26,11 @@ >>> (re.compile('hash=[0-9a-f]{40}'), >>> 'hash=b16422d09fabdb45d4e4325e4b42d7d6f021d3c3')]) >>> >>> +class RepozoTests(unittest.TestCase): >>> + >>> + def test_importability(self): >>> + from ZODB.scripts import repozo >>> + >> >> I appreciate your dedication to quality. >> >> That test passes under ZODB 3.9.3 with Python 2.6. > > It emits the same DeprecationWarning (the "bug" Withers fixed). I did > have to munge the 'test_suite' dinosaur as well. Fair enough. Given that this provokes the symptom that was fixed. I can live with this. I also would fear to rely on repozo in it's current state, but I don't have to. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
On Nov 20, 2009, at 11:52 AM, Jim Fulton wrote: >> If configured, sure. > > I would not trust repozo + blob backups. I don't trust Blobs. A year ago, I reverted all of the code I had for our CMS that supported Blobs as it was impossible to copy their data in a basic Zope 3 copy/past/move. I think that may have been fixed, but since there's practically zero documentation on how to use blobs, use them wisely, use them well, and use them in fairly plain Zope 3-ish applications, I lost confidence. > Fair enough. Given that this provokes the symptom that was fixed. > I can live with this. > > I also would fear to rely on repozo in it's current state, but I don't have > to. We live in fear of most of the tools in their current state. So they pass the tests - whoopee! But the documentation is so bare and minimal that we (Bottlerocket) still don't properly configure anything. I guessed at some default values and those get copied around, hoping for the best. We've encountered problems with ZEO in the long-ago past that I'm sure were due to configuration misunderstandings that still hold over us to this day. We use ZEO, but I don't think we use it well. Last time I looked for documentation, many of the documents seemed either from the early Zope 2 era or came from the Plone community. While thankful for some of the documents from the Plone community, we're not a Plone shop, nor a Zope 2 shop any more (except for a couple of customers). Some of the tools they reference are unavailable to us. I still don't know what the data from zc.monitor means or how we can use that to adjust ZEO / ZODB configuration settings to improve performance of a site. So while the tests can make us all confident that the ZODB/ZEO system works, how about some documentation coverage that could help us be confident that we're using it correctly? For what it's worth, I rely on but also somewhat fear repozo - not because of its tests or lack thereof (it seems to work for me!), but because documentation and help for the scripts are lacking (fortunately, 'repozo' is the one exception. 'fsrefs' and 'fsdump' are mysterious. There's a lot of other tools in ZODB.scripts that seem rather antiquated, or intimately tied to Zope 2. But chances are that no one even knows those tools are there). Thanks, Jeff Shell j...@bottlerocket.net ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] 3.9.4 release?
On 11/20/2009 08:07 PM, Jeff Shell wrote: > On Nov 20, 2009, at 11:52 AM, Jim Fulton wrote: >>> If configured, sure. >> >> I would not trust repozo + blob backups. > > I don't trust Blobs. A year ago, I reverted all of the code I had for our CMS > that supported Blobs as it was impossible to copy their data in a basic Zope > 3 copy/past/move. I think that may have been fixed, but since there's > practically zero documentation on how to use blobs, use them wisely, use them > well, and use them in fairly plain Zope 3-ish applications, I lost confidence. I think I just found a way to fix this particular issue on the ZODB-level for good. I re-opened https://bugs.edge.launchpad.net/zodb/+bug/240381 and assigned it to myself. I'll see if I can do something over the weekend. Christian -- Christian Theune · c...@gocept.com gocept gmbh & co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] blobs
Jeff Shell wrote: > On Nov 20, 2009, at 11:52 AM, Jim Fulton wrote: >>> If configured, sure. >> I would not trust repozo + blob backups. > > I don't trust Blobs. A year ago, I reverted all of the code I had for our CMS > that supported Blobs as it was impossible to copy their data in a basic Zope > 3 copy/past/move. I think that may have been fixed, but since there's > practically zero documentation on how to use blobs, use them wisely, use them > well, and use them in fairly plain Zope 3-ish applications, I lost confidence. That fairly accurately sums up how I feel about blobs... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] errors logged at info in ZEO's connection
Hi All (although I guess predominantly Jim), ZEO/zprc/connection.py contains this rather odd piece of code: raise except Exception, msg: if not isinstance(msg, self.unlogged_exception_types): self.log("%s() raised exception: %s" % (name, msg), logging.INFO, exc_info=True) Any objections to me changing this on the 3.9 branch and trunk to: raise except Exception, msg: if not isinstance(msg, self.unlogged_exception_types): self.logger.exception( "%s() raised exception: %s" % (name, msg), ) ...and making sure there's a test of some sort that demonstrates this? It seems a bit bizarre that exceptions are logged at INFO rather than ERROR... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] errors logged at info in ZEO's connection
On Fri, Nov 20, 2009 at 3:15 PM, Chris Withers wrote: > Hi All (although I guess predominantly Jim), > > ZEO/zprc/connection.py contains this rather odd piece of code: > > raise > except Exception, msg: > if not isinstance(msg, self.unlogged_exception_types): > self.log("%s() raised exception: %s" % (name, msg), > logging.INFO, exc_info=True) > > Any objections to me changing this on the 3.9 branch and trunk to: > > raise > except Exception, msg: > if not isinstance(msg, self.unlogged_exception_types): > self.logger.exception( > "%s() raised exception: %s" % (name, msg), > ) > > ...and making sure there's a test of some sort that demonstrates this? Yes, I object on 2 levels. First, as I said earlier, all check ins must go into branches for my review and merge. The branch should be from the 3.9 branch. Second, the change should simply change the level passed to self.log. The log method adds a log label that can be very useful. > It seems a bit bizarre that exceptions are logged at INFO rather than > ERROR... Agreed. A simple fix and test on a branch would be appreciated. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] errors logged at info in ZEO's connection
Jim Fulton wrote: >> It seems a bit bizarre that exceptions are logged at INFO rather than >> ERROR... > > Agreed. A simple fix and test on a branch would be appreciated. Done: http://svn.zope.org/ZODB/branches/chrisw-error_logging/ It would be great if this, too, could make it in a 3.9.4 release... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev