Re: [Zope-dev] misterious btree issue
[Jim Fulton] In particular, it might have a reference to the key in an internal BTree node. (I don't know this to be true, but I wouldn't be surprised if it was true.) It was true when I was working on BTrees ... here, from http://svn.zope.org/ZODB/trunk/src/BTrees/Development.txt?rev=85198view=markup ... + When a key is deleted, it's physically removed from the bucket it's in, but this doesn't propagate back up the tree: since keys in interior nodes only serve to guide searches, it's OK-- and saves time --to leave stale keys in interior nodes. ... If this is the case, we can definitely fix it. Right, just delete that paragraph ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] difference between OOSet and OOTreeSet?
[Chris Withers] Wondering if someone could tell me the difference between an OOSet and an OOTreeSet? OOSet is to OOTreeSet as OOBucket is to OOBTree. An OOTreeSet is built out of leaf-level OOSets in exactly the same way an OOBTree is built out of leaf-level OOBuckets. More at: http://wiki.zope.org/ZODB/guide/node6.html#SECTION00063 They seem to have different interfaces ? and yet seem to be used in similar circumstances in PluginIndexes/common/UnIndex.py... I'm looking for a set-like data structure which will likely get pretty large over time and so I don't want the whole data structure written to disk each time I add a new item to the set. What should I be using? OOTreeSet. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Maintainer of Zope 2 Windows builds?
[Andreas Jung] Who is currently in charge or who feels responsible for the Windows builds? [Sidnei da Silva] Tim Peters. [Chris Withers] Er, no. Tim isn't even at Zope Corp anymore, as far as I know... [Sidnei] That shouldn't be related, should it? Don't know about should be, but it /is/ related ;-) I had job-related reasons before to make at least minimal ongoing efforts toward keeping the Zope test suites happy on Windows, and that in turn meant I kept many full checkouts of various Zopes current, at least ran the tests routinely, gave real attention to discussions of possible Windows glitches, built Windows installers routinely, and so on. But I don't do anything related to web development anymore, and bit rot is setting in wrt my once-encyclopedic knowledge of the umpteen quirky build procedures for the umpteen versions of Windows Zope. When, months ago, someone else volunteered to take over the Zope3 Windows builds, and actually followed up on it (yay!), I mentally resigned from these tasks. You (ZC, ZF, the community) want someone who /uses/ Zope on Windows to make Windows releases. It was at best half nuts that I kept doing it after building testing an installer once each N weeks became my only contact with Zope. I built the last couple of Zope 2.9.x release, but my access to a Windows build environment is gone now... Odd. I believe Tim built 2.9.4 because you could not? I don't remember, but it's possible. I'm still willing to build an installer when there's no alternative, but it's long past time that was treated as a last-ditch fallback instead of business as usual. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Windows Installers
[Sidnei da Silva] Seems like we are seeing a shortage of volunteers for building the Zope Installers for Windows. IOW, nothing has changed :-) I haven't seen Tim Peters around for a while, and Chris Withers seems to have stepped down now. I'm around, but don't pay particular attention to all the Zope releases. If someone needs a Windows installer, I'm usually willing to make one (but probably won't notice a need for one without being poked). I've chatted with Alan Runyan and he agreed to dedicate a person for doing this in exchange of a little publicity. Quoting him: Our build system has become quite extensive over the past two years. I would love to build the Zope installers. I would also appreciate some attribution on the first graphic that is displayed. Basically at the bottom about 20% of the graphic would say: '[LOGO] windows installer courtesy of Enfold System' Not sure on exactly what that means, but doubt anyone would object, provided he's also willing to volunteer the time to produce the graphic and wrestle with InnoSetup to get it displayed. Another alternative would be to add the installer as build to the buildbot @ zope.org, then we would have 'nightly' installer builds and once a release was cut one could just take one of these installers and declare it official. Thoughts? Building the installer is usually easy. The hour or so it takes to make a real Windows release is usually consumed by testing, including manual testing of running as a Windows service. Sometimes there are also Windows-specific problems running the tests from an installation tree that never showed up running from a checkout tree. Of course if nobody cares about testing a release, a nightly installer build would work fine ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Windows Installers
[Sidnei da Silva[ So we need a 2.8.8 and 2.9.4 installers. OK, I uploaded a Windows installer for 2.8.8. There were two -all test failures: ERROR: test_OFS_ObjectManager__importObjectFromFile_xml (OFS.tests.test_XMLExportImport.XMLExportImportTests) -- Traceback (most recent call last): File C:\Code\Zope2.8\inst\build\lib\python\OFS\tests\test_XMLExportImport.py, line 102, in test_OFS_ObjectManager__i mportObjectFromFile_xml sub._importObjectFromFile(ostream.name, 0, 0) File C:\Code\Zope2.8\inst\build\lib\python\OFS\ObjectManager.py, line 592, in _importObjectFromFile ob=connection.importFile( File C:\Code\Zope2.8\inst\build\lib\python\ZODB\ExportImport.py, line 59, in importFile f = open(f, 'rb') IOError: [Errno 13] Permission denied: 'c:\\docume~1\\owner\\locals~1\\temp\\tmppr7uqt.xml' == ERROR: test_export_import_as_file_idempotent (OFS.tests.test_XMLExportImport.XMLExportImportTests) -- Traceback (most recent call last): File C:\Code\Zope2.8\inst\build\lib\python\OFS\tests\test_XMLExportImport.py, line 75, in test_export_import_as_file _idempotent newobj = importXML(connection, ostream.name) File C:\Code\Zope2.8\inst\build\lib\python\OFS\XMLExportImport.py, line 104, in importXML file=open(file) IOError: [Errno 13] Permission denied: 'c:\\docume~1\\owner\\locals~1\\temp\\tmpgj2cnm.xml' -- Ran 6229 tests in 1765.442s FAILED (errors=2) Note 2.9.4 changed the build procedure to be the same as 2.8.x again. I've made the changes, and hopefully it should work just fine. The 2.9.4 build started failing here: ... mkdir -p /cygdrive/c/Code/Zope2.9/inst/src cd /cygdrive/c/Code/Zope2.9/inst/src tar xzf ../tmp/Zope-2.9.4.tgz gzip: stdin: unexpected end of file tar: Child returned status 1 tar: Error exit delayed from previous errors make: *** [src/Zope-2.9.4/inst/configure.py] Error 2 I renamed the Zope 2.9.4 tarball download to worm around that, and then it failed here: mkdir -p /cygdrive/c/Code/Zope2.9/inst/src cd /cygdrive/c/Code/Zope2.9/inst/src tar xzf ../tmp/Zope-2.9.4.tgz touch src/Zope-2.9.4/inst/configure.py touch: cannot touch `src/Zope-2.9.4/inst/configure.py': No such file or directory make: *** [src/Zope-2.9.4/inst/configure.py] Error 1 Again it's fatally confused about whether the Zope tarball and/or unpacked directory root does or does not have -final in its name. I didn't repair this, I just wormed around it again (I have no idea what the /intent/ is now). Finally, the code for finding Inno Setup in common.mk was broken, trying to feed a native Windows path to the Cygwin bash shell. I replaced that on the tag (but not the branch) with similar (but working ;-)) code from the 2.8.8 build, essentially reverting this patch: http://svn.zope.org/Zope/branches/2.9/inst/WinBuilders/mk/common.mk?rev=69118r1=65837r2=69118 After that, everything worked fine, although the functional tests crapped out. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope 2.9.4
[Andreas Jung] ... I think we should make another internal ZODB release. The current svn:externals for the ZODB (and other modules) use a revision number instead of a tag. This reminds me of some former discussion whether to use revision numbers or tags...what was the result of this discussion. I am very much in favor of using tag names...revision number tell you nothing...provide at least a reasonable version information...opinions? The things people complain about sometimes astonish me -- just as the things I complain about sometimes astonish others :-) I used tags for ZODB until I gave in to complaints about that, and switched to using revision numbers. The real complaint about using a tagged external is that when the tag changes, SVN isn't smart enough to do an incremental update. Instead, when you update after an external tag changes: - It wholly deletes you current checkout of the external. If non-version-controlled files (like .pyc) happen to be sitting in the directories, this leaves behind useless OLD directories. - It does a complete checkout of the new tag. This generally takes more time. And forces a recompile too even if no C code in the external has actually changed, It's true that changing an external revision number instead suffers none of those drawbacks.Like I cared ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope 2.9.4
[Tim] I used tags for ZODB until I gave in to complaints about that, and switched to using revision numbers. The real complaint about using a tagged external is that when the tag changes, SVN isn't smart enough to do an incremental update. Instead, when you update after an external tag changes: [Andreas] What do you mean with when the tag changes? Not what I said: I said [when] an /external/ tag changes. I mean when you change an external tag _in_ project A _referencing_ the external tagged project B. Here A=Zope and B=ZODB, and the external tags in question are the ones referencing ZODB from within the svn:externals properties of various Zope directories. A tag IMO should *never* change. The revision number associated with a tag within project B indeed shouldn't change. That's not what I'm talking about. I'm talking about how other projects reference project B: they have a choice of: 1. making their own copy of B 2. using svn:externals with a tag for B 3. using svn:externals with a revision number for B Read my msg again with that in mind, and it will make much more sense to you. I was describing the complaints about using choice #2. Zope Corp has tried all of those at various times. They all have different drawbacks, but the only one I found horribly unusable was #1 (because then N different projects have copies of B, and changes to copies of B's code get made from N different projects, and nobody bothers to merge those changes back into B itself -- people changing a copy of B from a different project often don't even _know_ they're changing conceptually external code; in effect, without absurd effort on the part of B's maintainer(s), the world ends up with B plus N forks of B). ... ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope3-dev] Re: [Zope-dev] Re: 64-bit BTrees
[Tres Seaver] ... I would guess that if you could do a census of all the OIDs in all the Datas.fs in the world, a significant majority of them would be instances of classes declared in IOBTree / IIBTree (certainly the bulk of *transaction* records are going to be tied up with them). Provided it still works, people can use ZODB's analyze.py to figure that out. But supposing I flavors of BTrees are the only objects that exist, what follows from that? It's not clear. I can guarantee that multiunion() will run slower, because its workhorse radix sort will need 8 (instead of 4) passes. Beyond that, it requires someone to try it. I'm reminded that when the MEMS Exchange wrote Durus (a kind of ZODB lite ;-): http://www.mems-exchange.org/software/durus/ ) they left their entire BTree implementation coded in Python -- it was fast enough that way. The difference between ZODB's BTree C code pushing 4 or 8 bytes around at a time may well be insignificant overall. If done carefully, pickle sizes probably won't change: cPickle has a large number of ways to pickle integers, and picks the smallest one needed to hold an integer's actual value. Provided the internal getstate() functions are careful to avoid Python longs when possible, bigger pickles won't happen unless more than 32 bits are actually needed to hold an integer. There's also that ZODB's current I trees are badly broken on 64-bit boxes (except for Win64) in at least this way: http://collector.zope.org/Zope/1592 That problem would go away by magic. looks-like-a-case-of-measure-twice-cut-once-ly y'rs - tim ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Windows Binary for 2.8.6?
[Chris Withers] I notice 2.8.6 doesn't have a Windows binary either. What's the build process for that? Try to detect the pattern between this and previous answers ;-): http://svn.zope.org/Zope/branches/Zope-2_8-branch/inst/WinBuilders/README.txt I have a feeling I won't be able to help with that one since it probably need CV++ 6.0 Correct. which I don't have access to :-/ Can someone else pick this one up? If nobody beats me to it, I will, eventually. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.9 releases for Windows?
[Chris Withers] ... How should I run all the Zope tests once I have a candidate build? Section Testing Zope in http://svn.zope.org/Zope/trunk/inst/WinBuilders/README.txt ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.9 releases for Windows?
[Chris Withers] I see there are no Zope 2.9 releases for Windows. There probably should be ;-) There's a Windows installer for Zope 2.9.0 on its download page: http://www.zope.org/Products/Zope/2.9.0 Or do you mean 2.9.1? What can I do to help with that? Provoke Sidnei into responding :-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Python warnings behavior and stacklevel=2
[Julien Anguenot] I'm having some problems with the warnings module behavior. (Python-2.4.2 and Zope-2.9 trunk) [... traceback ... ] - Line 71 Module zLOG, line 140, in LOG Module warnings, line 61, in warn Module warnings, line 67, in warn_explicit TypeError: unsubscriptable object It seems to be referenced on the Python tracker since Python-2.3.3. Has been fixed and closed but has been updated in January this year. https://sourceforge.net/tracker/?func=detailatid=105470aid=890010group_id=5470 I expect that referencing that bug report is just misleading here: none of the bad behaviors listed in that bug report occur under Python 2.4.2 (I just tried all of 'em). Specifying a stacklevel of a workaround, instead of 2 within the zLOG/__init__.py for instance1, as works fine. (and this seems to appear within the Python but report) None of the provoking code in the bug report used stacklevel. There's a line of _output_ in the bug report, from a pdb session, where pdb showed the first line of the warnings.warn() function, showing that `stacklevel` is a formal argument of `warn()`, and that it defaults to 1: (Pdb) s --Call-- /usr/lib/python2.3/warnings.py(24)warn() - def warn(message, category=None, stacklevel=1): # this is pdb output, not input There's no other mention of `stacklevel` in the report. I actually get the same error and behavior within CPS code using the warnings module with a stacklevel of 2. Has someone a proper way to fix this from Zope and / or Python or can we simply change the StackLevel of the deprecation warnings to 1 waiting for a proper fix in Python ? All the symptoms in the bug report are already fixed. In the absence of a new bug report, nothing else _will_ be fixed in Python related to this. The _cause_ of those bugs in the first place was an internal Python error: one of the internal functions didn't propagate exceptions properly back to the eval loop. It's possible that other cases like that exist, in Python itself or in a C extension module (it's actually a pretty common error in extension modules). Progress requires a small test case demonstrating the problem; the bug report contained several small test cases illustrating symtpoms, but all of those have been repaired, so if there's another bug it requires another test case to track it down. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] December release post-mortem
... [Stephan Ricther] I have seen you take a similar approach to zope.testing and I found that painful just by watching the checkins. [Jim Fulton] I don't understand what you mean. Having a separate zope.testing project has been extremely useful. For example, in our comercial apps, we often used newer versions than existed in Zope 3. We often needed enhacements to zope.testing at times that Zope 3 was feature frozen. We could have made a Zope 3 branch just to modify zope.testing, but that would have been a hassle for us and for Zope 3 developers. Note that the new test runner (from zope.testing) was used in ZODB long before it was used in Zope 3. I want to note that this was good for Zope3, too: as a willing early adopter of zope.testing, ZODB hit bugs and platform-dependent glitches first, and got them fixed before the larger Zope3 development community could be bit by them. Also want to note that ZRS needed to add new features to zope.testing, and ZRS doesn't include _any_ Zope code (ZRS builds on ZEO, not Zope). Having zope.testing be its own project without all the adminstrative overheads of having its own official releases made it very easy to add new code for ZRS's benefit without disturbing any of zope.testing's other users. In all, zope.testing is a poster child for the value of package development outside of a Zope tree. It's true that, to make changes in zope.testing, I needed to have a distinct checkout of zope.testing. I didn't feel that to be a burden, though. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope3-dev] December release post-mortem
We should distinguish between authoring the Windows
build-the-installer code, and running that code. Making a Zope 2
Windows release consists of _running_ the build-the-installer code,
and is easy. It's actually easier than building a Zope 3 Windows
release: once the Python tarball, Zope 2 tarball, and pywin32
installer are downloaded, building a Zope 2 release consists of typing
one command in a Cygwin shell:
WinBuilders/buildout zope
That's it.
_Authoring_ the Zope 2 build-the-installer code has been a royal PITA,
but work is needed there only when packaging gimmicks change. Like
moving to zpkgtools, or moving away from them again, or moving to a
different packaging system for one of the components (like Python
moving from a Wise installer to an MSI installer for 2.4, or pywin32
earlier moving from a Wise installer to a distutils installer).
Across the long life of Zope 2.7, those things changed very little,
and as a result very little needed to change in the
build-the-installer code across 2.7's life. 2.7 Windows releases
remained easy to build across that time (granting that Mark Hammond
did the work to adjust to pywin32's packaging changes).
2.8 Windows releases were/are similarly easy to build.
A lot of new work was needed for 2.9, due to Python and zpkgtools
changes. I also took the opportunity to upgrade from InnoSetup 4 to
version 5, which introduced a much easier scheme for managing the
custom dialog screens in the Zope installer.
All of that ended up making the build-the-installer code substantially
simpler, to the point where it now seems kinda goofy to use makefiles
at all in the process. Makefile mazes are inscrutable, and are
particularly ill-suited to the true nature of this task: unpack
various tarballs, and rearrange them like so.
Finishing a Windows release of Zope 2 or Zope 3 takes much longer than
just minutes, unless you skip testing. If you do skip testing, then
it's minutes for both. Testing Zope as a Windows service is harder
for Zope 3, because the Zope 3 installer doesn't (but the Zope 2
installer does) install or start the service, or auto-stop and remove
the service when Zope is uninstalled. BTW, Zope3 _could_ do that too
with a suitable post-install/pre-uninstall script, which is something
InnoSetup automates for Zope2.
It can take a Windows expert some days to author Zope2
build-the-installer changes when packaging fashions change (whether
Zope's or the repackaged components'). InnoSetup is the least of the
problems there, and while MSI is the coming fashion, slinging MSI code
appears to require much more knowledge than slinging InnoSetup
declarations.
The idea that it takes Windows expertise to write an InnoSetup
declaration is wrong: such lines merely say take the files you find
_here_ now, and put them _there_ when the installer runs. For
example, this is the entire Inno code needed to install Zope 2's lib/
directory:
Source:lib\*.*; DestDir: {app}\lib; Flags: ignoreversion recursesubdirs
That kind of stuff is truly trivial. It does take Inno expertise to
create and manage the installer's dialog screens (manage == e.g., if
the user chooses not to have the installer create an instance home,
then the dialog asking for the instance home's directory shouldn't be
shown), but those typically don't change across releases.
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Removing 'inst'
[Philipp von Weitershausen] ... Therefore, just to reduce confusion, I would move Makefile.in and configure.py to the root (and remove the decoys). I'd also suggest we rename 'inst' to 'installer' so that it won't be confused with instance. Then again, this is just me and my weird sense of aesthetics ;). That all sounds good to me, assuming people still find a Makefile to be useful (I've been on Windows for a lng time now ;-)). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Seeking brave souls to try Zope 2.9 Windows installer
[Lennart Regebro] Only one real bug: No user is created (even though you type in name and password). Sorry, I'm not following this. The installer never offers to create a user (although it does ask you to supply a password for the fixed admin user). So you must be talking about something else, but I don't know what. For example, when logged in to the installed Zope as admin, I had no problems creating new users from the acl_users thingie. And of course zopectl doesn't work, so zopectl adduser is out of the question. Luckily creating an inituser still works! :-) Also, installing the shell files runzope and zopectl is kinda pointless. I'm sure the Zope 2.8 (etc) Windows installers did the same here. This is controlled by the content of the top-level (in a Zope checkout) skel/ directory: C:\Code\Zope2.9dir/b skel\bin runzope.bat.in runzope.in zopectl.in zopeservice.py.in So (exactly) those 4 things (without the .in suffix) get created in an instance home's bin/ directory, regardless of platform. Someone may wish to change Zope's utilities/mkzopeinstance.py to do different things on different platforms, but that's out of scope for this little project (which makes changes only under Zope's inst/WinBuilders/: the Windows installer-builder code). Minor, I know, but it means you have to typetab twice when tab expanding to run runzope in from a commandline. ;-) ? Assuming you're running a native Windows NT+ shell (cmd.exe), you set tab as your file completion character, and you're using NTFS as the filesystem (so that tab completion suggests possibilities in alphabetical order), then the sequence r TAB in an instance's bin directory completes to runzope and that's what you want. That doesn't actually run the file named runzope, it actually runs the file named runzope.bat on Windows. I'm starting to suspect you don't normally use Windows, Lennart ;-) I certainly agree there's no _reason_ to create the runzope file to begin with on Windows (except perhaps to keep mkzopeinstance.py simple), but it doesn't actually get in the way of using tab completion. When you run the instance creation you get a commandline, but you have no idea where the current directory is, so if you just type in a relative path, you don't know where the directory was created. (Turns out it's in C:\Documents And Settings\username\ ) I'm sure all earlier Zope Windows installers created stuff that did the same here too. If that isn't wanted, then Zope's utilities/mkzopeinstance.py is again the thing that would need to be changed. That's it so far! Great job! Thank you for trying it! I'm just relieved it didn't melt your hard drive ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Seeking brave souls to try Zope 2.9 Windows installer
[Sidnei da Silva] Way to go Tim! You beat me to it. I was supposed to look at this soon but got back from vacation just this tuesday. I will make sure your installer gets testing and try to fix any eventual issues. Excellent! This may actually work ;-) While I'll be on vacation the next two weeks, I'll check email each day, and will be happy to give minor wink help with mysteries. The build-the-installer code got substantially simpler, but I think it's hard to jump in cold and understand it (in part because the makefile-maze infrastructure is better suited to more complicated jobs than this one has become). WinBuilders/README.txt is the best place to start, and I tried to leave it telling only truths on the branch. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Removing 'inst'
[Philipp von Weitershausen] It seems that 'inst' holds nothing of value anymore except 'Makefile.in' and 'WinBuilders'. WRT Windows, that's certainly true on my Windows-installer branch. I don't know whether any of it is still useful on Linux. You seem to think Makefile.in is still useful, but if that's true then I expect inst/configure.py is also still useful (it looks like configure.py is the intended way to create an actual makefile from the Makefile.in template). One thing for sure is that it will be helpful to get rid of as many decoys as possible; e.g., I burned several hours staring at the stuff in inst/ wondering how to make it work again, then digging in to why it existed at all, and finally concluding that everything it ever did is of no use on Windows anymore ;-). I propose moving those two items to the root and remove 'inst'. I'd rather just remove the decoys. The process of building a Windows installer needs/creates three not-checked-in directories that are siblings of WinBuilders, and it's nicer to have those hiding under inst/ than cluttering the root of a checkout. The Windows stuff will have no use for anything other than WinBuilders/, so if Makefile.in's Linux purpose would be better served by moving that elsewhere, that would be fine. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Removing 'inst'
[Sidnei da Silva] I think there's a Makefile.win too, that is used by inst/configure.py on Windows. (I know because use it). There is a Makefile.win.in, but the build-the-Windows-installer process no longer uses it on my branch -- the branch doesn't use anything in the root of inst/ anymore, except for the inst/WinBuilders/ subdirectory. (Note that I haven't removed inst/Makefile.win.in on the branch, because I promised to change only stuff under inst/WinBuilders/ on that branch.) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Seeking brave souls to try Zope 2.9 Windows installer
Most of the last two days I've been whacking on Zope 2.9 to build a Zope 2 style Windows installer that bundles Python 2.4.2, and cooperates with the new zpkgtools-created tarball layout. The build-the-Windows-installer code got simpler as a result, but ... This work is being done on branch: //svn.zope.org/repos/main/Zope/branches/tim-2.9-windows-installer which was branched off the 2.9 tag. All changes on the branch are under inst/WinBuilders/, so the idea is that when this works, it can be merged into the 2.9 tag as well as the trunk (the stuff under WinBuilders/ isn't used for anything except building a Windows installer, and on the 2.9 tag the WinBuilders/ part is wholly broken). It's in a state now where it's not obviously broken to my eyes, but I have little experience running Zope2 so my eyes aren't reliable here. People on Windows who want to try it, knowing in advance that it may be broken in various ways, can download an experimental installer: Zope-Experimental-2.9.0-win32.exe from my member page: http://www.zope.org/Members/tim_one Please report problems to zope-dev, not to me directly. After tomorrow, I'll be on vacation for 2 weeks, and won't be terribly responsive :-) Note that I'm copying Sidnei in the hope that he can be persuaded to take up the slack. If there are problems, they're likely to be in the layout of the code: missing pieces, wrong directories, stuff like that. The 2.9 tarball's install.py installed everything it was told to install, and if there are problems there they're in Zope 2.9, not in the chain of code building the Windows installer. Nevertheless, the Windows installer could be taught to patch over them. And, of course, the build-the-installer dance may introduce new errors of its own. I hope it works for someone ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [ZODB-Dev] Re: [Zope-dev] Re: zLOG module deprecated
[Andreas Jung] ... Obviously ZEO (using TRACE) runs on Zope 3 without zLOG so specific extension can be handled locally. ZEO also runs on Zopes 2.8 and 2.9 without zLOG -- zLOG hasn't been used in ZODB since 3.2 (ZODBs 3.3, 3.4, 3.5, 3.6, and current trunk contain no references to zLOG). If Zope folk generally want more than the 5 logging levels that come built in to Python's logging module, that's easy to supply, but then it's also important that they define them with the same names and numeric values. ZODB's loglevels.py does so, and is the only extension to Python's logging module ZODB made. This is the code in its entirety (but skipping the copyright boilerplate at the top -- Copyright (c) 2004 Zope Corporation and Contributors): import logging __all__ = [BLATHER, TRACE] # In the days of zLOG, there were 7 standard log levels, and ZODB/ZEO used # all of them. Here's how they map to the logging package's 5 standard # levels: # #zLOG logging #---- #PANIC (300) FATAL, CRITICAL (50) #ERROR (200) ERROR (40) #WARNING, PROBLEM (100) WARN (30) #INFO (0) INFO (20) #BLATHER (-100) none -- defined here as BLATHER (15) #DEBUG (-200) DEBUG (10) #TRACE (-300) none -- defined here as TRACE (5) # # TRACE is used by ZEO for extremely verbose trace output, enabled only # when chasing bottom-level communications bugs. It really should be at # a lower level than DEBUG. # # BLATHER is a harder call, and various instances could probably be folded # into INFO or DEBUG without real harm. BLATHER = 15 TRACE = 5 logging.addLevelName(BLATHER, BLATHER) logging.addLevelName(TRACE, TRACE) When ZODB wants to use BLATHER or TRACE, it imports the symbolic name from ZODB.loglevels instead of from logging. That's all there is to it -- it's trivial. If Zope decided to keep BLATHER and TRACE too, then a future ZODB release would presumably change to import BLATHER and TRACE from the same place Zope gets them. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Twice-annual releases and bugfixes?
[Jeff Kowalczyk] What does the twice-annual release policy say about bugs and/or packaging errors that are identified and fixed within a very short time of the official release announcement? I think the answer you're looking for is that the policy says nothing about that. Every-6-months applies to the 'j' position in i.j.k (like moving from Zope 2.9 to Zope 2.10). Those can introduce new features. Changes only in the 'k' position are bugfix-only releases, and in theory there could another one of those every day ;-). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Re: zLOG module deprecated
[Andreas Jung] ZODB defines these levels but I can not see any code in the ZODB package that actually uses these levels. Nevertheless, grep'ing the ZODB source for TRACE and BLATHER will find them. TRACE is used only in modules under ZEO/zrpc/, and gives extremely verbose output about bare-metal ZEO communications. Only someone trying to debug low-level ZEO socket or protocol problems would want to use TRACE. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zLOG module deprecated
[Fred Drake] Nobody should be using the zLOG levels with the logging package, but rather use the logging package levels. So in the end, there's no need for Zope to be defining levels at all, only conventions for how the levels are used. The logging package supports defining as many additional named levels as you feel like adding, and ZODB added BLATHER and TRACE zLOG-workalike levels at the time ZODB was converted to use the logging package. People may want to restrict themselves to using logging's built-in levels, but logging was designed to be extended in this way, so there's no technical barrier here. In particular, nobody would want to get ZEO's voluminous but micro-purpose TRACE output at debug level. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: SVN: Zope/branches/2.9/ Move to ZODB 3.6 final.
[Florent Guillaume] Why not use the tag you just made? ZODB svn://svn.zope.org/repos/main/ZODB/tags/3.6.0/src/ZODB etc. Ask Jim ;-) (Hi have a vague feeling this may have been discussed before but I'm not sure ;)). I'm not sure it's been coherently discussed on a mailing list: - While a tag is conceptually read-only, sometimes people do make changes on tags. You can't cheat (or get screwed by someone else cheating) with -r N, though, - Updates to a different revision on the same SVN path typically go much faster, replacing only the files that have changed. When the SVN path changes (as was true when stitching in ZODB by tag), the entire project is refetched. When moving from ZODB 3.6.0b6 to 3.6.0 final, that's the difference between getting all the ZODB code again, or just getting the 3 (or so) files that changed (more files than that _did_ change in ZODB 3.6 final, but Zope doesn't include them -- the others are part of the standalone ZODB release). - When the SVN path changes, people who leave old .pyc (or other not-checked-in) files behind end up with OLD.n directories created by SVN too. Overall, I like tags better myself: - They're self-documenting. - I never leave old .pyc (etc) files around, always recompile everything, and do something else while `svn up` is running, so in all I just don't notice any of the bad aspects of using tags. - It's less error-prone to edit svn:externals to change a character or two in a mnemonically-named tag path than to replace an arbitrary-looking revision number. For example, I stitched ZODB 3.6 into 4 Zopes yesterday, doing propedit on 8 directories total. I screwed it up at first but didn't notice until after running tests, and then had to repair it and run them all over again: it turned out that the older -r 41065 wasn't unique to ZODB in all of them. s/41065/41153/g was semantically wrong, unintentionally updating some _non_-ZODB externals to rev 41553 too. If we had stuck to using tags, s/3.6.0b6/3.6.0/g would been right the first time around. So there are tradeoffs, and people won't agree on which is best overall. Jim has a strong preference for -r N, and much stronger than my will to argue about it ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: SVN: Zope/branches/2.9/ Move to ZODB 3.6 final.
[Andreas Jung] I _like_ tags in externals because they are self-documenting. Dealing with externals where you have several different revision numbers is a PIA since you never know what version of a module is behind the revision. You always have look through the log for the referenced module...that's very time consuming. I see this as a major risk factor when working on releases. If you want the tag style for ZODB in Zope 2.9 and Zope trunk, I doubt anyone would yell at you if you changed them to use the ZODB 3.6 tag instead (pieces of ZODB are stitched in under lib/python/, doc/, and utilities/ in Zope 2, so that requires propedit'ing 3 directories). Because ZODB has its own advertised version number (ZODB.__version__, ZEO.version, and src/ZEO/version.txt), I always make a new tag for a new ZODB release (whether internal or external). ZODB isn't like, e.g., zope.testing in that way (and I understand why Jim would think it a pain to make a new tag of zope.testing for every little change there). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Re: Connection pool makes no sense
[EMAIL PROTECTED] Not agree. Can you answer the question? Does self.all.remove(c) mean that we WANT to destroy connection instance? [Tim Peters] It means that _ConnectionPool no longer has a reason to remember anything about that Connection. Application code can continue keeping it alive forever, though. [Denis Markov] But what about RDB-Connection what stay in cache forever? Sorry, I don't know anything about how your app uses RDB connections. ZODB isn't creating them on its own ;-) On the next peak load we get some next ZODB-Connections with RDB-Connection After repush() old ZODB-Connections will be killed (if pool_size) I don't like the word killed here, because it seems highly misleading. ZODB doesn't destroy any Connections or any caches. ZODB destroys all its strong references to old Connections, and that's all. Nothing can be done to _force_ Connections to go away forever. It's ZODB's job here to make sure it isn't forcing Connections (beyond the pool_size limit) to stay alive, and it's doing that job. It can't kill Connections. but RDB-Connection stay in cache forever And so on There's one cache per Connection. If and when a Connection goes away, its cache goes away too. So when you say something stays in cache forever, I don't know what you mean -- you apparently have many (hundreds? thousands?) of Connections, in which case you also have many (hundreds or thousands) of caches. I don't know how an RDB-Connection gets into even one of those caches to begin with. as a result we get many RDB-Connections what will never use but hang our RDB At this point I have to hope that someone else here understands what you're doing. If not, you may have better luck on the zope-db list (which is devoted to using other databases with Zope): http://mail.zope.org/mailman/listinfo/zope-db ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RE: [ZODB-Dev] Re: Connection pool makes no sense
Oops! I sent this to zope-dev instead of zodb-dev by mistake. Nevertheless, if you can help Denis Markov, don't let my mistake inhibit you ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildbot failure in Zope branches 2.9 2.4 Windows 2000 zc-bbwin
[Florent Guillaume]
FYI test output is:
...
'zope[.]app[.])' is not recognized as an internal or external command,
operable program or batch file.
program finished with exit code 255
That was due to an ill-formed command line, which Jim repaired this
morning. Zope trunk and Zope 2.9 branch are still failing on Windows,
though, with different symptoms now:
Failure in test test_checkPermission_proxy_role_scope
(AccessControl.tests.testZopeSecurityPolicy.C_ZSPTests)
Traceback (most recent call last):
File c:\python24\lib\unittest.py, line 260, in run
testMethod()
File C:\buildbot\.org\zc-bbwin--Windows
2000--Zope---trunk--2.4\build\lib\python\AccessControl\tests\testZopeSecurityPolicy.py,
line 326, in test_checkPermission_proxy_role_scope
self.failUnless(self.policy.checkPermission('Kill', r_subitem, context))
File c:\python24\lib\unittest.py, line 309, in failUnless
if not expr: raise self.failureException, msg
AssertionError
.
Failure in test test_checkPermission_proxy_roles_limit_access
(AccessControl.tests.testZopeSecurityPolicy.C_ZSPTests)
Traceback (most recent call last):
File c:\python24\lib\unittest.py, line 260, in run
testMethod()
File C:\buildbot\.org\zc-bbwin--Windows
2000--Zope---trunk--2.4\build\lib\python\AccessControl\tests\testZopeSecurityPolicy.py,
line 302, in test_checkPermission_proxy_roles_limit_access
self.failIf(self.policy.checkPermission('Foo', r_item, context))
File c:\python24\lib\unittest.py, line 305, in failIf
if expr: raise self.failureException, msg
AssertionError
.
Failure in test test_checkPermission_respects_proxy_roles
(AccessControl.tests.testZopeSecurityPolicy.C_ZSPTests)
Traceback (most recent call last):
File c:\python24\lib\unittest.py, line 260, in run
testMethod()
File C:\buildbot\.org\zc-bbwin--Windows
2000--Zope---trunk--2.4\build\lib\python\AccessControl\tests\testZopeSecurityPolicy.py,
line 291, in test_checkPermission_respects_proxy_roles
self.failUnless(self.policy.checkPermission('View', r_item, context))
File c:\python24\lib\unittest.py, line 309, in failUnless
if not expr: raise self.failureException, msg
AssertionError
I don't have time to look at it. Since the Windows buildbot slave
`bbwin` doesn't have a C compiler, I suppose it's possible that the
old, canned .pyd files it uses are out of synch with the current C
code.
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildbot failure in Zope branches 2.9 2.4 Windows 2000 zc-bbwin
[Tim Peters]
...
Failure in test test_checkPermission_proxy_roles_limit_access
(AccessControl.tests.testZopeSecurityPolicy.C_ZSPTests)
Traceback (most recent call last):
File c:\python24\lib\unittest.py, line 260, in run
testMethod()
File
C:\buildbot\.org\zc-bbwin--Windows2000--Zope---trunk--2.4\build\lib\python\AccessControl\tests\testZopeSecurityPolicy.py,
line 302, in test_checkPermission_proxy_roles_limit_access
self.failIf(self.policy.checkPermission('Foo', r_item, context))
File c:\python24\lib\unittest.py, line 305, in failIf
if expr: raise self.failureException, msg
AssertionError
...
I don't have time to look at it. Since the Windows buildbot slave
`bbwin` doesn't have a C compiler, I suppose it's possible that the
old, canned .pyd files it uses are out of synch with the current C
code.
That was the problem. Turns out `bbwin` does have a compiler, but the
buildbot recipe didn't use it. After Jim fixed that, we have another
Windows-specific failure, in new code from ChrisM (this is on Zope
trunk, of course):
Failure in test test_get_env (ZServer.tests.test_clockserver.ClockServerTests)
Traceback (most recent call last):
File c:\python24\lib\unittest.py, line 260, in run
testMethod()
File C:\buildbot\.org\zc-bbwin--Windows
2000--Zope---trunk--2.4\build\lib\python\ZServer\tests\test_clockserver.py,
line 87, in test_get_env
self.assertEqual(env['PATH_TRANSLATED'], '/a /b')
File c:\python24\lib\unittest.py, line 333, in failUnlessEqual
raise self.failureException, \
AssertionError: '\\a \\b' != '/a /b'
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: buildbot failure in Zope branches 2.9 2.4 Windows 2000 zc-bbwin
[Tim Peters] ... That was the problem. Turns out `bbwin` does have a compiler, but the buildbot recipe didn't use it. After Jim fixed that, we have another Windows-specific failure, in new code from ChrisM (this is on Zope trunk, of course): Failure in test test_get_env (ZServer.tests.test_clockserver.ClockServerTests) Traceback (most recent call last): File c:\python24\lib\unittest.py, line 260, in run testMethod() File C:\buildbot\.org\zc-bbwin--Windows 2000--Zope---trunk--2.4\build\lib\python\ZServer\tests \test_clockserver.py, line 87, in test_get_env self.assertEqual(env['PATH_TRANSLATED'], '/a /b') File c:\python24\lib\unittest.py, line 333, in failUnlessEqual raise self.failureException, \ AssertionError: '\\a \\b' != '/a /b' [Chris McDonough] Hmmm... I *think* I just fixed this. Luckily for us, the buildbot doesn't give a rip what either of us think. Its judgment is that you _did_ fix it, and there's not a court in the land that will convict you given that perfect defense. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Unit Test Failures
[Benji York] The Zope2 unit tests have been failing for some time on buildbot.zope.com. Looks like a Five-related problem: http://buildbot.zope.org/Zope%20trunk%202.4%20Linux%20zc-buildbot/builds/109/test/0 [Tres Seaver] The failing test is a functional test, not a unit test; I don't run them by default when I check in. I see a couple of problem points: the biggest is the use in the output of 'checkSiteManager.html' of a repr'ed dict, which might render differently between different machines (or even the same machine across Python versions). This is a classic case where the doctest does *not* provide clarity over a traditional 'unittest' style test, IMNSHO. Possibly, but that clearly doesn't apply to the specific failures we're seeing (True versus False outcomes, and getting lots more output than expected). I'm not sure what it is testing, either; CC'ing Phillip, whose fingerprints are on it, according the 'svn blame', for clarification. These tests have always failed, and Phillip doesn't know why. Because they were failing, he changed them to run at level 2. That's not what level 2 is for, though, and the failures became visible to everyone when the buildbot was changed to pass `--all` to test.py (--all runs tests at all levels, level 2 among them ;-)). I opened a Collector issue on this about a month ago (I always run with `--all`, so these failures are old news to me): http://www.zope.org/Collectors/Zope/1947 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Unit Test Failures
[Tim] I opened a Collector issue on this about a month ago (I always run with `--all`, so these failures are old news to me): http://www.zope.org/Collectors/Zope/1947 [Philipp] Rereading that issue again, it's totally surprising to see that there's no failure on Windows, which makes its source even harder to debug. No, these tests fail on Windows too. In fact, the problem was orginally reported against Windows: see entry #2 in: http://www.zope.org/Collectors/Zope/1931 What does not fail on Windows is the specific variation you asked me to try in your comment #2 on issue 1947: When you run it by itself, it passes, though. Maybe you can confirm that. In my comment #3 on issue 1947, I confirmed that: when I ran the functional.txt test _by itself_ on Windows, it passed. I wouldn't know where to start (having tried to debug this problem in the past). Anyone got an idea? For a start, disabuse yourself of the illusion that it acts differently on Windows than on Linux ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Unit Test Failures
...
[Philipp]
Well, if you look closer you find that it uses pprint.pformat which
always outputs the
same on all machines (because it provides output sorted by the
dictionary key).
[Tres Seaver]
I see that in the implementation; it isn't documented as part of
pprint's contract, however.
It's not part of the implementation either. For example,
d = {z: 1, m: 2}
pprint.pprint(d)
{'z': 1, 'm': 2}
That is, it's not true that pprint always sorts a dict for display.
Looks like Jim's suggested
from zope.testing.doctestunit import pprint
inherits this insecurity.
In the example above, two things conspire to give unsorted output:
1. pprint(dict) doesn't try to sort at all if _repr(dict) is short.
2. On my 32-bit box, hash('z') hash('m'), which leads to platform accidents
in dict insertion putting 'z' before 'm' in the natural dict
iteration order:
d
{'z': 1, 'm': 2}
Because of #1, pprint passes on that order.
On a 64-bit box, the order may differ (or across Python versions on
one box if the string hash, or dict insertion, algorithms change).
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Unit Test Failures
[Tim]
Looks like Jim's suggested
from zope.testing.doctestunit import pprint
inherits this insecurity.
[Jim]
No, it doesn't.
from zope.testing.doctestunit import pprint
pprint({z: 1, m: 2})
{'m': 2,
'z': 1}
Note both the sorting and the wrapping.
See below.
Cool! I confess that the 'width' fiddling in:
def pprint(ob, **opts):
if 'width' not in opts:
opts['width'] = 1
return PrettyPrinter(**opts).pprint(ob)
was incomprehensible to me just now -- you might want to add a comment
to that ;-)
zope.doctestunit.pprint creates and uses a pretty printer with width
set to 1, so all dicts are long and thus sorted.
Well, I understand why that works, but it's not part of pprint's
contract either.
Note that, in Python 2.4, you can now pass a width to pprint without creating
a separate pretty printer:
from pprint import pprint
pprint({z: 1, m: 2})
{'z': 1, 'm': 2}
pprint({z: 1, m: 2}, width=1)
{'m': 2,
'z': 1}
So maybe we can phase out the use of docutestunit's pprint.)
Perhaps we should push to get the sorting behavior of pprint
documented to allay your concerns.
I think it's harder than you yet realize: dicts don't require that
keys be orderable, so pprint can't promise to sort dict displays.
As-is, pprint simply blows up if, e.g., you pass it a big dict using
complex numbers as keys. Could be nobody has noticed that yet, but
since the trend in Python is to raise exceptions for non-equality
comparisons of objects of different types, this kind of failure is
bound to become more common. In the end, I wouldn't be surprised if
pprint(dict) got changed to sort on the repr of the keys instead of on
the keys themselves.
Better yet, perhaps we can get all dicts to be sorted.
As above.
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Unit Test Failures
... [Tim] Well, I understand why that works, but it's not part of pprint's contract either. [Jim] What contract. :) pprint's docs. Aren't you always telling me to read the source? Indeed, if you hadn't, you wouldn't have known that forcing width=1 forces dict sorting ;-) It's common as mud to close a bug report in Pythonland too when it's just griping about changes in undocumented behavior. ... I think it's harder than you yet realize: dicts don't require that keys be orderable, so pprint can't promise to sort dict displays. As-is, pprint simply blows up if, e.g., you pass it a big dict using complex numbers as keys. Could be nobody has noticed that yet, but since the trend in Python is to raise exceptions for non-equality comparisons of objects of different types, this kind of failure is bound to become more common. That's fine. In practice, when we do this sort of thing, the keys are almost always strings. My point is that you're relying on undocumented behavior now, and so there's no guarantee it will continue to behave that way. To the contrary, because sorting dicts _started_ creating problems when Python gave up its original any pair of objects can be compared behavior, and Python continues moving away from that, there's reason to worry that pprint may give up sorting dict displays entirely. Sorting isn't free, is currently a source of bugs in pprint, and adds a large memory burden (to materialize a giant dict.items() list) when pprint'ing a large dict. An obvious alternative is for pprint to change to do dict.iteritems() when building a display, which is much better on all those counts. If I have anything to say about it, it will continue to sort, but nobody listens to me anymore ;-) In the end, I wouldn't be surprised if pprint(dict) got changed to sort on the repr of the keys instead of on the keys themselves. That wouldn't be bad. It's debatable for sure. Anyway, I guess we should make an issue of this on python-dev, so that either we can count on documented behavior going forward or so that we write our pwn pretty printer for use in doctest. I'll try to make time for this next week (I'm on vacation then). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: svn.zope.org borked
[Jim] ... The whole repository is only about 800 megs. There are over 8 gigs free. Are the dump file or the file-based repo much larger in size the the Berkeley database? FYI, if you don't want to read the code ;-), the book says an FSFS repository is slightly smaller than the same thing under BDB: http://svnbook.red-bean.com/en/1.1/ch05.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope-2.9 r40780 make install doesn't finish, files missing from bin
[Stefan H. Holek] make install does currently not work on 2.9 branch and trunk. I am told that this is because zpkg cannot do it. I am also told that the tarball would support make install, just not the checkout. I never use tarballs, so I don't know for sure. There's no longer any necessary relationship between the shape or contents of a checkout tree and the shape or contents of a distribution tree (tarball). How the two relate now is defined by input to zpkgtools, and it's possible to create multiple kinds of distributions from the same checkout tree now (by giving zpkgtools different input). Plain setup.py build no longer necessarily works from a checkout tree either, BTW. This is all in line with zpkgtools's goals: http://www.zope.org/DevHome/Wikis/DevSite/Projects/ComponentArchitecture/Zope3PackagingProposal I guess that the process for making a Zope3 release applies to Zope 2.9 now too: http://www.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/MakingARelease ZODB's release process also had to change similarly when it moved to zpkgtools. I'd very much like to see the canonical ./configure; make; make install continue to work as it did in 2.7 and 2.8. Sysadmins go crazy if Zope's installation procedure changes with every other release. Nobody should be installing from a checkout to begin with, right? The comfortable old dance should continue to work from a distribution. checkout != distribution. Arguments about zpkgtools design probably need to involve Jim. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Test Failures
[Sidnei da Silva] I've seen the following tests fail today, after updating Zope 2.8 branch for all variants of BTrees: == ERROR: testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IIBucketTest) -- Traceback (most recent call last): File /usr/lib/python2.4/unittest.py, line 260, in run testMethod() File /home/sidnei/src/zope/2.8/lib/python/BTrees/tests/testBTrees.py, line 353, in testUpdateFromPersistentMapping self.t.update(pm) TypeError: Sequence must contain 2-item tuples This is on a Powerbook running Ubuntu Breezy PPC. Python 2.4.2 (#2, Nov 20 2005, 17:20:59) [GCC 4.0.3 20051023 (prerelease) (Ubuntu 4.0.2-3ubuntu1)] on linux2 Type help, copyright, credits or license for more information. Works for me: [EMAIL PROTECTED]:~/Zope2.8$ svn info Path: . URL: svn://svn.zope.org/repos/main/Zope/branches/Zope-2_8-branch Repository UUID: 62d5b8a3-27da-0310-9561-8e5933582275 Revision: 40815 Node Kind: directory Schedule: normal Last Changed Author: sidnei Last Changed Rev: 40805 Last Changed Date: 2005-12-16 07:28:10 -0500 (Fri, 16 Dec 2005) Properties Last Updated: 2005-10-04 14:30:34 -0400 (Tue, 04 Oct 2005) [EMAIL PROTECTED]:~/Zope2.8$ svn up Fetching external item into 'doc/ZEO' External at revision 40815. Fetching external item into 'lib/python/zope' External at revision 40815. Fetching external item into 'lib/python/ZConfig' External at revision 40815. Fetching external item into 'lib/python/BTrees' External at revision 40815. Fetching external item into 'lib/python/Persistence' External at revision 40815. Fetching external item into 'lib/python/persistent' External at revision 40815. Fetching external item into 'lib/python/ThreadedAsync' External at revision 40815. Fetching external item into 'lib/python/transaction' External at revision 40815. Fetching external item into 'lib/python/ZEO' External at revision 40815. Fetching external item into 'lib/python/ZODB' External at revision 40815. Fetching external item into 'lib/python/ZopeUndo' External at revision 40815. Fetching external item into 'lib/python/zdaemon' External at revision 40815. Fetching external item into 'utilities/ZODBTools' External at revision 40815. At revision 40815. [EMAIL PROTECTED]:~/Zope2.8$ python2.4 Python 2.4.2 (#1, Dec 2 2005, 10:17:25) [GCC 4.0.2 20050808 (prerelease) (Ubuntu 4.0.1-4ubuntu9)] on linux2 ... [EMAIL PROTECTED]:~/Zope2.8$ python2.4 setup.py build_ext -i running build_ext running build_ext [EMAIL PROTECTED]:~/Zope2.8$ python2.4 test.py -vv . testUpdateFromPersistentMapping Running unit tests at level 1 Running unit tests from /home/tim/Zope2.8/lib/python testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IIBucketTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IIBTreeTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IFBucketTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IFBTreeTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IOBucketTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.IOBTreeTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.OOBucketTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.OOBTreeTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.OIBucketTest) ... ok testUpdateFromPersistentMapping (BTrees.tests.testBTrees.OIBTreeTest) ... ok -- Ran 10 tests in 0.007s OK No idea why it failed for you. The only thing that rings a bell here is that this test was added in ZODB 3.4.2b1, corresponding to this ZODB news entry: BTrees -- - (3.4.2b1) Collector 1873. It wasn't possible to construct a BTree or Bucket from, or apply their update() methods to, a PersistentMapping or PersistentDict. This works now. So my only guesses are that you have some older (than 3.4.2) version of ZODB on your PYTHONPATH, or that your checkout is screwed up. Here are the externals you _should_ have: [EMAIL PROTECTED]:~/Zope2.8$ svn propget svn:externals lib/python zope svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.1-Zope-2.8/src/zope ZConfigsvn://svn.zope.org/repos/main/ZConfig/tags/ZConfig-2.3 BTrees svn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/BTrees Persistencesvn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/Persistence persistent svn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/persistent ThreadedAsync svn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/ThreadedAsync transactionsvn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/transaction ZEOsvn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/ZEO ZODB svn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/ZODB ZopeUndo svn://svn.zope.org/repos/main/ZODB/tags/3.4.2/src/ZopeUndo zdaemon
[Zope-dev] RE: [ZODB-Dev] Test Failures
[Sidnei] I've seen the following tests fail today, after updating Zope 2.8 branch ... Python 2.4.2 (#2, Nov 20 2005, 17:20:59) ... BTW, I think the Official Story is that Python 2.4+ is still not supported for Zope 2.8. I ran all the stuff in my reply with 2.4.2 too. Doesn't matter, though -- I got the same results (no problems) with Python 2.3.5. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Test Failures
[Sidnei da Silva] But, why only the 2.8 tests would fail then? Hey, it's your machine, you figure it out ;-) Note that test.py in 2.8 has little in common with the test.py in 2.9 or Zope trunk, and they may very well react in different ways to quirks in your environment. I'll try a 'make clean' before running the tests and see if it helps. Probably not, but who knows. If that doesn't help, add code to dump sys.path and stare at it. And/or add code to import ZODB and print ZODB.__version__, to see which version you're really getting during the tests. Ah, since the fix for Collector 1873 required changing C code, I do expect those tests would fail if you didn't recompile Zope 2.8 since the time the C code changed. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Retiring ZODB 3.2 (was: Re: [Zope-dev] FTP Upload killing Zope)
[Andreas Jung, on zope-dev; Tim added zodb-dev to this msg] I would like to consider the 2.7 branch closed for any kind of fixes except security related fixes. I don't plan any further 2.7 releases. In that case (which is fine by me), I'll stop porting fixes to the ZODB 3.2 line too. No changes have been made to that since ZODB 3.2.10 was released on 12-Oct-2005, corresponding to the ZODB shipped with Zope 2.7.8. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [ZODB-Dev] Re: [Zope-dev] Re: sessions in the presence of conflicts
[Chris McDonough] Note that changing the transientobject conflict resolution algorithm won't get rid of all write conflict errors, because the BTree-based indexes in the transient object container will still conflict during a bucket split and other situations that I can't exactly recall (they're documented in the BTrees source code). A more readable account is here: http://www.zope.org/Wikis/ZODB/BTreeConflictResolution BTrees are mappings too, and looks like Dennis is trying to apply similar conflict-resolution rules to session mapping objects. Conflict resolution algorithms are difficult and any algorithm will have DWIM-y tradeoffs, so it's useful to keep it as simple as possible. Or no more complex as is actually helpful ;-) ... [Dennis Allison] I have yet to figure out how to map a TransientObject state back to the object it represents, but it clearly is possible. I didn't see a response to that bit yet, so: the state of an object P is whatever P.__getstate__() returns. Given such a return value `state`, and some object Q of the same type as P, Q.__setstate__(state) gives Q the same state P had. What state means is entirely up to the type's __setstate__() and __getstate__() implementations (if any). Objects deriving from Persistent inherit (by default) implementations that retrieve and update an instance's __dict__. BTrees.Length is a good example of a class that overrides these methods, using an integer as the state. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New ways of getting transactions?
A number of ZODB gimmicks were officially deprecated in ZODB 3.4, and removed in ZODB 3.6. Zope 2.9 uses the latter. From the ZODB NEWS file for 3.6: Removal of Features Deprecated in ZODB 3.4 -- (3.6b2) ZODB 3.6 no longer contains features officially deprecated in the ZODB 3.4 release. These include: - ``get_transaction()``. Use ``transaction.get()`` instead. ``transaction.commit()`` is a shortcut spelling of ``transaction.get().commit()``, and ``transaction.abort()`` of ``transaction.get().abort()``. Note that importing ZODB no longer installs ``get_transaction`` as a name in Python's ``__builtin__`` module either. - The ``begin()`` method of ``Transaction`` objects. Use the ``begin()`` method of a transaction manager instead. ``transaction.begin()`` is a shortcut spelling to call the default transaction manager's ``begin()`` method. - The ``dt`` argument to ``Connection.cacheMinimize()``. - The ``Connection.cacheFullSweep()`` method. Use ``cacheMinimize()`` instead. - The ``Connection.getTransaction()`` method. Pass a transaction manager to ``DB.open()`` instead. - The ``Connection.getLocalTransaction()`` method. Pass a transaction manager to ``DB.open()`` instead. - The ``cache_deactivate_after`` and ``version_cache_deactivate_after`` arguments to the ``DB`` constructor. - The ``temporary``, ``force``, and ``waitflag`` arguments to ``DB.open()``. ``DB.open()`` no longer blocks (there's no longer a fixed limit on the number of open connections). - The ``transaction`` and ``txn_mgr``arguments to ``DB.open()``. Use the ``transaction_manager`` argument instead. - The ``getCacheDeactivateAfter``, ``setCacheDeactivateAfter``, ``getVersionCacheDeactivateAfter`` and ``setVersionCacheDeactivateAfter`` methods of ``DB``. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] New ways of getting transactions?
[Jim Fulton] ... BTW, This is a good example for why I want to start using time-based deprecation. In a world with time-based releases, is there a difference? ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Windows Installer for Zope 2.9+
...
[Sidnei da Silva]
Simplifying a lot what the existing Zope 2 installer does, it
basically creates a 'software home', a default 'instance home' and
registers the services. All but the first part is done manually for
Zope 3, so I can see the lack of glow there.
There's also that Zope2 sticks Python inside of Zope, but Zope3 sticks
Zope inside of Python. A consequence is that you can't install more
than one instance of Zope3 under a single Windows account, but can
install any number of Zope2s. In the other direction, it's always a
puzzle for Zope2 Windows users to figure out how to give their Zope
access to packages installed in their (own, separate) Python. For
Zope3 that's a no-brainer.
Supposing there is a existing python installation, how difficult is it
to get a distutils-based windows installer to be extracted to a random
directory outside site-packages?
distutils has no support for that.
My guess is that it's basically unzip it.
What are you trying to accomplish? (I know you're trying to push code
into some directory other than under Lib/site-packages, but I don't
know which code or why.)
If that's the case, we can simplify the Zope 2 inno installer to:
1. include a distutils-based windows installer for the Zope 2 source
2. include some setup scripts
Then on installation
1. find or install according python
Note that Zope also needs pywin32 to be installed.
2. unpack the distutils-based windows installer into Program
Files\Zope
OK, so you're trying to preserve that ... what? You're neither
sticking Python inside Zope nor Zope inside Python?
BTW, _how_ do you unpack this at install time? You can't, for
example, assume that a Windows box has any unzip utility (let alone
some specific one). So this part would probably be simpler if you
point Inno at a Zope tree and let _it_ package it and unpack it.
Maybe the way to get such a tree is to build a distutils installer for
Zope and run it on your own box, pointing Inno at the tree it creates,
then throw the distutils installer away 0.3 wink.
3. create a default 'instance home', pointing to the installed python
An instance home _certainly_ doesn't belong inside the user's Python
-- but maybe I don't know what you mean by these words (I'm probably
misreading pointing to). It should be possible to create any number
of distinct instance homes.
4. register the services
Steps 2-4 are reasonably simple to me, the tricky one is 1.
Maybe somone will hit me for this ;-), but we have Inno code for #1 in
one of our other installers. It goes like this:
[Code]
var
// Path to the user's Python installation, found from the registry.
// Procedure SetPythonInstallPath deduces this.
gPythonInstallPath: String;
function InitializeSetup(): boolean;
begin
gPythonInstallPath := ''; // don't know yet
Result := True;
end;
...
// Look up Python's install path in the registry. Use HKCU first, because
// a user-only instance is supposed to take precedence. If no Python is
// installed, this will leave gPythonInstallPath as an empty string.
procedure SetPythonInstallPath();
begin
RegQueryStringValue(HKCU,
'Software\Python\PythonCore\2.4\InstallPath',
'',
gPythonInstallPath);
if gPythonInstallPath = '' then // not in HKCU, so try HKLM
RegQueryStringValue(HKLM,
'Software\Python\PythonCore\2.4\InstallPath',
'',
gPythonInstallPath);
end;
// Expose gPythonInstallPath to {code:...} clauses.
function GetPythonDir(Default: String): String;
begin
Result := gPythonInstallPath;
end;
...
procedure SetupDependencies();
var
PythonSetupFile: String;
PyWin32SetupFile: String;
...
begin
PythonSetupFile := 'python-2.4.2.msi';
PyWin32SetupFile := 'pywin32-205.win32-py2.4.exe';
...
// Setup python, unless it is already installed.
SetPythonInstallPath();
if gPythonInstallPath = '' then begin
if not ShellExec('open', ExpandConstant('{tmp}\files\' +
PythonSetupFile),
'', '', SW_SHOW, ewWaitUntilTerminated, ErrorCode) then
FatalError('Unable to install python');
// Try again to find the install path.
SetPythonInstallPath();
if gPythonInstallPath = '' then
// This shouldn't happen ... but if it does, we dare not leave this
// variable empty: the installer blithely goes on to copy files
// into the Windows system32 directory if we do. That should never
// be allowed.
gPythonInstallPath := ExpandConstant('{tmp}\NoPython\');
end;
// Setup pywin32
if not Exec(ExpandConstant('{tmp}\files\' + PyWin32SetupFile), '', '',
SW_SHOW, ewWaitUntilTerminated, ErrorCode) then
FatalError('Unable to install PythonWin32');
...
The key is that the Python installer creates a registry entry, so you
can guess whether Python is installed by looking at that.
Re: [Zope-dev] Logging changes in ZEO break zeoup.py
[Jens Vagelpohl] In Zope 2.7 I'm using zeoup.py to check on a ZEO server. This script can be run from anywhere as long as the PYTHONPATH is set correctly. For Zope 2.8.4, the ZEO logging has been switched to use the logging module. This leads to an error when running zeoup.py now: CRITICAL - zeoup status 1: No handlers could be found for logger ZEO.zrpc.Connection(C) ClientDisconnected The logging module is just trying to drive you mad by saying someone is _trying_ to log a CRITICAL problem, but I'm not going to show it to you. This is an informative msg, not an exception. It looks like you've got pieces of stdout and stderr scrambled together there, BTW. Can anyone advise me how to make this work under Zope 2.8.4? I expect zeoup.py needs to be changed. I did a similar thing for runzeo.py some time ago, and I bet the latter's setup_default_logging() method could be mostly reused. To get unstuck quickly, try adding just this: import logging logging.basicConfig() That always helps me in a pinch, but I never understood why (neither why logging insists that you call _something_ before it will stop annoying you, nor why the docs that seem to claim that logging will itself call basicConfig() (if you don't) appear to be telling fibs). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zLOG deprecation?
[Tim Peters] Also note that the code Florent pointed at is part of ZODB, not part of Zope. Zope shouldn't use it directly. Growing its own copy would be OK, or refactoring so that ZODB and Zope both get it from a shared new mini-project. [Chris Withers] Sorry, which code are we talking about here? Sorry, it wasn't Florent, it was Philipp; you replied to it: http://mail.zope.org/pipermail/zope-dev/2005-December/026039.html As he says there, the comment block was taken from ZODB/loglevels.py, which (besides just documenting this mess) implements TRACE and BLATHER levels for ZODB's use. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Windows Installer for Zope 2.9+
[Sidnei da Silva] Just noticed a checkin talking about the Windows Installer builder. I hope to find some time soon to take a look at that, since we now require Python 2.4 and Python 2.4 uses the 'Microsoft Installer'. I recall talking with Mark about it and he said it would take some time to fix the build process. I just asked Andreas (off-list) what his Windows plans were for 2.9 -- for various reasons I assumed someone else was already looking at this, but that assumption may be wrong (and given the lack of relevant checkins recently, almost certainly wrong frown). WRT Python 2.4, I never found a sane way to just _extract_ files from an .msi installer, so that part of the build process is dead meat now. In other bundle everything Windows installers at Zope Corp (such as for ZRS), I copy Python .pyd. .dll and .exe files from my own Python 2.4.2 installation instead. These binary files are all the build-Zope-installer process really needs from the Python Windows installer; the rest (like .py and .h files) can be taken from a Python tarball. You can avoid wrestling with .msi entirely this way. Possible headache: Python 2.4.2 requires msvcr71.dll, which is a Microsoft DLL (it's akin to msvcrt.dll for Visual Studio 6 -- it's the C runtime for VC 7.1), and one for which the redistribution conditions are unclear (at least to me). You can't assume that all Windows boxes already have this DLL. Another: I have no idea how the new zpkg-based build process will work with a Zope2-style installer. A Zope3-style installer is different in many ways (it's a plain distutils-based installer, and requires that the end user get and install Python pywin32 first). Plan on pain-time here. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Windows Installer for Zope 2.9+
... [Tim] | Possible headache: Python 2.4.2 requires msvcr71.dll, which is a | Microsoft DLL (it's akin to msvcrt.dll for Visual Studio 6 -- it's the | C runtime for VC 7.1), and one for which the redistribution conditions | are unclear (at least to me). You can't assume that all Windows boxes | already have this DLL. [Sidnei] Indeed. From reading around, seems like the saner thing is to make it a bold warning in the installer that the said dll is required instead of shipping it. If you go the Zope3-route, it becomes a non-issue: the Windows Python installer will install msvcr71.dll if needed. Redistribution there isn't a problem because the PSF builds the binaries using a duly licensed Microsoft compiler. It's much fuzzier for derivative works (do the PSF's redistribution rights pass through to them? ask two lawyers, get four answers). Zope Corp could presumably invoke the same rights because parts of Zope are compiled with a legitimately licensed VC 7.1 -- but that might depend on who does the compliing. | Another: I have no idea how the new zpkg-based build process will | work with a Zope2-style installer. A Zope3-style installer is | different in many ways (it's a plain distutils-based installer, and | requires that the end user get and install Python pywin32 first). | Plan on pain-time here. That's something I can play with :) Feedback from users hasn't been exactly glowing, but it's much easier to build an installer that way (no externals, no makefiles, no Cygwin involved, ...). Here's how it's done for Zope3; I don't know / can't guess what would need to change for Zope2: http://www.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/ZopeWindowsRelease ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: zLOG deprecation?
[Dennis Allison] We probably want an ALL level as well which would map to the NOTSET of the Python logging code and log everything. Why not call it NOTSET? Then you already have it ;-) Or forget it -- TRACE gets everything anyway. Florent, I don't see a TRACE level in this list. Did you think one was needed? See Florent's original message; Chris chopped TRACE away in his reply. Also note that the code Florent pointed at is part of ZODB, not part of Zope. Zope shouldn't use it directly. Growing its own copy would be OK, or refactoring so that ZODB and Zope both get it from a shared new mini-project. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] How bad _are_ ConflictErrors
[Dennis Allison] ... Conflict errors are not always errors. At the ZODB level, an unresolved conflict always raises an exception. Whether such an exception is considered to be an error isn't ZODB's decision -- that's up to the app. My understanding (which may be wrong) is that Zope tries up to 3 times to perform commit a given transaction, suppressing any conflict exceptions for the duration, before giving up. As I understand it, Zope retries when a conflict occurs and usually is able to commit both sides of the conflicting transaction. Right (although note that there may be more than two sides). Sometimes Zope cannot commit conflicting transactions--and it is at that point that an error occurs. Right, Zope eventually gives up on a transaction that keeps on raising conflict exceptions. There are supposed to be significant changes in the Zope 2.8.4/ZODB 3.4.2 system. There are. ZODB 3.3 introduced multiversion concurrency control (MVCC), which eliminates read conflicts in normal operation. Read-read conflicts no longer generate conflict errors Not really: under MVCC, there simply aren't any read conflicts. There may still be write conflicts. and the retry mechanism has been reworked at the ZODB level to retry once and then raise a POSKEY exception. Nope, no version of ZODB ever retries a transaction on its own. If an application (like Zope) wants to retry, it's entirely up to it do so. The optimistic locking used by Zope ZODB's transactional approach is optimistic, precisely because it _doesn't_ lock objects modified by a transaction. Any number of transactions are free to modify the same object at the same time -- no locking mechanism attempts to stop that. If multiple transactions do modify the same object at the same time, and that object doesn't implement conflict resolution, then only the first transaction to commit its changes to that object can succeed. can cause problems, particularly when the conflicting method changes external state. Yes -- but do note it's not a transactional system then (ZODB can roll back all changes _it_ makes, so that a failure to commit does no harm to the database state; external resources that can't take back provisional changes are indeed challenging to use in a transactional system). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Reminder: feature freeze November 1.
[Tres Seaver] test.py in the root is the likely culprit, as it is mucking with sys.path. Does this patch make the Windows tests pass? - --- test.py (revision 40087) +++ test.py (working copy) @@ -30,7 +30,7 @@ if shome: shome = os.path.abspath(shome) else: - -shome = os.path.join(zhome, 'lib/python') +shome = os.path.join(zhome, 'lib', 'python') elif shome: shome = os.path.abspath(shome) zhome = os.path.dirname(os.path.dirname(shome)) Well spotted! It (plus the later patch) does fix the checkDuplicate test failure on Windows, and I closed issue 1931. Turns out the Five tests that were failing on Windows also fail on Linux, but the failing tests don't run unless you pass ``--all`` to test.py (which I normally do, but I guess most people don't, in which case most people wouldn't see these failures). I opened a new issue about that: http://www.zope.org/Collectors/Zope/1947 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Reminder: feature freeze November 1.
[Tres Seaver] ... Which bugs? Sombebody needs to define this, or else risk having the outsiders just walk away. Insiders too ;-) *I* know of no showstoppers: all unit tests are passing, Not on Windows: Windows test failures on Zope trunk http://www.zope.org/Collectors/Zope/1931 CMF-trunk runs fine on the Zope trunk, etc. Certainly agree it would help to have a specific list of what (if anything) still needs to fixed. FWIW, I don't expect Windows test failures to hold up a beta release (note that I didn't say that's a policy I agree with ;-)). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Reminder: feature freeze November 1.
[Tim] Windows test failures on Zope trunk http://www.zope.org/Collectors/Zope/1931 [Tres] Without Windows-centric developers who are motivated to investigate and fix those bugs, I don't know what else we can do. [Mark Hammond] That bugs points at http://mail.zope.org/pipermail/zope-dev/2005-October/025512.html, which quotes Tim as saying: : No idea where this slash-vs-backslash confusion ultimately comes from, : though. Who recently checked code in hard-coding / as a path : separator? So in this specific example, the problem seems less a lack of Windows centric developers, but more an abundance of non-Windows-centric developers :) These test failures appear at first glance to not be windows specific at all - just possibly pointing at non-portable code written by others. As a Windows developer, I'm afraid I have no idea where I would start looking for this bug. Alas, I was directed not to work on this bug report on the clock, and I haven't had spare time to donate to it (of course there's the usually irony with that: by now I've probably spent 3x as long typing about these bugs as it would have taken to fix them :-( ...). Because I'm sure I noticed the bug within a day or two of its first appearance, the obvious approach is to revert back to earlier revisions of the trunk until finding the checkin that caused it. I thought I wrote up enough clues on zope-dev at the time that whoever checked in the responsible change would think ah, that's related to what I did! at once. Alas again, nobody noticed. So that's a clear path to fixing this one: pinning the blame should be sufficient ;-) In the absence of the guilty party noticing they were to blame, it takes someone on Windows to do the binary search required (because someone on Linux won't see the failure). BTW, notice that the Python tracebacks had exactly the same \ vs / mixup in the same place (between lib and python) as the two originally failing tests. That suggests (but doesn't prove) that a change to sys.path is the ultimate cause. BTW2, I have no idea why the later-failing Five test started failing on Windows, and didn't spend any time investigating that one. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] commit(1) in ZCatalog
[Chris Withers] ZCatalog still contains a commit(1) at around line 589 of ZCatalog.py, which is causing the familiar 'Savepoints unsupported' error for us :-( Would there be any problem changing this to a savepoint(optimistic=True) on the 2.8 branch and trunk? There shouldn't be any problem -- go for it. I see that the 2.8 branch already uses transaction.savepoint() here, but should probably use transaction.savepoint(optimistic=True) instead. In the other direction, I see that zope/app/file/file.py still contains two subtxn commits on 2.8 branch, but not anymore on the trunk. That should also be changed on 2.8 branch. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] commit(1) in ZCatalog
[Chris Withers] ZCatalog still contains a commit(1) at around line 589 of ZCatalog.py, which is causing the familiar 'Savepoints unsupported' error for us :-( Would there be any problem changing this to a savepoint(optimistic=True) on the 2.8 branch and trunk? [Tim Peters] There shouldn't be any problem -- go for it. I see that the 2.8 branch already uses transaction.savepoint() here, but should probably use transaction.savepoint(optimistic=True) instead. [Chris] Done. Looks good. Thanks! In the other direction, I see that zope/app/file/file.py still contains two subtxn commits on 2.8 branch, but not anymore on the trunk. That should also be changed on 2.8 branch. Hmm, that smells Zope 3-ish to me, and my Zope checkout links that in by svn:externals, so I'll leave that for someone more Zope 3-savy... Ah, yes. That should go away by magic then if/when someone stitches in a newer version of whichever Zope3 2.8 branch is using. Or not wink. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Tim Peters] Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)): Check out a fresh, new copy of Zope trunk. Merge your branch into that. [Chris McDonough] Thank you! Eagds. I *thought* I had done just that. I had even printed out your previous handholding email about this before starting. But no. I did not. Instead, to my horror, I realized had *copied* a trunk working copy (of an unknown vintage, although up-to-date according to svn status after an svn up) and then I'd merged the branch into that. So about a minute ago, I followed your instructions literally, figuring that I'd be able to sheepishly move on afterwards, but unfortunately it comes out the same. Literally, here are the commands: $ svn co svn+ssh://svn.zope.org/repos/main/Zope/trunk $ cd trunk $ svn merge -r 38624:HEAD svn+ssh://svn.zope.org/repos/main/Zope/branches/zodb-blobs-branch Didn't you get any output here? I saved mine the last time I tried it, and I expected you'd see the same (except with Linux path separators): U doc C lib\python\Zope2\Startup\datatypes.py U lib\python\Zope2\Startup\zopeschema.xml U lib\python\Products\ZODBMountPoint\tests\testMountPoint.py U lib\python\Products\ZODBMountPoint\MountedObject.py D lib\python\Products\ZODBMountPoint\Mount.py D lib\python\DBTab\DBTab.py D lib\python\DBTab\__init__.py D lib\python\DBTab\CHANGES.txt D lib\python\DBTab\Exceptions.py D lib\python\DBTab\ClassFactories.py D lib\python\DBTab D lib\python\DBTab U lib\python U setup.py U utilities $ svn up ... which comes out with Fetching external item into 'lib/python/zope/thread' External at revision 39684. Fetching external item into 'doc/ZEO' A doc/ZEO/cache.txt A doc/ZEO/ZopeREADME.txt A doc/ZEO/README.txt A doc/ZEO/trace.txt A doc/ZEO/howto.txt Updated external to revision 39684. Fetching external item into 'lib/python/zope' svn: Working copy 'lib/python/zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) Drat. My apologies! Tried again I see that too. Since we _started_ with current Zope trunk here, and your branch doesn't change anything at or under lib/python/zope, I didn't expect that. I was wrong. I am reasonably confident that drinking just one more Yuengling will solve this problem in one way or another. Did it work out? I'm ill today and saw this on my way back to bed, but in 5 minutes of trying I didn't find any combination of rm -rf lib/python/zope and svn cleanup that got me unstuck. Will try again later. [Jim Fulton] svn:externals suck. A lot. As Tim suggested, you could throw away this check out and start over. That's what he tried above -- didn't work for him. Doesn't work for me either, but Chris may have clouded my mind ;-) A simpler thing you could do is to remove the zope directory and do an svn up. He tried that before and reported endless failures. That's why I suggested starting over to begin with. Should have worked ;-) Since we switched to using externals, we see lots of things like this. You just learn to delete the directories in question. That alone wasn't enough for my own Zope trunk checkout -- also needed to do svn cleanup. But the start over from scratch sequence above appears to leave Chris and me in a state where no amount of directory removal and svn cleanup gets rid of Fetching external item into 'lib\python\zope' svn: Working copy 'lib\python\zope' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) upon the next svn up attempt. Ah, it's the properties on lib/python that are screwing us here! Chris, throw svn revert lib/python into the mix too. That got me unstuck. The problem is that both Jim and I (at least) changed the set of externals listed in lib/python, and SVN absolutely sucks at merging property changes. The revert above gets you back to the externals Zope trunk actually uses today. That's vital because Zope trunk stitches in zope in an entirely different way now. Reverting also stitches in a version of ZODB we don't want to use, but after reverting you can do svn propedit svn:externals lib/python to put back the right version of ZODB for your branch (s/3.4.2/3.6.0b1/g). Alternative: I haven't tried this, but it should work wink too, and would be easier: instead of reverting lib/python, do svn propedit svn:externals lib/python and just delete the part stitching in `zope`; that's this line: zope svn://svn.zope.org/repos/main/Zope3/tags/ZopeX3-3.0.1-Zope-2.8/src/zope The goal here is to end up with externals on lib/python that do _not_ stitch in zope
Re: [Zope-dev] Mountpoints
[Chris McDonough] This merge has been done. Woo hoo! Thank you, Chris! I since stitched in ZODB 3.6.0b2, which is the most recent 3.6 (internal) release. That didn't seem to create any new problems. Since zopectl test ProductName no longer appears to do the right thing Sorry, never used it, don't know anything about it. and I can't seem to get test.py to run anything except the entire test suite, Do test.py -h for help. The -t option will probably help you. For example, $ \python24\python.exe test.py -vv -t test_pop Running tests at level 1 C:\python24\lib\whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) Running unit tests: Running: test_pop_default (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_raises (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_simple (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_raises (AccessControl.tests.testZopeGuards.TestListGuards) test_pop_simple (AccessControl.tests.testZopeGuards.TestListGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestListGuards) Ran 7 tests with 0 failures and 0 errors in 0.000 seconds. $ \python24\python.exe test.py -vv -t pop_val Running tests at level 1 C:\python24\lib\whrandom.py:38: DeprecationWarning: the whrandom module is deprecated; please use the random module DeprecationWarning) Running unit tests: Running: test_pop_validates (AccessControl.tests.testZopeGuards.TestDictGuards) test_pop_validates (AccessControl.tests.testZopeGuards.TestListGuards) Ran 2 tests with 0 failures and 0 errors in 0.000 seconds. I didn't create any new tests because I wouldn't have had the time to create tests and run them iteratively. Sorry to ruin your weekend, then ;-) That said, all existing tests pass and I did do some interactive stress-testing of the sessioning mount point, both of which made me feel comfortable enough to go ahead and do the merge. The tests appear to be in exactly as good shape on Windows now as they were before (there are test failures on Windows; opened a Collector issue about them Thursday), so you have a plausible defense in court. I won't sue you if you won't sue me ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiencing TypeError: The object is not a PySECURITY_ATTRIBUTES object
The bad news is that I don't think I'll ever put in enough time to fully understand what went wrong here. The good news is that the newly-released Zope 2.8.4 Windows installer, at http://www.zope.org/Products/Zope/2.8.4 includes pywin32 build 205. If that doesn't fix PySECURITY_ATTRIBUTES problems for Plone users, you know which Mark Hammond to contact ;-) Thanks to all for the help! ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Tim, trying to comfort a suffering Chris] ... End of story. Unless you feel you need to make another branch. In that case, still do the two steps above first. Then create a new branch from Zope trunk, svn switch your merged sandbox to that new branch, then svn checkin. That last part should have said svn commit. If it had, you'd already be done ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] FWIW, I know a couple of people are depending on this, so here's an update. I am working on merging multidatabase support, but I'm having some merge/update troubles (if you're interested in the symptoms, see http://www.plope.com/Members/chrism/heres_to_cvs). I suspect I'll work it out, but I've got my nose in Subversion documentation at the moment. Jim redid the way Zope trunk stitches in Zope3 since you last looked at this, and that can create some mechanical problems (of the kinds you're seeing, in fact). The svn docs probably won't help. Suggestion (which is repetition of what I suggested before this happened, but we'll gracefully let that pass ;-)): Check out a fresh, new copy of Zope trunk. Merge your branch into that. End of story. Unless you feel you need to make another branch. In that case, still do the two steps above first. Then create a new branch from Zope trunk, svn switch your merged sandbox to that new branch, then svn checkin. This way you shouldn't have any of the problems you've been seeing. If you do, double your money back ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] New(ish) Windows test failures on Zope trunk
I see two test failures today on Zope(2) trunk, WinXP, Python version
doesn't matter (same thing under 2.3.5 2.4.2).
Failure in test testRegisterTranslations
(zope.app.i18n.tests.testi18ndirectives.DirectivesTest)
Traceback (most recent call last):
File C:\Code\Zope\lib/python\zope\app\i18n\tests\testi18ndirectives.py,
line 55, in testRegisterTranslations
eq(util._catalogs, {'en': [unicode(path)]})
File C:\python23\lib\unittest.py, line 302, in failUnlessEqual
raise self.failureException, \
AssertionError: {u'en':
[u'C:\\Code\\Zope\\lib\\python\\zope\\i18n\\tests\\locale\\en\\LC_MESSAGES\\zope-i18n.mo']}
!=
{'en':
[u'C:\\Code\\Zope\\lib/python\\zope\\i18n\\tests\\locale\\en\\LC_MESSAGES\\zope-i18n.mo']}
One dict has a Unicode key there while the other dict doesn't, and the
python part of the value is preceded by a backslash in one but a
forward slash in the other.
Then:
Failure in test checkDuplicate (zope.configuration.config.ConfigurationContext)
Failed doctest test for
zope.configuration.config.ConfigurationContext.checkDuplicate
File C:\Code\Zope\lib/python\zope\configuration\config.py, line
259, in checkDuplicate
File C:\Code\Zope\lib/python\zope\configuration\config.py, line 281,
in zope.configuration.config.ConfigurationContext
.checkDuplicate
Failed example:
try:
c.checkDuplicate(d + os.path.normpath('/bar.zcml'))
except ConfigurationError, e:
str(e).endswith(bar.zcml' included more than once)
Expected:
True
Got nothing
_Looks_ like the test expected ConfigurationError to be raised, but
that ConfigurationError was not raised.
Oddly enough, I believe this is related to the first test failure.
Dumping some prints in checkDuplicate() shows that, when the test
fails, `path` is
'C:\\Code\\Zope\\lib\\python\\zope\\configuration\\bar.zcml'
and self._seen_files is a set with two elements:
'C:\\Code\\Zope\\lib/python\\zope\\configuration\\bar.zcml'
'\\foo.zcml'
As in the first test too, the character preceding the python part
differs. The path passed is indeed not in the set of _seen_files, so
ConfigurationError is indeed not raised.
No idea where this slash-vs-backslash confusion ultimately comes from,
though. Who recently checked code in hard-coding / as a path
separator?
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiencing TypeError: The object is not a PySECURITY_ATTRIBUTES object
[Mark Hammond] FYI, there is a new pywin32 build out now that should solve this problem without requiring any imports to be reordered. Yay! It would be great if whoever turns the crank for the next Zope/Windows builds (which may even turn out to be me! :) uses build 205. Andreas Jung made a surprise release of Zope 2.8.4 today, but only the tarball, not a Windows installer. If you want to make the latter, more than fine by me, else I'll try to make one tomorrow (with your build 205, of course -- will require some retroactive patching of the 2.8.4 tag no matter who does it). Sadly, I believe it is not trivial to install a new pywin32 build into a Zope binary. You could patch it up though by opening the pywin32 release executable in WinZip (or similar), then replacing 'pywintypes.py' and extracting a new _win32sysloader.pyd module. Ya, like Windows users are gonna do _that_ wink. Finally, I believe another way to solve this problem would be to remove pywintypes23.dll from the system32 directory (the the underlying problem is that 2 copies of this DLL are being loaded into memory). However, doing this may prevent other things (such as your existing Python installation) from working correctly, so do this with caution. Zope does not install anything into system32, so presumably something else on your system is also using Python. All recent PySECURITY_ATTRIBUTES complaints I know about have come from people using both Zope and Plone. I don't know anything about Plone installation, but it's natural to suspect that Plone is the source of the other pywin32 installation, and possibly of compounding sys.path convolutions too. So, a natural question based on this ignorance: is it enough for just Zope to install build 205, if Plone also installs its own (older) pywin32 and mangles sys.path so that its pywin32 is also visible? I suspect (but don't know) that's what's happening. It would be a lot better if a Plone user tested the proposed solution before we release another Windows Zope that may still turn out not to solve Plone's problems here. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Experiencing TypeError: The object is not a PySECURITY_ATTRIBUTES object
[Chris Mattmann] Okay, I have reproduced the error even with Zope 2.8.2-final on win32. The same PySECURITY_ATTRIBUTES object error appears even with 2.8.2. Any ideas? Short of not using Plone wink, see this Collector item, which I expect is the same issue: http://www.zope.org/Collectors/Zope/1925 I believe what it says there. Until Mark Hammond can make a new release of pywin32, you'll have to search for code that: imports win32api [and] add a line above that: import pywintypes That's already been done in Zope. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] There is a wrinkle about performing this merge that eluded my memory until now. To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code databases and database_name options that may now be passed into ZODB.DB's constructor: http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=38626r1=38574r2=38626 The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. ... Chris, here's the current state: - The following is on current ZODB trunk, and on the new tag svn://svn.zope.org/repos/main/ZODB/tags/3.6.0b1 I suggest changing the zodb-blobs-branch to use that tag now (and merging that branch to Zope trunk when it's all happy again). - I added an optional new database_name key to zodb config. I understand that you may not want to use it in Zope 2.9, taking the database name from Zope's zodb_db section instead. That's fine. I expect that whatever name (however decided) should be used can be poked into config.database_name before calling ZODBDatabase.open(). Should check that the section name and database_name key aren't both specified? Probably, ya. - ZODBDatabase.open() has an optional new databases= argument, so that part's still exactly the same way you did it (except that I didn't add that argument to BaseConfig.open() too -- I don't think it belongs there, as BaseConfig is also a base class for classes whose open() overrides don't support a databases= argument; the ZODBDatabase subclass _extends_ BaseConfig's open() in this respect). Is there more I can do to help this along (or, perhaps, delay it more ;-))? If so, just ask. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
Chris, FYI, I stitched ZODB 3.6.0b1 into zodb-blobs-branch, and changed ZopeDatabase.createDB() to plug database_name into config instead of passing it to ZODBDatabase.open(). The checkin msg summarizes test results; since I haven't work on this branch before, I'm not sure what was expected here (I didn't get a clean test run before stitching 3.6.0b1 in either): http://mail.zope.org/pipermail/zope-checkins/2005-October/029904.html How would you like to proceed? ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: SVN: Zope/branches/zodb-blobs-branch/ Move to ZODB 3.6.0b1.
[Tim Peters] Log message for revision 39583: Move to ZODB 3.6.0b1. ZopeDatabase.createDB(): Plug database_name into config rather than passing it to ZODBDatabase.open(). More should be done to detect conflicting zodb_db section name and database_name, but I'm not sure where all the relevant code is. There are a number of DeprecationWarnings about subtransactions when running the tests. Should be repaired. One test failure, but doesn't look like it's related to ZODB: [Tres Seaver] Nope. This one can be remediated by merging the fix on the current Five trunk: $ svn merge -r18808:18809 svn://codespeak.net/svn/z3/Five/trunk/ . That one should go away by magic then when the branch is merged to the trunk (this test is passing on Zope trunk -- or doesn't exist anymore on Zope trunk -- or something ;-)). A number of deprecation warnings should go away by magic too (e.g., OFS/Image.py on Zope trunk no longer uses subtransactions). The bulk of the deprecation warnings are coming out of ZODB's own tests, most from testZODB.py. I'm still baffled by that, because they don't show up when running tests from a ZODB checkout, or when running the Zope3 test suite. testZODB.py is trying to stop them: # deprecated37 remove when subtransactions go away # Don't complain about subtxns in these tests. warnings.filterwarnings(ignore, .*\nsubtransactions are deprecated, DeprecationWarning, __name__) While that's been effective in ZODB and Zope3, for some reason it doesn't seem to accomplish anything in zodb-blobs-branch ... ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] Thanks for this! Not required, so long as I get to thank you for finishing it ;-) Looks like that test failure is incidental and not symptomatic of changes made to ZODB. I think Tres may have said that it can be fixed by merging in a fix from the Five HEAD, but I don't know this for fact first-hand. I'm sure that failure will go away by itself when you're working on the trunk instead of the branch. What I'd do now: - Check out Zope trunk. - Merge the branch into your trunk sandbox, and forget the branch. - Fix merge conflicts. I got one, in datatypes.py, and I didn't know immediately what to do about it so stopped there. You'll have better luck ;-). Note that, under SVN, after you fix a conflict, you have to do svn resolved path/to/conflicted/file; that's a gimmick to make sure you don't forget about conflicts. - svn up to make sure you've got all the externals the merged files point at. - svn up from time to time thereafter, to suck in other trunk changes as they get made. - Check it in when it's stable. - If it takes longer than expected, make a _new_ branch _from_ your merged-into-trunk local trunk sandbox. (That's easy: make a branch directory, svn switch to it from your local merged trunk sandbox, and svn commit -- all done). It's encouraging that most of the tests pass but there are a paucity of tests that specifically test Zope 2 multidatabase-based mount points. There are a few convincing-looking decoys in Products.ZODBMountPoint.tests but I think I'll need to create a few more to get the warm and fuzzies before doing the merge. As above, you can do a _local_ merge right away. This would save you from other decoys (like the DeprecationWarnings that would no longer exist if you were using the trunk instead of the brach, and the failing-on-branch-but-not-trunk Five test). I recall that, historically, the Zope tests never failed when Zope mounting was in fact broken, so a fat +1 to beefing test coverage there. I have this on my plate for Wednesday evening. Understood; there really isn't any good TV on Wednesdays anymore ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [Zope-CMF] Better DeprecationWarnings (was Re: SVN: CMF/trunk/CMFDefault/Portal.py - reverted Portal.py change of r39125 to fix BBB temporarily)
[Tim Peters] Note: sometimes _internals_ use deprecated gimmicks in order to support deprecated gimmicks too, and then stacklevel=3 is too small. It's happened so rarely in ZODB that I haven't tried to do something about that yet. [Chris Withers] Interestingly, I've found that even this is sometimes not enough, since you don't know whether you want the caller, the caller's caller or further up the chain than that. I haven't seen much of that. One place I did is in deprecating subtransactions, where many paths thru the ZODB code have to pass on the original is this a sub or a 'real' transaction? flag. In those cases, the relevant methods also grew an optional `deprecation_wng` argument defaulting to True, and _internal_ calls to such methods explicitly pass deprecation_wng=False. Is there any way to get the warnings stuff to actually emit a traceback so it can be followed? No; the `warnings` module doesn't even import the `traceback` module, let alone use it. You can print a traceback yourself by using the `traceback` module, and if you're determined enough you could replace warnings.showwarning() with a function of your own (see the docs for warnings.showwarning, and possible for traceback.print_stack). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Mountpoints
[Tim Peters] I think it's worse, but mostly because a key with name name is also an option in _related_ sections, but with unrelated meaning. For example, if you had a nested zeoclient section there it could also have specified a name key, which would have nothing to do with the zodb key named name. Nesting options with the same name gets confusing quickly. OTOH, I would like the explicit key better if it had a different name, say zodb multidb-name main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb multidb-name a filestorage path $DATADIR/A.fs /filestorage /zodb [Florent Guillaume] Yes, please. There is already confusion for cache-size, let's not repeat that with another key. Note that database-name is more expressive, I think Since the name of the corresponding DB argument is database_name, and all the docs that exist for this call it database_name too, that's hard to argue against ;-) (the multi seems like an implementation detail to me). Not really: a DB's database_name was introduced specifically for the new-in-ZODB-3.5 multidatabase feature, and has no meaning or use apart from its multidatabase role. That's better explained in the ZConfig description section for the key than in the name of the key, though. If Jim doesn't object soon, I'll proceed with adding a database-name key to ZODB's config. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Mountpoints
[Chris McDonough] | Note that I don't have a strong opinion about this either way but I will note that at least Zope 2's subclass of the zodb config handler will need to continue to be willing to use the section title as the database name for backwards compatibility reasons, as people who have older Zopes will want to use their older config files (which have zodb_db sections that have section titles, and no database-name key) with new Zope releases. Note that when you look at Zope2's zopeschema.xml's zodb_db config, there isn't a clue there that the section's name is used for something, let alone what it's used for. This lack of discoverability goes away when using an named key, and that's a better long-term place to be. I don't expect that adding an optional named key to zodb config will _stop_ zodb_db config from doing whatever it wants to do instead. If it does, I agree that would be a problem. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [Zope-CMF] Better DeprecationWarnings (was Re: SVN: CMF/trunk/CMFDefault/Portal.py - reverted Portal.py change of r39125 to fix BBB temporarily)
[Tres Seaver -- I think; I'm missing context for this email] Note that I have just figured out that we can make DeprecationWarnings more useful by passing the 'stacklevel' argument to 'warnings.warn'; passing a value of 2 for that argument causes the warning to be reported against the *caller* of the code issuing the warning, which makes it possible to find and remove the deprecated use. I can recommend the approach I use in ZODB. There's a utility module in ZODB, containing (among other things) functions like this one: # Raise DeprecationWarning, noting that the deprecated thing will go # away in ZODB 3.6. Point to the caller of our caller (i.e., at the # code using the deprecated thing). def deprecated36(msg): warnings.warn(This will be removed in ZODB 3.6:\n%s % msg, DeprecationWarning, stacklevel=3) So every gimmick that's going to go away in ZODB 3.6 imports `deprecated36` from the utility module, and calls it with an appropriate message. As an intended bonus, when I release ZODB 3.6 I can just grep for deprecated36 to _find_ the code that's supposed to go away (I also annotate tests and docs that should go away with deprecated36). Using a common function also ensures that every deprecation warning starts with the same string (identifying the release in which the thing will go away). Note: sometimes _internals_ use deprecated gimmicks in order to support deprecated gimmicks too, and then stacklevel=3 is too small. It's happened so rarely in ZODB that I haven't tried to do something about that yet. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Tim Peters, on multidatabase support in Zope3] ... Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that? [Jim Fulton] Do you mean an example of a zope.conf that uses it? Yes, that's all -- just a concrete example. I could have guessed one (and did later ;-)), but guesses can be wrong. From a customer engagement: zodb main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb a filestorage path $DATADIR/A.fs /filestorage /zodb Thanks! That did the trick. We decided to use the section names for the database names. This was to avoid changing ZODB. I'm not sure that that was a good idea. Let's change it. This approach has two disadvantages: - Because section names are case insenstive, database names end up being lower case, whether we want them to be or not. - It may not be obvious that the section name is also the database name. It isn't obvious -- and unlike for other zodb keys, a user can't look in ZODB's component.xml now to find out anything about this usage. They can, e.g., look there to see that they can specify cache-size, that it's an integer, and that it defaults to 5000. They can also find helpful description sections for many keys. Using the section name is pure out-of-the-blue magic in contrast. I'm really unsure about whether this is a disadvantage. I'm not sure if: zodb name main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb name a filestorage path $DATADIR/A.fs /filestorage /zodb is better or worse than the first version. I think it's worse, but mostly because a key with name name is also an option in _related_ sections, but with unrelated meaning. For example, if you had a nested zeoclient section there it could also have specified a name key, which would have nothing to do with the zodb key named name. Nesting options with the same name gets confusing quickly. OTOH, I would like the explicit key better if it had a different name, say zodb multidb-name main filestorage path $DATADIR/Data.fs /filestorage /zodb zodb multidb-name a filestorage path $DATADIR/A.fs /filestorage /zodb I'm inclined to think that any time you have sections of the same type, it is desireable to give them names, in which case we might be tempted to list the names twice. Sounds orthogonal to me. If it's desirable that whenever multiple sections of the same type appear, they must be given names, that's fine, but the way to enforce or encourage that isn't to make all sections that may appear more than once give some _meaning_ to the section name. It was just expedient in this specific case, right? ... I'd be happy to plumb this through the factories open method. It would seem to me that we only need to be able to pass a databases argument. Right. The factory presumably knows it's own name. It could then pass the databases dict and the name to the DB constructor. If I change the zodb config to say there's a new optional key for the multidatabase name (like the multidb-name key in the made-up example above), then the factory will have the same access to that as it has for other existing ZConfig-specified keys (like cache-size). BTW, I think there's a related buglet in Zope3's multi_database(): name = factory.name or '' ... db.database_name = name Defaulting to an empty string for the name is really a bit of abuse, since the documented default database_name for ZODB.DB is unnamed, and I doubt Zope3 documented that it changed this default ;-). An explicit ZConfig key here would supply that correct default. ... I haven't looked at Chris's changes. I was pretty happy that the changes we made in Z3 were fairly localized and small. Adding the optional new key to the zodb config, and threading the `databases` arg thru ZODB's config.py, are also small changes. But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense. OTOH, this feature has hardly been used in Z3. I added to ZODB because I had been meaning to for some time ad because we needed it for a customer. I don't think anyone else has used it, so I don't think there's much established pattern in Z3. Then again, I'm not sure, except for the case insentitivity issue that we didn't do it the best way. I'd much rather revisit the case insenstitivity of section names in ZConfig. I think that if ZConfig section names were case sensitive or at least case preserving, I'd be happy with the approach we took. Note that if we add an explicit new key, the case issue goes away (for this specific
Re: [Zope-dev] Mountpoints
... [Chris McDonough] Thanks for the offer! I won't be able to visit ZC world HQ tomorrow, though unless you'd be there and willing to start around 10pm. Alas, they're still under the delusion that 10 _am_ is late here, so while I agree 10pm is saner on all counts, I'll be gone before then. Other duties is my official excuse but I'm also horrified by the idea that I'd be expected to wear pants if I came over there. That's just uncivilized. :-) There was such a backlash against the pants required policy that it's OK to wear diapers instead now. Progress, anyway. ... [Tim] Check. Question: does zodb-blobs-branch contain anything you _don't_ want to see on Zope trunk now? You didn't mention anything like that here. No (save for inappropriate svn:externals to ZODB and ZEO). In that case, and since you say later there's no reason to keep this branch after the merge is done, I suggest merging directly from zodb-blobs-branch to Zope trunk. Unless I'm missing something, there really doesn't seem to be a point to creating another intermediate branch first. ... Yes. The zodb-blobs-branch can just die after this merge if there's a way to get delete branches entirely. Yes and no ;-) A branch or a tag in SVN is just another named directory, with non-mandatory conventions for choosing its path name. It can be deleted, like any other directory. That doesn't get rid of it entirely, just as no directory can be gotten rid of entirely: you can always revert the checkin in which it was deleted, and then it will magically reappear. Unlike as under CVS, though, deleted branches don't keep punching you in the face if you don't go out of your way to find them. For example, these are all the visible ZODB branches today: $ svn list svn://svn.zope.org/repos/main/ZODB/branches 3.3/ 3.4/ 3.5/ blob-merge-branch/ ctheune-blobsupport/ Note that none of the other branches we created at the ZODB sprint, or most of the branches created since then, still show up (I deleted them after merging to then-current ZODB trunk). SVN is very nice this way! If there is to be a long-lived branch, it will be the blob-merge-branch of ZODB. Afraid so, yes. Is it time to delete the ctheune-blobsupport branch? Given that Zope 2.9 is not going to ship with blob support due to feature freeze, I think this means that we have until May to allow the blob-merge-branch to get utterly out of sync with the ZODB trunk. We can then easily wait until, say, the last week in April to worry about issues caused by that desynchronization. The work necessary to remerge should provide just the appropriate amount of delay to allow blobs to miss the next major Zope release. ;-) Excellent! I'm glad you're thinking hard about this -- it gets hard delaying release after release all by myself ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] There is a wrinkle about performing this merge that eluded my memory until now. To support multidatabases within Zope, it was reasonable to change ZODB.config.ZODBDatabase to support the heretofore likely-unused-by-real-world-code databases and database_name options that may now be passed into ZODB.DB's constructor: http://svn.zope.org/ZODB/branches/blob-merge-branch/src/ZODB/config.py?rev=38626r1=38574r2=38626 The current blob-merge-branch code depends on this change being available in the ZODB revision it uses. In case you're interested, the code that actually makes use of this feature in the zodb-blobs-branch is in the Zope2.datatypes.DBTab.getDatabase method. Is this change acceptable for a merge into the ZODB HEAD? Turns out that a release of Zope3 has already been made that supports multidatabases, and I'd naturally prefer to follow the lead of a Zope that's already out there. Jim showed me the Zope3 implementation code and an example today. I found the code easily (on Zope3 trunk), but can't for the life of me find anything there that looks like his example. Jim, where is that? The Zope3 code in question is in src/zope/app/appsetup/appsetup.py function multi_database(). Note that they didn't change any ZODB files, instead they give values to a DB's .databases and .database_name attributes after constructing the DB. While that might be questionable in general cough, the implementation of multidatabases was meant to be both concrete and public. It's not an accident that ZODB's tutorial tests/multidb.txt doctest explains and exploits details of the concrete implementation -- it's not meant to be abstract. IOW, poking in new values for these attributes isn't considered to be evil. I believe (here's where the example I can't find would nail it) they use the name on a zodb section as the DB's database_name. Fred points out that ZConfig section names are case-insensitive, forced to lowercase, so that zodb CHRIS and zodb cHris have the same name. That's not ideal, and threading these attributes throughout ZODB's config.py instead (as you did) would be a sane way to worm around that. But for right now, I think doing it differently than Zope3 does it would cause needless confusion more than it would help. Enhancing Zope3 and Zope 2.9 in the same way(s) here could make sense. Some mechanics: if we do need to make changes, ya, ZODB trunk is the place to do it. Work on the branch should use ZODB trunk now. When that's as ready to go as it's going to get, let me know and I'll make an internal release of ZODB 3.6 so we can use a ZODB tag before merging into Zope trunk. (An internal release just means I update ZODB's NEWS.txt, fiddle version numbers and dates in umpteen places, and make a tag so other projects can use that -- it's the tag that's important here; an internal release does not involve making tarballs, Windows installers, announcements (etc), so it's much cheaper and goes much faster (minutes vs hours) than making a public release.) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] svn.zope.org borked?
[Andreas Jung] all changes I made for the last hotfix release (patch for the reST .. include problem) have disappeared from the SVN (both from 2.8 branch and the trunk). Are you sure? The SVN log on Zope trunk shows what I guess are the relevant checkins, revs 39013 and 39014, on 2005-10-09, both with checkin comment update to docutils 0.3.9 If that's it, they appear to be there. There is a problem with them, though: the svn:eol-style property isn't set to native on any of the files added by these checkins, so they look like gibberish under many Windows tools (and, e.g., windiff thinks every line differs between the SVN Zope trunk and CVS Zope 2.7 branch copies of docutils). While Python doesn't care (Linux-style .py files work fine on Windows), that should get fixed. ... Is there something broken with the SVN database? Maybe, but don't see any real evidence of that yet. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). [Jim Fulton] Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. FYI, Zope trunk (in SVN) is current 2.9 development. [Chris] OK, I will do so. In that case, I'm willing to share wink this set of probably-related open Collector issues: ZODBMountPoint should not monkey-patch ZODB http://www.zope.org/Collectors/Zope/1525 Clean up mounts, TemporaryFolder http://www.zope.org/Collectors/Zope/1800 ZODB: Mounting broken for non default transaction manager http://www.zope.org/Collectors/Zope/1875 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Mountpoints
[Chris McDonough] This is already done on the zodb-blobs-branch. I would be happy to create a mountpoint-branch that does not externally link the ZODB with blob support, then merge the changes in to the Zope 2 HEAD (and 2.9 branch if one exists). [Jim Fulton] Ah, I had forgotten about that. It would be great to merge the mountpoint work into the head. There isn't a 2.9 branch afaik. FYI, there's a problem with that: Zope trunk (2.9) is still using ZODB 3.4, and multi-databases weren't introduced before ZODB 3.5. The _intent_ has been that 2.9 would use ZODB 3.5 or even 3.6, but that got hung up due to problems with switching the 5 integration part to use zpkgtools. I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the 5 guys to do something before trying again. Sorry for not being more specific ... ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Five status on Zope trunk (was Re: [Zope-dev] Mountpoints)
[Tim] I'm copying Fred because he may remember more about this than I do. Fred, do you know of a reason why I can't stitch a newer ZODB into Zope(2) trunk? I have a dim, fading memory of the last attempt failing, and of agreeing in email to wait for the 5 guys to do something before trying again. Sorry for not being more specific ... [Fred] We need to do the zpkg/ZODB switch all at once because it affects how extensions get their include files. Aha! That vaguely rings another vague bell ;-) The ZODB 3.5+ source code spells #include in its C files in a different way, and that can't work with the way Zope trunk is currently built. When I tried switching the Zope 2 trunk before, there was a problem due to Five tests failing. I don't remember the details, but the Five-folks seemed to think things would be better with newer versions of Five. Since Five development is done elsewhere, though, it's hard to tell what the deal is with that. Can anyone working on Five comment here, please? I gather that the first 2.9 beta release occurs in 2 weeks, and if Fred can't do his zpkgtools work, I can't update the trunk to a newer ZODB, and then Chris McDonough can't do his rework of mount points (the original topic of this thread), and then (best case) the world economy collapses. I snapshotted what I did get done as the zpkg-build-branch, so we can get back to it without duplicating work. So there's no reason yet to be depressed ;-) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] [Zope 2.8.2] Minor delay
[Andreas Jung] I originally scheduled Zope 2.8.2 for October 12th but I have to delay the release for some days (possibly one week) because of unforeseeable travel and a lot of important work. I hope this is fine for everyone. Works for me ;-) Is Zope 2.7.8 also affected (2.7.8 was also scheduled for October 12)? I need to know in advance so I can get matching ZODB 3.2.10 and 3.4.2 releases made and stitched in at the right times. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] indexing multiple strings with ZCTextIndex broken in Zope 2.7.7
[Martijn Faassen] The behavior of ZCTextIndex was intended to be able to index strings, but also lists of strings. [Tim Peters] Note that there's a long discussion of this here: http://www.zope.org/Collectors/Zope/1815 Entry #31(!) claimed the patch is also applied on the 2.7 branch since 2.7.7b1. If that's not so, probably best to reopen that issue. [Martijn] Thanks for pointing out the issue. It does not appear to be so, as the wrong behavior persists on Zope 2.7.7 and I couldn't find it on the Zope 2.7 branch. Can you reopen the issue? I've tried to log in to see whether I can, but it don't seem to log in properly into the tracker or something. I tried to reopen it, but unsure of the outcome. I can see the new entry via this URL: http://www.zope.org/Collectors/Zope/1815/ but not via this one: http://www.zope.org/Collectors/Zope/1815 (the difference is that the first has a trailing slash). If history is a guide, over time the difference in displayed content will go away. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Move Zope trunk to ZODB 3.5
Heads up! If you have a Zope trunk checkout, you'll need to recursively delete directory lib/python/Persistence before an update will succeed. If you try to update before deleting that directory, you'll see something like: Failed to add directory 'lib/python/Persistence': object of the same name already exists. You may also need to do svn cleanup and try again, if you don't delete the directory before trying to update. [Tim Peters] If there are no sane wink objections, I'd like to move Zope trunk to using ZODB 3.5 tomorrow (Friday). ... This didn't happen. There's a chicken-and-egg problem with incorporating zpkg changes too, and that's probably going to wait for a newer release of Five. A related changed would happen soon after (probably also on Friday): the ExtensionClass-based Persistence package still lives in the ZODB part of the repository, despite that it can't even be compiled from a ZODB checkout (the prerequisite ExtensionClass implementation lives in the Zope part of the repository). So the plan there is to remove the svn:externals stitching Persistence into Zope from ZODB, and move the Persistence package from ZODB trunk to Zope trunk. That part did happen. Removing the svn:externals line for Persistence from Zope trunk's lib/python, followed by an ``svn move`` of the Persistence package (from ZODB trunk to Zope trunk), caused the headaches at the top of this message. I'm afraid current SVN gets a bit lost when switching from copies to externals, or vice versa. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] RE: [ZODB-Dev] Subtransaction backward compatibility
[Dieter Maurer] Subtransactions used to be used for two purposes: * ensure that newly created objects get _p_ attributes (especially _p_jar and _p_oid) * release memory in the mid of large transactions (i.e. reading and/or writing large amounts of objects) With ZODB 3.4, subtransactions are implemented as savepoints. They can still be used for the first purpose. But, they no longer start cache garbage collection. As a consequence, subtransactions/savepoints can be dropped at places where only the second purpose has mattered, e.g. in Zope's ZCatalog. Over on zodb-dev, Jim (Fulton) confirmed that it was his intent that making a savepoint would trigger cache gc. It's a ZODB bug that it currently does not; I'll fix that: -Original Message- From: [EMAIL PROTECTED] On behalf of Jim Fulton Sent: Tuesday, August 23, 2005 2:53 PM To: Tim Peters Cc: zodb-dev@zope.org Subject: Re: [ZODB-Dev] Subtransaction backward compatibility ... Assuming that we no longer call incrgc, that would be an oversight. When a connection does a savepoint, it should also do an incrgc. Note that applications that *really* want to reduce memory after a savepoint may and often should make explicit cache-management calls on the transaction. This should still work. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] BTrees.Length conflict resolution
[Martijn Pieters] Did the conflict resolution code for BTrees.Length ever work? Because as it stands now the code will fail You haven't seen this fail, you're _deducing_ that it must fail, right? as it assumes that integers are passed in, instead of state dictionaries: def _p_resolveConflict(self, old, s1, s2): return s1 + s2 - old Don't overlook this other Length method: def __getstate__(self): return self.value That is, when a Length instance is asked for its state, it returns an integer. Similarly setting its state expects an integer: def __setstate__(self, v): self.value = v See: http://docs.python.org/lib/pickle-inst.html ... If a class defines both __getstate__() and __setstate__(), the state object needn't be a dictionary and these methods can do what they want. As there are no tests for this that I can see (the BTrees tests are kinda very dense), Confirmed: there are no Length tests in ZODB. I am not too keen to go touch this, Good instincts wink. but I think this should read: def _p_resolveConflict(self, old, s1, s2): s1['value'] += s2['value'] - old['value'] return s1 I expect that would blow up (TypeError: unsubscriptable object), unless you also removed Length's __getstate__ and __setstate__ methods. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Is there a Zope 2.7.8 planned?
[Paul Winkler] I'm looking at the two patches that were uploaded for http://www.zope.org/Collectors/Zope/1810 which is assigned to me... BUT: 1) Have we reached the end of the 2.7 line? I expect that's Andreas's call now. I still plug away adding patches to the ZODB 3.2 line (which is maintained solely for the benefit of the Zope 2.7 line). If there won't be another 2.7-line release, I'd be delighted to retire the ZODB 3.2 line too. Do I need to apply this to the 2.7 branch? Or can I just put this on the trunk and 2.8 branches and leave it at that? I really hate having to deal with both CVS and SVN for one fix... Do it one place, do cvs diff or svn diff to create a patch, then apply the patch to the other place? That often works fine the first time. 2) Since 2.8.1 is planned for tomorrow, should I a) hurry up and get it in before that, or b) wait because I don't want to risk merging a not-widely-tested patch so close to the final release? Does this require ZODB changes? If so, ZODB 3.4.1 final tarballs+installers have already been cut tested (just awaiting upload announcements). If I have to cut another ZODB release for Zope 2.8.1 (which is fine, if that's what's needed), the Zope 2.8.1 release should be delayed by a day to make enough time to do that. or c) just create a bugfix branch and leave it for somebody else to sort out the merge, since I'll be out of town for the next 2 weeks starting tomorrow and can't help if there's anything wrong with the fix? :-) Sounds like starting on #c now wouldn't be a waste of time in any case, and would leave all other options open. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Fixing Windows bugs in asyncore trigger code
As developed in a long thread starting at http://mail.zope.org/pipermail/zope/2005-July/160433.html there appears to be a race bug in the Microsoft Windows socket implementation, rarely visible (but disastrously when so) when multiple processes try to create an asyncore trigger simultaneously. Unfortunately, the relevant code appears to have originated in Medusa, and then got copy/paste'd all over creation. ZEO's copy is in ZEO/zrpc/trigger.py, and I've fixed it for ZODBs 3.2.10, 3.4.1 and 3.5. Zope-2_7-branch is using a 3.2.10 pre-release now, but no other version of Zope has stitched in a repaired ZODB yet. The original Medusa code appears to be in lib/python/ZServer/medusa/thread/select_trigger.py I've repaired that in CVS Zope-2_7-branch, and am in the process of repairing the same file in SVN Zope trunk and SVN Zope 2.8 branch. SVN Zope trunk and Zope 2.8 branch appear to inherit yet a 3rd copy of this code, in their lib/python/zope/server/trigger.py which appears to come from an older Zope3 snapshot. Zope3 trunk does not appear to contain any Medusa code (at least not under that name), but does contain zope/server/trigger.py which appears more to be a copy of an older version of ZEO's trigger.py than of Medusa's select_trigger.py. The short course here is that I'm repairing all but only the copies of this code that do _not_ appear in anybody's zope/server/trigger.py That file appears to have been introduced in Zope3, and I've lost track of which branches of Zope3 are still active now. I'll open a Zope3 collector issue on this, and will be happy to help repair it, but because I spend most of my Zope time trying to help Zopes 2.7 and 2.8 along, those are the only Zope versions where I'm confident about changing the right stuff in the right places. I'll repair Zope3 trunk's zope/server/trigger.py unless someone can tell me it's no longer used, but someone else will have to port that change to the still-active Zope3 branches (including the one(s) used by Zope 2.8 and Zope trunk). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope-Coders] Upcoming 2.8.1 release
[Andreas Jung] Just the usual reminder from the release management :-) Zope 2.8.1 b1 will be released next Wednesday [Tim Peters] Is this correct? Please confirm. http://www.zope.org/Wikis/DevSite/Projects/Zope2.8/MilestonePlan still says 2.8.1 b1: 2005/7/27 2.8.1 final: 2005/08/10 [Andreas] Argh...you're right again...I wonder why my Ical has all release dates one week earlier. I was not my goal to annoy or anger you :-) Good thing, too, since you accomplished neither -- I was simply in a state of panic. I'm much calmer now -- thanks wink. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope-Coders] Upcoming 2.8.1 release
[Andreas Jung] Just the usual reminder from the release management :-) Zope 2.8.1 b1 will be released next Wednesday Is this correct? Please confirm. http://www.zope.org/Wikis/DevSite/Projects/Zope2.8/MilestonePlan still says 2.8.1 b1: 2005/7/27 2.8.1 final: 2005/08/10 and next Wednesday is a week earlier than what that says. and 2.8.1 final two weeks later. Which would be 8/3, again a week earlier than the published 8/10. Please commit any pending fixes for 2.8.1 before the beta 1 release and not between beta and final release. This gets difficult for me (read ZODB) if I've got only 2 days now instead of the 9 I planned on. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Default ZODB cache size
[Florent Guillaume] How about boosting the default ZODB cache_size to something less ridiculous than the default 4000 ? I propose changing etc/zope.conf.skel to have an explicit value of 2. [Dieter Maurer] | That may already be a bit large: We use 15.000 with 6 workers and our Zope easily comsumes about 1 to 1.5 GB of RAM. I would keep the 4.000 default in order not to bring newbies with low RAM into problems. Experts can easily reconfigue the cache size. It's unclear to me which default we're talking about. ZODB's DB.__init__ has cache_size=400 (not 4000) as the default. The example ZEO client in skel/etc/zope.conf.in sets # cache-size 5000 as its default. Since this isn't a one size fits all setting anyway, I doubt that changing the default would help more than it would hurt. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: [Zope-Checkins] CVS: Packages/ZODB - FileStorage.py:1.135.6.9
[Chris Withers] O... is this surfaced through the Zope undo tab? Try it. I'm sure that was Dieter's intent: http://mail.zope.org/pipermail/zodb-dev/2003-October/006157.html ,,,' The patch also enhances FileStorage and lets its UndoSearch._readnext provide information about the transaction size. This is very valuable when you want to spot strange transaction sizes via Zope's Undo tab (much more convenient than fsdump) ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Windows Builds
[Tim Peters] Anyone else run on Windows routinely who's eager for glory? As Christian learned the hard way, it doesn't really work if running on Windows is an occasional afterthought in your daily life. [Andy McKay] Mark Hammond and I will take a look at this and see if we can help. Cool! Last I saw, Andreas was planning to release Zope 2.7.7b1 this Sunday (3 July), if you want some practice wink. I'm pretty sure Mark already understands the Zope Windows-installer build process about as well as anyone. If not, I'm happy to help. For someone who runs on Windows routinely, the machinery usually works smoothly. It can be a bear when it doesn't. One predictable problem is that Zope's setup.py gets out of date over time, because people run Zope tests from CVS or SVN checkouts in place, and the repackaging done by setup.py for an installation never gets tested that way. It's broken then for all platforms, but typically whoever builds the WIndows installer is the first to _notice_ it. For a micro release (like 2.7.6 - 2.7.7) a problem there is unlikely. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope 2.8: ZODB fix breaks undoable_transactions
[yuppie] http://svn.zope.org/?view=revrev=30334 changed the behavior of undoInfo() in a way that is not backwards compatible. That's true, or at least off-by-one different than recent ZODB 3.2s. Rev 30334 fixed two bugs in the implementation, so that the behavior matched what the documentation has always said undoInfo() did. I don't know when the implementation got out of synch with the docs, but however people want to resolve this I will not leave the implementation disagreeing with the docs. See http://www.zope.org/Collectors/Zope/1822 for details. I added details there. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope 2.8: ZODB fix breaks undoable_transactions
[yuppie]
...
Don't know what other people think. I believe restoring the old undoInfo
behavior and adjusting the documentation would be the best solution.
Fixing this in undoable_transactions would fork the behavior of both
methods and fixing all products that depend on the old behavior would
cause unnecessary trouble.
Can you document the behavior you want? Because there were multiple
bugs in the implementation before, I'm not exactly sure what old
behavior was in all cases; I'm certain that _some_ of it was purely
accidental, never intended, and utterly surprising (when last 0).
ZODB/interfaces.py's IStorageUndoable.undoLog documents the current
behavior, which matches what ZODB's UML docs have always claimed
behavior should be. This behavior is tested in ZODB too now, so any
change here requires fiddling code, docs and tests. If Zope requires
particular behaviors, it should grow tests for those too.
I'd be happy enough changing `first` and `last` to act like Python
slice indices instead, with the caveat that because there's other
weird non-Python behavior mandated when last 0 (then undo{Log,Info}
are documented as taking the absolute value of `last` as being an
upper bound on the # of results to return -- and old behavior was
related to that, albeit with bugs of its own), they cannot act like
Python slice indices unless `first` and `last` are both non-negative.
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope 2.8: ZODB fix breaks undoable_transactions
[yuppie] ... These are the two use cases I'm aware of. Both only use last 0 and both expect slicing behavior for positive values, e.g. these conditions should be True if we don't change undoable_transactions:: db.undoInfo(0, 20) == db.undoInfo(0, 99)[0:20] db.undoInfo(20, 40) == db.undoInfo(0, 99)[20:40] I'm willing to change undoInfo to do that; the old UML docs will just be wrong then. I don't care very much *how* this is resolved. All I want is to get the regressions in Zope 2 and CMF fixed. If it continues to be the case that Zope contains no tests verifying the behavior(s) it relies on, I'll have no way to know whether that stuff is fixed or not. ZODB will pass its own tests, but that's not enough (the ZODB tests have been passing all along). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZOBD and pointers
[Yair Benita] Reading this answer I understand that anything I store should be persistent, even if its a list I don't plan to edit. [Tim Peters] I wouldn't say that. For example, for _most_ applications it would be foolish to create a subclass of Persistent to store an integer, as opposed to just storing an integer directly. I can conceive of (unlikely!) applications where there may be advantages to storing integers as perisistent objects, though. [Tres Seaver] As, for instance, where the integer changes much more frequently than the other attributes, which are large enough that re-storing them just because the integer attribute changed is painful. Yup, that's a possible reason. Another recently popped up, which I'll exaggerate to make the point: you have 100,000 distinct integer ids, and you have 10,000 objects each with a (Python) list containing 10,000 of those ids. If you load those all into memory, Python will allocate space for 1*1 = 100 million integer objects, and that will consume more than a gigabyte of RAM. But if integers are stored as one unique persistent object per unique integer, it can't require more than 100 thousand distinct persistent integers in memory (because that's the total number of distinct integer ids). The RAM difference is a factor of about 1000 (but ignoring that it takes more RAM to hold a persistent wrapper than to hold a straight integer). I'll note that IISets avoid this problem via a different route: they hold their integers as raw bits, not as Python integer objects. When you extract an element from an IISet, a Python integer object is created on-the-fly to wrap the bits. Making the attribute a persistent sub-object also eliminates the chance of a ConflictError based on changes to the other attributes. I didn't follow that one. If other attributes change, they can trigger conflict errors, right? This is the use case which drives BTrees.Length, right? The important part of that is its conflict resolution method, which keeps track of the correct final size of a BTree in the face of concurrent mutations. BTrees don't keep track of their own size because every addition or deletion would have to percolate the change in size back up to the root of the BTree, and we'd get conflict errors on the root object then. As is, most additions and deletions change only the leaf Bucket node where the mutation takes place, giving mutation often-useful spatial locality in the face of concurrent mutations. I wish we could do better than that, though: from what I see, most people don't realize that len(some_BTree) takes time linear in the number of elements, and sucks the entire BTree into RAM. The rest seem to have trouble, at least at first, using BTrees.Length correctly. I suppose that's what you get when a scheme is driven by pragmatic implementation compromises instead of by semantic necessity. Give enough pain, it should be possible to hide the BTrees.Length strategy under the covers, although I'm not sure the increase in storage size could be justified to users who have mastered the details of doing it manually (the problem being that many uses for BTrees never care to ask for the size, so wouldn't want to pay extra overheads for keeping track of size efficiently). ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZOBD and pointers
[Yair Benita] ... suppose I have two different classes and both contain a list of a objects from a third class: class x has the attribute x.elements = [objects of class z] class y has the attribute y.elements = [objects of class z] As far as I understand python the lists x.elements and y.elements contain pointers to the z objects previously defined. Yes, Python lists always contain pointers -- even if it's a list of integers, the list actually contains pointers to integer objects. But since that's always true, it's not much help in answering your real question. In general, pointers make sense only so long as an object resides in memory. What I wanted to know is how ZODB handles that (or maybe I should say: how pickle handles that) when saving to a file. Will the pointers be converted to a copy of the z class objects or will one copy of the z class objects be saved and than the x.elements and y.elements will still be a list of pointers? Persistence has its own rules: if an object is persistent (an instance of a subclass of Persistent|), then its current state is stored uniquely in the database, and all references to it just save away (in effect) its persistent object id (oid, usually a 64-bit identifier uniquely assigned to each persistent object, and which retains its value for as long as the database exists). There are no exceptions to this for persistent objects. Oids are effectively a mechanism for building persistent pointers, and apply only to persistent objects. If an object is not persistent (is not an instance of a subclass of Persistent), it doesn't have an oid, and then there's very little possibility to share references to it on disk. Instead, on disk a copy of its state will usually get made everywhere it's referenced. So the answer to your specific question depends mostly on something you didn't reveal: does class z derive from Persistent? If it does, then _every_ reference on disk to an instance z1 is via z1's oid. If z doesn't derive from Perisistent, then almost all references on disk to an instance z1 will be via a physically distinct copy of z1's full state. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: ZOBD and pointers
[Laurence Rowe] ... Unless you use a special PersistentList ZODB will have no choice but to store a new copy of the whole list when that list is modified. Caution: that's true of a PersistentList too. The purpose of PersistentList isn't realy to supply more-effecient storage (that's the purpose of the various BTree classes). The purpose of PersistentList is this: myobject.my_list_attibute[3] = 4 If my_list_attribute is a plain Python list, the persistence machinery has no way to know that my_list_attribute's state mutated, so the assignment above will not get stored to disk at the next commit unless you _also_ do myobject._p_changed = True # or 1 If my_list_attribute is a PersistentList, then the persistence machinery does know when its state mutates, and there's no need to manage _p_changed manually. But in either case, the entire state of my_list_attribute gets stored to disk whenever any part of it changes. The only difference in what gets stored in the example above is that myobject's state also gets stored to disk if my_list_attribute is a Python list (assuming myobject._p_changed gets set to a true value by hand), while myobject's state does not need to get written to disk again if my_list_attribute is a PersistentList (then myobject refers to my_list_attribute via the latter's oid, and that oid hasn't changed, so there's no need to store myobject's state again). The entire state of the list attribute gets written out in either case. If you have long lists then this can be a big problem. Very true. The Persistent classes have special handling to make them more efficent. Sometimes true, but not in the PersistentList case. So instead of lists use PersistentLists If the goal is to save space, generally no, PersistentList won't help that; to the contrary, their state takes a little more space on disk than a plain list. and instead of dicts use BTrees, That one's differenent: a BTree is really a graph of (potentially _very_) many distinct perisistent objects, and BTrees were designed to support space- and time- efficient mutation. as these may be stored more efficiently in the ZODB. For BTrees, yes. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] ZOBD and pointers
[Yair Benita] ... Reading this answer I understand that anything I store should be persistent, even if its a list I don't plan to edit. I wouldn't say that. For example, for _most_ applications it would be foolish to create a subclass of Persistent to store an integer, as opposed to just storing an integer directly. I can conceive of (unlikely!) applications where there may be advantages to storing integers as perisistent objects, though. In the same vein, if there aren't multiple references to a single small list that doesn't change, there seems little (if any) point to making that a PersistentList. Note that there are other tradeoffs here too. For example, an attribute whose value is persistent is not loaded into RAM when its parent is loaded into RAM, but the full state of non-persistent attributes is loaded into RAM at the time their parent is loaded into RAM. That can have a major effect on time and memory demands, and in opposing directions. Or it may not -- it depends on details of the application's object access patterns. I was under the impression that a subclass of persistent will be larger in size for storage, so I avoided it in the cases mentioned. Is this true? Create a specific class definition, and it's easy to measure. It depends on the class. Certainly it costs more space to create a persistent version of a builtin Python type, and for the same reason it costs more space too to create any user-defined subclass of a builtin Python type. But for an object of a user-defined class, a persistent version takes more RAM when it's in memory (because it has to store info like the oid, and _p_changed, that non-persistent objects don't have), but the on-disk size is at worst roughly the same (e.g., the values of persistent attributes like _p_changed and _p_state don't get stored to disk, they only exist while the persistent object is in RAM). If I were you, I'd spend some quality time with fsdump, and figure out where the bulk of your space is going. Details matter more than general principles here. If you use the fsdump.py from ZODB 3.4 (which can be used with .fs files created by ZODB 3.1 and 3.2 too), it displays the byte size of data records, which can be a real help in such analysis. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope(ish) Windows services vs shutdown
[Tim Peters] [...] So, best guesses (please scream where I'm wrong): - This is because service.py doesn't define a SvcShutdown method, just a SvcStop method, - It's a good idea to add a SvcShutdown method to service.py. - It would suffice to add SvcShutdown = SvcStop to service.py. | If nobody disagrees (or even if everyone disagrees except Mark wink), I'll add that to the various Zope branches. [Mark Hammond] Thanks Tim - I'd never noticed that omission. You are completely correct. I'll test this next time I need to reboot (which contrary to popular opinion isn't that often wink). Cool! I can testify that adding SvcShutdown to ZRS's service subclass (just invoking the base class SvcStop) did cure ZRS's Windows shutdown oddities and didn't appear to create any new problems, so I'm much more confident now. I'll fiddle the various Zopes' base service classes instead tomorrow. Thanks for the brain cells! ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
