[Zope-dev] HEADS UP: QA Problems with Zope 2
Hi, as I'm the guy who clicks the build button for the Zope 2 Windows releases I have the unfortunate honor to write his complaining mail: Situation = a) Zope 2.8 as of release a2 does not run bin/test.py anymore. Neither on Linux nor on Windows. Unfortunately this is the only testrunner worked on Windows at all. Now it is broken as well. (Import problems from Five.products or something) b) The WinBuilder environment isn't version managed along the Zope branches. The change of the Zope module to Zope2 required manual changes in the WinBuilder structure that broke and didn't got detected during continous tests. (See c) Additionally the requirements of specific versions Python, ZODB, winutils etc. are hardcoded in the Makefiles right now. c) There are no nightly tests for Windows at all. I know there is some suspended effort already around. Please notice this as a friendly reminder that I'd be very lucky having those catch errors *before* Andreas makes a release and I run the tests manually. d) zopectl on windows doesn't work (it barfs about SIG_CHILD not existing). I'm not sure if it ever did or if it should. If it should not work on windows anyway, maybe we should remove it in the windows distribution ... All in all, it's a very unsatisfying situation for the Windows users, and I'm pretty scared telling anybody about the situation when I'm asked. This really has to change at least a bit. What to do == Things I can do and propose to do to make this better: -- b): I can either create branches for WinBuilders responding to the Zope Versions. Or (what I like better) I can put the WinBuilders somewhere in the Zope 2 tree to allow versioning along a branch automatically so continuous tests know where to get the WinBuilders from. c): If there is some infrastructure with the build bot around already and someone gives me pointers, I can set up nightly builds and tests on a Win2k machine over here. Additionally the tests should run on XP Server (or professional) and Win 2003 server as well. I don't have those around unfortunately + I don't have enough Visual Studio licenses around. Anyway, having at least one continuously running test is better than having none. Things I can't do anything right now on my own and need support: a) I'm not able to look into the test runners (that are not broken solely on windows) nor to look into Five integration problems. I'd love if someone could a) either fix test.py or b) deprecate it and give me the hint to some other runner that works. test.py is included and should therefore work within a release or be removed. I'm not sure what is right here. d) I have no idea about zopectl on windows. Is there some knowledge around on this? Final conclusion I have a compiled Zope 2.8a2 around here, but I'm not able to run the unit tests on it and I'm not willing to publish it therefore because I have the suspicion that this branch never has seen windows before except the one pass of unit tests before 2.8a1. Cheers, Christian -- gocept gmbh co. kg - schalaunische str. 6 - 06366 koethen - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 3496 30 99 112 - fax +49 3496 30 99 118 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: HEADS UP: QA Problems with Zope 2
Christian Theune wrote: a) I'm not able to look into the test runners (that are not broken solely on windows) nor to look into Five integration problems. I'd love if someone could a) either fix test.py or b) deprecate it and give me the hint to some other runner that works. test.py is included and should therefore work within a release or be removed. I'm not sure what is right here. I just added some missing files to setup.py. Could you please try again? There are still some broken z3 tests (setup works different for z3), but at least on linux test.py works now. Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] HEADS UP: QA Problems with Zope 2
b): I can either create branches for WinBuilders responding to the Zope Versions. Or (what I like better) I can put the WinBuilders somewhere in the Zope 2 tree to allow versioning along a branch automatically so continuous tests know where to get the WinBuilders from. +1 from me on merging WinBuilders into Zope. It took me quite some time to locate it first I looked, and having it external reinforces the 2nd-class status of Windows. d) I have no idea about zopectl on windows. Is there some knowledge around on this? I recall a message from Tim saying it has never worked and probably never will. IIUC correctly, its functionality isn't as desired on Windows due to the service support. Enfold has a simple log rotation strategy I have detailed in a previous mail to this list, and we will contribute code shortly. I have on my todo list for the next week (or 2) to steer through my service changes on both the 2.7 and 2.8 brances as discussed here recently. If that goes well, the Zope3 trunk will get a look-in too :) Printing-the-contrib-form-now ly, Mark ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] HEADS UP: QA Problems with Zope 2
Hi, Am Montag, den 04.04.2005, 21:12 +1000 schrieb Mark Hammond: d) I have no idea about zopectl on windows. Is there some knowledge around on this? I recall a message from Tim saying it has never worked and probably never will. IIUC correctly, its functionality isn't as desired on Windows due to the service support. Enfold has a simple log rotation strategy I have detailed in a previous mail to this list, and we will contribute code shortly. Yeah, I just wanted to try it for running 'zopectl test' but it fails earlier. Maybe it shouldn't be delivered then. I have on my todo list for the next week (or 2) to steer through my service changes on both the 2.7 and 2.8 brances as discussed here recently. If that goes well, the Zope3 trunk will get a look-in too :) Printing-the-contrib-form-now ly, Great to see some reinforcements coming in! Christian -- gocept gmbh co. kg - schalaunische str. 6 - 06366 koethen - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 3496 30 99 112 - fax +49 3496 30 99 118 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] post publishing hook
Am Montag, den 04.04.2005, 13:39 +0200 schrieb Florent Guillaume: Yes we need it for CPS which currently works with 2.7 (no customer is ready to move to 2.8 yet). Neither is 2.8. ;) -- gocept gmbh co. kg - schalaunische str. 6 - 06366 koethen - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 3496 30 99 112 - fax +49 3496 30 99 118 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] opinion: speeding up large PUT uploads
Chris McDonough [EMAIL PROTECTED] wrote: Zope's ZPublisher.HTTPRequest.HTTPRequest class has a method named processInputs. This method is responsible for parsing the body of all requests. It parses all upload bodies regardless of method: PUT, POST, GET, HEAD, etc. In doing so, it uses Python's FieldStorage module to potentially break apart multipart/* bodies into their respective parts. Every invocation of FieldStorage creates a tempfile that is a copy of the entire upload body. So in the common case, when a large file is uploaded via HTTP PUT (both DAV and external editor use PUT exclusively), here's what happens: - ZServer creates a tempfile T1 to hold the file body as it gets pulled in. - When the request makes it to the publisher, processInputs is called and it hands off tempfile T1 to FieldStorage. - FieldStorage reads the entire body and creates another tempfile T2 (an exact copy of T1*, in the case of a PUT request). - T2 is eventually put into REQUEST['BODYFILE']. (*) At least I can't imagine a case where it's not an exact copy. This is costly on large uploads. I'd like to change the top of the processInputs method to do this: if method == 'PUT': # we don't need to do any real input processing if we are # handling a PUT request. self._file = self.stdin return Can anyone think of a reason I shouldn't do this? Is stdin the medusa stream or T1 at this point ? Because for ConflictError retry we need an input that is seekable (HTTPRequest.retry does self.stdin.seek(0)). Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of RD +33 1 40 33 71 59 http://nuxeo.com [EMAIL PROTECTED] ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: brain.getObject and traversal
Hi Tres, We really need to follow a deprecation-style model here: the risk of breaking major third party components is pretty high. Agreed. I see you started working on this, thanks! Since this is a bug, and it looks like it's going to be fixed with a config option, would anyone mind if I ported this code to the 2.7 branch with the option set to do whatever 2.7.5 does? The CHANGELOG should highlight the change, and include the zope.conf snippet required to restore the old behavior. We could add a deprecation warning (if that entry is activated), that the old-style option would be removed in 2.10. Sounds good to me. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] brain.getObject and traversal
Florent Guillaume wrote: Is everyone ok with returning - the object if it can be accessed - raise Unauthorized if it can't be accessed - raise NotFound if it's not there Please don't catch any exceptions and re-raise them in a different type, just let them pass through. I specifically don't think raising a normal NotFound when the catalog has a brain that doesn't map to an object is the right thing to do. I'd much prefer a BrainHasNoMatchingObject exception or some such which is nice and clear... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: brain.getObject and traversal
Chris Withers wrote: Florent Guillaume wrote: Ok, thanks a lot to Tres for having gone ahead and done that. I just merged his branch. All 5645 tests pass (man, with Zope 3 included that's way more than before!) Did you check with Tres that his branc hwas ready to merge? ;-) Yes. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of RD +33 1 40 33 71 59 http://nuxeo.com [EMAIL PROTECTED] ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] HEADS UP: QA Problems with Zope 2
[Christian Theune] ... I have a compiled Zope 2.8a2 around here, but I'm not able to run the unit tests on it and I'm not willing to publish it therefore because I have the suspicion that this branch never has seen windows before except the one pass of unit tests before 2.8a1. FYI, I usually run the Zope trunk tests once a day, on WinXP. None of this goes thru WinBuilders, though -- this is doing test.py -vv --all from the root of a trunk checkout immediately after setup.py build_ext -i. The same two tests have been broken there since last October, but all other tests pass: http://www.zope.org/Collectors/Zope/1728 As recorded in the bug that points at, for related reasons an installed Zope 2.8 is probably unusable on Windows (I haven't tried this since last October -- don't see a point so long as the 1728 bug remains): http://www.zope.org/Collectors/Zope/1507 ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: brain.getObject and traversal
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Withers wrote: We really need to follow a deprecation-style model here: the risk of breaking major third party components is pretty high. Agreed. I see you started working on this, thanks! Since this is a bug, and it looks like it's going to be fixed with a config option, would anyone mind if I ported this code to the 2.7 branch with the option set to do whatever 2.7.5 does? - -0. This change is not a bugfix -- this is a new feature, which changes the documented behavior of the catalog brains. It is really up to Andreas whether or not to accept such a change on the 2.7 line. The CHANGELOG should highlight the change, and include the zope.conf snippet required to restore the old behavior. We could add a deprecation warning (if that entry is activated), that the old-style option would be removed in 2.10. Tres. - -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCUU1UGqWXf00rNCgRAlG7AJ4w88icN4H4pw7/ZtDSV22RlR41OACgoU9R Ia2qEpT7DHGRKY7VbwYwxrk= =NqFr -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] HEADS UP: QA Problems with Zope 2
... [Christian Theune] d) I have no idea about zopectl on windows. Is there some knowledge around on this? [Mark Hammond] I recall a message from Tim saying it has never worked and probably never will. Well, everything that builds on zdaemon is Unix-specific -- the underlying machinery uses stuff like this freely and ubiquitously: + Unix domain sockets. + os.fork() + The signals signal.SIGTERM, signal.SIGHUP, signal.SIGINT, and signal.SIGCHLD, with Unix semantics. If nothing else, it's a wonderful demonstration of how core Python allows writing wholly platform-specific code. IIUC correctly, its functionality isn't as desired on Windows due to the service support. Unix-heads certainly want it anyway; Windows-heads aren't used to anything better than the Windows services API, so they don't even bring it up. Enfold has a simple log rotation strategy I have detailed in a previous mail to this list, and we will contribute code shortly. Check it in too! Nobody is volunteering to look at your patches, and they're important. I can't make time for it myself (would if I could). I have on my todo list for the next week (or 2) to steer through my service changes on both the 2.7 and 2.8 brances as discussed here recently. If that goes well, the Zope3 trunk will get a look-in too :) The Zope3 Windows installers to date are produced via setup.py bdist_wininst, and that's all. No Windows service support, no bundling of Python, no bundling of win32all, ..., the Zope3 Windows story really has nothing in common with Zope2's so far. That's a discussion for zope3-dev, though. Printing-the-contrib-form-now ly, Thank you. Poor Mark 0.5 wink. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: HEADS UP: QA Problems with Zope 2
Tim Peters wrote: [Christian Theune] ... I have a compiled Zope 2.8a2 around here, but I'm not able to run the unit tests on it and I'm not willing to publish it therefore because I have the suspicion that this branch never has seen windows before except the one pass of unit tests before 2.8a1. FYI, I usually run the Zope trunk tests once a day, on WinXP. None of this goes thru WinBuilders, though -- this is doing test.py -vv --all from the root of a trunk checkout immediately after setup.py build_ext -i. AFAICS the problem is that *everybody* runs the tests in-place. So nobody tests if setup.py installs Zope correctly and people often forget to make sure setup.py installs newly added packages and files. Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] HEADS UP: QA Problems with Zope 2
On Apr 4, 2005 4:53 PM, Tim Peters [EMAIL PROTECTED] wrote: Unix-heads certainly want it anyway; Windows-heads aren't used to anything better than the Windows services API, so they don't even bring it up. Well, it's not so much as API, but the fact that Windows people are used to starting a service through the control panel, and not through a python-script. After all, you don't need the python script. ;) Possibly zopectl could start the service if it's called on a windows machine, instead of trying to pretend that it runs on unix... I don't know how hard that would be. Also, I don't know how hard it would be to get rid of the unix specific things for running zopectl test and such... -- Lennart Regebro, Nuxeo http://www.nuxeo.com/ CPS Content Management http://www.cps-project.org/ ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] post publishing hook
Jim Fulton wrote: Florent Guillaume wrote: I really could use a post publishing hook. Standard use case: delay indexing at the end of the request to only do it once per object even if the object has been modified 4 times. Today there's the REQUEST._hold() hack with an instance having a __del__, but this gets executed outside the main transaction, and REQUEST is already dying. I'd like a post-publishing hook that's called in the initial REQUEST and transaction. I haven't been folowing this thread, so I asked Gary what it was about. :) Based on that, I'd like to suggest: There are two possibilities: 1. A post publishing hook. I think this would be appropriate in the case where you really want to augment the publishing process. For example, I hpe someday to use something like this to provide another way (other than metal) to provide standard look and feel. Unfortunately, I think there are a lot of open issues, at least in my mind, about how something like this should work. What I had in mind was, just after Publish.publish calls result = mapply(object, request.args, request, ...) add: if hasattr(request, 'runPostPublishingHooks'): result = request.runPostPublishingHooks(result, request, response) And a simple system for registering hooks. Florent -- Florent Guillaume, Nuxeo (Paris, France) CTO, Director of RD +33 1 40 33 71 59 http://nuxeo.com [EMAIL PROTECTED] ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope2.8a2] ...to be released by tomorrow....
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 yuppie wrote: Tres Seaver wrote: I still have a notion that some improvements made on the 2.7 branch are not merged into the SVN trunk, e.g. http://cvs.zope.org/Zope/lib/python/AccessControl/Attic/ZopeGuards.py.diff?r1=1.16.2.3r2=1.16.2.4 There is some debate here whether those changes (which Jim made only relunctantly back in January 2004) should be promulgated to 2.8. We will fold them in after the alpha, if so. The link above points to an other unmerged change. I don't understand the complete change, but at least removing Python 2.1 compatibility code should also be done on the trunk. Hmm, that change hadn't landed on the gmane.org version of the checkins. I have it merged in my sandbox now, along with two apparently related changes (to the 'actual_python' and 'testZopeGuards' modules in lib/python/AccessControl/tests). Evan, do you recall whether you made related changes outside of the AccessControl package (e.g, in PythonScripts, ZPT, etc.)? Tres. - -- === Tres Seaver[EMAIL PROTECTED] Zope Corporation Zope Dealers http://www.zope.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCUYMqGqWXf00rNCgRAjEsAJ4q8c9PHFq5vYrS2XKo5yEJ1/CfjQCfVGR7 8FN4IF6GuQ0Q83y1qJj317k= =a6ji -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Zope2.8a2] ...to be released by tomorrow....
Tres Seaver wrote: http://cvs.zope.org/Zope/lib/python/AccessControl/Attic/ZopeGuards.py.diff?r1=1.16.2.3r2=1.16.2.4 [...] Hmm, that change hadn't landed on the gmane.org version of the checkins. I have it merged in my sandbox now, along with two apparently related changes (to the 'actual_python' and 'testZopeGuards' modules in lib/python/AccessControl/tests). That seems to be the complete change, see http://cvs.zope.org/query?branch=Zope-2_7-branchwho=evansortby=datedate=all#results Evan, do you recall whether you made related changes outside of the AccessControl package (e.g, in PythonScripts, ZPT, etc.)? HTH, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] opinion: speeding up large PUT uploads
On Mon, 2005-04-04 at 14:27, Dieter Maurer wrote: Even a PUT may get a multipart entity. But it never actually does in practice. Or if it does, I've never seen it. And if it did, would an implementation just store the multipart-encoded body? I suppose it could do anything, but it seems like it could be rather general and useless to allow multipart PUT bodies especially given that no one has seemed to need it in the last six years. That's what POST is for. At least, the HTTP specification does not tell anything to the contrary. No, it doesn't. Otherwise, (working) optimizations are of course welcome... This one works. ;-) - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ZPT: defer expression fix
Christian Heimes wrote: That's an interessting use case. Do you want me to keep the code and make up a new expression? I'm thinking about lazy:. If you have a particular use for defer: that would justify the split, please go ahead. I have no particular interest in keeping it. Cheers, Evan @ 4-am ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
