[Zope-dev] Zope Tests: 5 OK
Summary of messages to the zope-tests list. Period Wed Jul 2 11:00:00 2008 UTC to Thu Jul 3 11:00:00 2008 UTC. There were 5 messages: 5 from Zope Tests. Tests passed OK --- Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Tests Date: Wed Jul 2 20:53:47 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009795.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Tests Date: Wed Jul 2 20:55:17 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009796.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Tests Date: Wed Jul 2 20:56:47 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009797.html Subject: OK : Zope-2.11 Python-2.4.4 : Linux From: Zope Tests Date: Wed Jul 2 20:58:17 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009798.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Tests Date: Wed Jul 2 20:59:47 EDT 2008 URL: http://mail.zope.org/pipermail/zope-tests/2008-July/009799.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] zc.testbrowser.real support for mozlab 0.1.9
Hi, I've checked in a branch with changes to the testbrowser.real code to make it work with mozlab 0.1.9 (and firefox 3). Could someone please review the code and inform me, if I can merge it with the trunk? The branch can be found at svn.zope.org/repos/main/zc.testbrowser/branches/sweh-mozlab0.1.9 Regards, Sebastian -- Sebastian Wehrmann · [EMAIL PROTECTED] gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 12 · fax +49 345 1229889 1 Zope and Plone consulting and development smime.p7s Description: S/MIME Cryptographic Signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zc.testbrowser.real support for mozlab 0.1.9
On Thu, Jul 3, 2008 at 8:15 AM, Sebastian Wehrmann [EMAIL PROTECTED] wrote: Hi, I've checked in a branch with changes to the testbrowser.real code to make it work with mozlab 0.1.9 (and firefox 3). Cool. Could someone please review the code and inform me, if I can merge it with the trunk? I'll review it in the next couple of days and get back with you. -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Test runner: layers, subprocesses, and tear down
I'm working on making the zope.testing test runner run tests in parallelized subprocesses. The option will likely be spelled -j N, where N is the maximum number of processes. I have it basically working, but have noticed a couple odd corners of the test runner that I'd like to clean up. They may be controversial, so I'll ask about them here first. I'd like to 1) remove the layer tear-down mechanism entirely, and 2) make (almost) all layers run in a subprocess. I want to do #1 because it would simplify the test runner code and no one seems to be using the functionality anyway. It also appears from reading the code that any tests run in a subprocess (and most are) will never exercise the tear-down mechanism anyway. #2 will add some process start-up overhead, but it'll only be one more process than is already started (and any reasonably-sized test corpus already starts several processes for each test run). The one exception is for running with -D or with a pdb.set_trace() embedded in the code under test. For that case we need a switch to say don't start any subprocesses at all, I suspect that will be spelled -j0. For motivation, some speed comparisons: running a particular test suite with 3876 tests (mostly doctests, and mostly functional) without the patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3 minutes and 22 seconds (give or take) on a dual-core box with 3 simultaneous subprocesses. -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test runner: layers, subprocesses, and tear down
On Thu, 2008-07-03 at 17:22 -0400, Benji York wrote: I'm working on making the zope.testing test runner run tests in parallelized subprocesses. The option will likely be spelled -j N, where N is the maximum number of processes. I have it basically working, but have noticed a couple odd corners of the test runner that I'd like to clean up. They may be controversial, so I'll ask about them here first. I'd like to 1) remove the layer tear-down mechanism entirely, and 2) make (almost) all layers run in a subprocess. I want to do #1 because it would simplify the test runner code and no one seems to be using the functionality anyway. It also appears from reading the code that any tests run in a subprocess (and most are) will never exercise the tear-down mechanism anyway. +1 in general but -1 on removing the tear down functionality. We use it to destroy external databases that where generated for setup. #2 will add some process start-up overhead, but it'll only be one more process than is already started (and any reasonably-sized test corpus already starts several processes for each test run). The one exception is for running with -D or with a pdb.set_trace() embedded in the code under test. For that case we need a switch to say don't start any subprocesses at all, I suspect that will be spelled -j0. +1 as well. I'm actually wondering whether we might be able to control the pdb through a sub-process. For motivation, some speed comparisons: running a particular test suite with 3876 tests (mostly doctests, and mostly functional) without the patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3 minutes and 22 seconds (give or take) on a dual-core box with 3 simultaneous subprocesses. Yay! -- Christian Theune · [EMAIL PROTECTED] gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development signature.asc Description: This is a digitally signed message part ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
AW: [Zope-dev] Test runner: layers, subprocesses, and tear down
Hi Benji Betreff: [Zope-dev] Test runner: layers, subprocesses, and tear down [... ] #2 will add some process start-up overhead, but it'll only be one more process than is already started (and any reasonably-sized test corpus already starts several processes for each test run). The one exception is for running with -D or with a pdb.set_trace() embedded in the code under test. For that case we need a switch to say don't start any subprocesses at all, I suspect that will be spelled -j0. That's a very important point. I often use pdb if I write tests. For motivation, some speed comparisons: running a particular test suite with 3876 tests (mostly doctests, and mostly functional) without the patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3 minutes and 22 seconds (give or take) on a dual-core box with 3 simultaneous subprocesses. Yeah, great! Regards Roger Ineichen Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test runner: layers, subprocesses, and tear down
On Thu, Jul 3, 2008 at 5:37 PM, Christian Theune [EMAIL PROTECTED] wrote: On Thu, 2008-07-03 at 17:22 -0400, Benji York wrote: I'd like to 1) remove the layer tear-down mechanism entirely, and 2) make (almost) all layers run in a subprocess. I want to do #1 because it would simplify the test runner code and no one seems to be using the functionality anyway. It also appears from reading the code that any tests run in a subprocess (and most are) will never exercise the tear-down mechanism anyway. +1 in general but -1 on removing the tear down functionality. We use it to destroy external databases that where generated for setup. Ah! Good point. #2 will add some process start-up overhead, but it'll only be one more process than is already started (and any reasonably-sized test corpus already starts several processes for each test run). The one exception is for running with -D or with a pdb.set_trace() embedded in the code under test. For that case we need a switch to say don't start any subprocesses at all, I suspect that will be spelled -j0. +1 as well. I'm actually wondering whether we might be able to control the pdb through a sub-process. I don't think it'd be that hard, in general, but the current design of using stdout and stderr for IPC communication channels is a hindrance. For motivation, some speed comparisons: running a particular test suite with 3876 tests (mostly doctests, and mostly functional) without the patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3 minutes and 22 seconds (give or take) on a dual-core box with 3 simultaneous subprocesses. Yay! I have an 8 core machine that I can't wait to try it on. ;) -- Benji York Senior Software Engineer Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test runner: layers, subprocesses, and tear down
On Thu, Jul 03, 2008 at 05:22:11PM -0400, Benji York wrote: I'm working on making the zope.testing test runner run tests in parallelized subprocesses. The option will likely be spelled -j N, where N is the maximum number of processes. That's wonderful news! I have it basically working, but have noticed a couple odd corners of the test runner that I'd like to clean up. They may be controversial, so I'll ask about them here first. I'd like to 1) remove the layer tear-down mechanism entirely, and 2) make (almost) all layers run in a subprocess. -1 in general. +1 if you do that only for -j N where N 1. Running all the tests in a single process has the following benefits: * test coverage analysis produces results that are correct (well, correct often enough -- but it has no chance at all when the test runner forks a subprocess) * import pdb; pdb.set_trace() works I want to do #1 because it would simplify the test runner code and no one seems to be using the functionality anyway. That's news to me. A while ago I went through Zope 3 trunk (it was pre-eggsplosion IIRC) and made sure all test layers defined in it supported teardown. Granted, FunctionalTestLayer() has allow_teardown=False as the default, for two reasons: * backwards compatibility: in olden days functional test layers didn't support teardown * paranoia: it is in general impossible to determine whether calling CleanUp().cleanUp() will correctly clear all the global state (someone could easily write a custom ZCML directive that changed a global variable and forget to register a CleanUp hook), so disallowing teardowns were the conservative safe choice. It is entirely my fault that I haven't evangelized the allow_teardown=True option for creating new test layers. It also appears from reading the code that any tests run in a subprocess (and most are) will never exercise the tear-down mechanism anyway. I guess that's fine for process state, but not so fine for external state (temporary files etc.). Hey, this might explain why SchoolTool's tests tend to fill up my buildbot's /tmp without cleaning up after themselves! I'll have to investigate some day. #2 will add some process start-up overhead, but it'll only be one more process than is already started (and any reasonably-sized test corpus already starts several processes for each test run). The one exception is for running with -D or with a pdb.set_trace() embedded in the code under test. For that case we need a switch to say don't start any subprocesses at all, I suspect that will be spelled -j0. If that case needs to be supported anyway, what's the advantage of spawning exactly one subprocess when you run it with -j 1? I would also question whether pdb-unfriendly non-performance-enhancing option should be the default. For motivation, some speed comparisons: running a particular test suite with 3876 tests (mostly doctests, and mostly functional) without the patch takes 6 minutes, 42 seconds; my branch runs the same tests in 3 minutes and 22 seconds (give or take) on a dual-core box with 3 simultaneous subprocesses. I know; for large test suites (by large I mean 40 minutes) I've been using an ugly hack (--odd/--even test filtering) that lets me use both CPUs if I manually run two instances of the test runner in two xterms in parallel. Regards, Marius Gedminas -- Wipe Info uses hexadecimal values to wipe files. This provides more security than wiping with decimal values. -- Norton SystemWorks 2002 Manual signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] TALES iterator odd/even reversal
Recently I migrated a large-ish app built on Zope 3.2 to Zope 3.4. (About time I hear someone mumbling in the audience.) One strange difference was that TALES iterators swapped the meaning of odd and even, i.e. p tal:repeat=item python: [1, 2, 3, 4] span tal:condition=repeat/var/oddoddspan span tal:condition=repeat/var/evenevenspan /p produces different results on Zope 3.4 than it did on 3.2. Does anyone know why this is? Is this a bug? Should it be fixed? Marius Gedminas -- Perl is hard for most people to write. They write PERL or Pearl. -- Abigail signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Itemtraverser and Unauthorized vs Views
On Tue, Jun 24, 2008 at 01:39:28PM +0200, Christian Theune wrote: I have a problem with the standard item traverser provided by zope.app.container: The item traverser looks up a object using the given name and a __getitem__ call on the context. If this raises a KeyError it tries to look up a view given the same name. If the user does not have the permission to access __getitem__ it will let the Unauthorized exception pass through. I my situation I have two views for which the user doesn't really need the permission to access __getitem__ on the container but they can't access the views because the __getitem__ call will be tried anyway. I can explicitly make the URL use '@@viewname' and bypass the item traverser, but I don't like the @@s in the URL. I wonder whether adding Unauthorized to the KeyError would be reasonable. I think not. At least it should not convert Unauthorized into NotFound. If I can access a location (say, http://localhost/container/item) when I'm logged in, then if I try that as an anonymous user, I should get an authentication dialog rather than a 404 Not Found page. Marius Gedminas -- If nothing else helps, read the documentation. signature.asc Description: Digital signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zc.testbrowser.real support for mozlab 0.1.9
I've checked in a branch with changes to the testbrowser.real code to make it work with mozlab 0.1.9 (and firefox 3). Hi Sebastian, This is great. I spent all day yesterday trying to make this happen and didn't get anywhere. Sometimes I wake up with a better idea as to how to solve things, but I don't often wake up to find all the work's been done! I tried running the tests with Python 2.4. About half the time they pass fine, and the other half fail something like this: File doctest README.txt[40], line 1, in ? browser.open('navigate.html') File /Users/graham/development/patched_zc.testbrowser/src/zc/ testbrowser/real.py, line 201, in open self.wait() File /Users/graham/development/patched_zc.testbrowser/src/zc/ testbrowser/real.py, line 193, in wait assert self.execute('tb_page_loaded;') == 'false' AssertionError though for varying pages with browser.open(). I don't understand what could be causing it; why would that value be changed in between those two lines? I guess the time.sleep() on line 191 implies that there's something we might want to wait for. I increased it by 10x and only got 1 failure out of 10, so I guess that helps. Thanks, Graham ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Test runner: layers, subprocesses, and tear down
Benji York wrote: I'm working on making the zope.testing test runner run tests in parallelized subprocesses. I have some recent experience parallelising (and distributing across machines) test runs. This was in Java, with TestNG and Selenium, but we learned some interesting things. We basically cut a 45 minute test run to 10 minutes by distributing the tests across three machines, each running a full stack (Oracle, JBoss, Firefox) and Selenium Grid. I realise you're not trying to do anything quite as complex as that, but a parallel test runner ought to be extensible to support distribution across nodes in a grid. The main challenge there is to distribute deployment of the code to run, and to sync test setup so that all environments are identical. I suspect you'll find this out of scope to begin with, but I'd keep it in the back of your mind. You will likely need some way of declaring tests that have to run in series. Sometimes that's just for sanity's sake, other times it's a requirement due to shared resources. A nice way to do this is to make it possible to annotate tests to group them, and then to be able to declaratively configure some groups as serial. Any functional test that uses a shared external resource will require this. TestNG supports (as far as I recall): - Run all tests (methods) randomly and parallelise - Run groups of tests (classes or declaratively specified named groups) in parallel, but run tests within the groups sequentially - Run all tests in series (i.e. single-threaded) We should probably use test layers as the main grouping mechanism here. If you could declare a layer as can be run in parallel with other layers or tests in this layer run in series, that'd be pretty powerful. I'm not 100% sure how this works with layers that derive from one another, and where you'd have two layers with a shared base class, though. Parallisation can offer huge (!) speed increases, but it can also be hard to debug tests. I'd be tempted to let single threaded by the default, safe choice, and let people opt into parallisation only when they know what they are doing. Most test runs are quite quick anyway. Test result reporting can be difficult. You'll probably need to collect all failures with tracebacks and report at the end. For long running test suites, this may not be ideal, since it's helpful to get early warning, so if you can find a way to get test output to be atomically output, then that'd be nice. Debugging stuff that happens in parallel with pdb is also tricky. It must be easy to turn off parallel running and to run individual tests in a single process for each debugging. To make this work with Selenium grid, we ended up building some infrastructure to manage environments (i.e. an allocation of database, web server and so on), and locks on those environments. We'd spawn one thread for each environment and feed tests to those threads as fast as they could run them. Each test run then grabbed an environment on setup, executed, and then released the lock for another test. Oh, and please don't get rid of any tear-down. You'll definitely need it one day. Letting environments go dirty is generally troublesome, and gets only more difficult when you may have multiple threads trying to use those environments at once. I don't know how you've structured this, but I'd consider whether one layer could be shared across multiple threads/subprocesses, or if it's always a one-to-one thing. I realise this is somewhat rambling, but I hope it's useful in any case. :) Cheers, Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Itemtraverser and Unauthorized vs Views
On Fri, 2008-07-04 at 02:10 +0300, Marius Gedminas wrote: On Tue, Jun 24, 2008 at 01:39:28PM +0200, Christian Theune wrote: [...] I can explicitly make the URL use '@@viewname' and bypass the item traverser, but I don't like the @@s in the URL. I wonder whether adding Unauthorized to the KeyError would be reasonable. I think not. At least it should not convert Unauthorized into NotFound. If I can access a location (say, http://localhost/container/item) when I'm logged in, then if I try that as an anonymous user, I should get an authentication dialog rather than a 404 Not Found page. Actually, in my case its, when logged in I can use: http://localhost/container/view When not logged in, I get an Unauthorized, although when accessing http://localhost/container/@@view I can go ahead as anonymous. IMHO the code merging the namespaces should be more careful about that. Christian -- Christian Theune · [EMAIL PROTECTED] gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development signature.asc Description: This is a digitally signed message part ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: RE: [Zope] ZcxOracleDA, LocalFS, ZMySQLDA
After implementing the Ingres adapter ZEIngresDA I found I was left high and dry when it came to deprecation messages and all else by ZEOmega. I had to fix it myself. - Original Message - From: Jeff Peterson [EMAIL PROTECTED] Date: Saturday, June 28, 2008 3:24 Subject: RE: [Zope] ZcxOracleDA, LocalFS, ZMySQLDA To: Andreas Jung [EMAIL PROTECTED], zope@zope.org zope@zope.org I'm not actually having a problem. I updated those three products to work with 2.11.0 and was wondering if someone could give some guidance to me regarding getting the changes made permanent or at least looked at by the authors. I emailed each of them but have had no responses. Given that the authors themselves haven't replied, it occurred to me that the email addresses may be out of date so I turned to the list. I have been using Zope for some years now, but have never really needed to submit a patch or anything like that, so the process to get this done is really an unknown for me. It's even possible I am using the incorrect products for this release of Zope. I noticed too, a minor issue with ZPsycopgDA, it also exists with ZMySQLDA: /home/zope/base211/lib/python/OFS/Application.py:809: DeprecationWarning: __init__.py of Products.ZMySQLDA has a long deprecated 'methods' attribute. 'methods' support might be removed in Zope 2.11 or a later feature release. Please use the 'legacy' argument of registerClass instead if the methods are constructors. Or refactor the product using adapters. DeprecationWarning) /home/zope/base211/lib/python/OFS/Application.py:809: DeprecationWarning: __init__.py of Products.ZPsycopgDA has a long deprecated 'methods' attribute. 'methods' support might be removed in Zope 2.11 or a later feature release. Please use the 'legacy' argument of registerClass instead if the methods are constructors. Or refactor the product using adapters. DeprecationWarning) I am not sure how to fix this, I see refactoring using adapters, but though I have fiddled a bit with Zope3 it's magic is still a relative unknown to me. I was just looking for a little direction. For the most part I have no issues with the migration except with Page Templates, it feels like the expand macros while editing is backwards. I have to uncheck it for the fill-slots and macros to show up properly and I am seeing some strange errors where a slot that is clearly defined in the master template is being rendered as metal:slot rather than metal:fill-slot. But I have just come across this and haven't spent much time looking yet, I am sure a solution will present itself. Thanks, -- Jeffrey D Peterson Webmaster Crary Industries, Inc. -Original Message- From: Andreas Jung [mailto:[EMAIL PROTECTED] Sent: Thursday, June 26, 2008 11:34 PM To: Jeff Peterson; zope@zope.org Subject: Re: [Zope] ZcxOracleDA, LocalFS, ZMySQLDA --On 26. Juni 2008 17:03:27 -0500 Jeff Peterson [EMAIL PROTECTED]wrote: I upgraded my Zope server to 2.11.0 from 2.8.7. To do that I needed to update the above 3 products. I am not sure of the process to get the products updated so others can use them. What is your problem? Like in our old installation: you donwload the products and install them in the same way as you did earlier. Of course you have to check if those products are compatible with most recent Zope versions (check yourself by installing them, if necessary get back to the authors of the products). -aj ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Annoying when objectValues() doesn't work
from cStringIO import StringIO def workflowscripts2methods(self): out = StringIO() print out, Star converting workflow scripts into External Methods pw = self.portal_workflow for workflow in pw.objectValues('Workflow'): for container in workflow.objectValues('Workflow Scripts'): meta_type = 'Script (Python)' for script in container.objectValues(meta_type): assert script.meta_type == meta_type print out, script (%r, %s) % (script.meta_type, script.absolute_url_path()) value = out.getvalue() print value return value I'm getting an AssertionError on this!!! After having used objectValues() for so many many years without much trouble my confidence in it just dropped. Anybody else seen something like this? -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
--On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: I'm getting an AssertionError on this!!! After having used objectValues() for so many many years without much trouble my confidence in it just dropped. After years of Zope, you know that we need a full traceback :-) Andreas pgplCurBNuW8z.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
2008/7/3 Andreas Jung [EMAIL PROTECTED]: --On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: I'm getting an AssertionError on this!!! After having used objectValues() for so many many years without much trouble my confidence in it just dropped. After years of Zope, you know that we need a full traceback :-) The traceback will just say that the AssertionError happened. It doesn't mention anything interesting. Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Products.ExternalMethod.ExternalMethod, line 225, in __call__ - __traceback_info__: ((CMFSite instance at 9b924a0,), {}, None) Module /home/peterbe/virtualenvzope/Gosh/zope278/Products/SeamusCore/Extensions/wf_script_despatcher.py, line 41, in workflowscripts2methods AssertionError -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
+---[ Peter Bengtsson ]-- | 2008/7/3 Andreas Jung [EMAIL PROTECTED]: | | | --On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: | | I'm getting an AssertionError on this!!! | After having used objectValues() for so many many years without much | trouble my confidence in it just dropped. | | After years of Zope, you know that we need a full traceback :-) | | | The traceback will just say that the AssertionError happened. It | doesn't mention anything interesting. | | Traceback (innermost last): | Module ZPublisher.Publish, line 101, in publish | Module ZPublisher.mapply, line 88, in mapply | Module ZPublisher.Publish, line 39, in call_object | Module Products.ExternalMethod.ExternalMethod, line 225, in __call__ |- __traceback_info__: ((CMFSite instance at 9b924a0,), {}, None) | Module /home/peterbe/virtualenvzope/Gosh/zope278/Products/SeamusCore/Extensions/wf_script_despatcher.py, | line 41, in workflowscripts2methods | AssertionError It does tell you the script and the line of the External Method that is causing the problem when being instanced via _getOb() (from objectValues) -- Andrew Milton [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
2008/7/3 Andrew Milton [EMAIL PROTECTED]: +---[ Peter Bengtsson ]-- | 2008/7/3 Andreas Jung [EMAIL PROTECTED]: | | | --On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: | | I'm getting an AssertionError on this!!! | After having used objectValues() for so many many years without much | trouble my confidence in it just dropped. | | After years of Zope, you know that we need a full traceback :-) | | | The traceback will just say that the AssertionError happened. It | doesn't mention anything interesting. | | Traceback (innermost last): | Module ZPublisher.Publish, line 101, in publish | Module ZPublisher.mapply, line 88, in mapply | Module ZPublisher.Publish, line 39, in call_object | Module Products.ExternalMethod.ExternalMethod, line 225, in __call__ |- __traceback_info__: ((CMFSite instance at 9b924a0,), {}, None) | Module /home/peterbe/virtualenvzope/Gosh/zope278/Products/SeamusCore/Extensions/wf_script_despatcher.py, | line 41, in workflowscripts2methods | AssertionError It does tell you the script and the line of the External Method that is causing the problem when being instanced via _getOb() (from objectValues) I know. That was the script I showed and the code only has one obvious assert statement. That was my code the assert statement. The point is, it shouldn't happen, ever. -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
--On 3. Juli 2008 16:01:53 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: 2008/7/3 Andreas Jung [EMAIL PROTECTED]: --On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: I'm getting an AssertionError on this!!! After having used objectValues() for so many many years without much trouble my confidence in it just dropped. After years of Zope, you know that we need a full traceback :-) The traceback will just say that the AssertionError happened. It doesn't mention anything interesting. Traceback (innermost last): Module ZPublisher.Publish, line 101, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 39, in call_object Module Products.ExternalMethod.ExternalMethod, line 225, in __call__ - __traceback_info__: ((CMFSite instance at 9b924a0,), {}, None) Module /home/peterbe/virtualenvzope/Gosh/zope278/Products/SeamusCore/Extensions/ wf_script_despatcher.py, line 41, in workflowscripts2methods AssertionError Peter, you know that you can look at the code and check what's going on :-) Andreas pgpyua5i8WtfV.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
+---[ Peter Bengtsson ]-- | 2008/7/3 Andrew Milton [EMAIL PROTECTED]: | +---[ Peter Bengtsson ]-- | | 2008/7/3 Andreas Jung [EMAIL PROTECTED]: | | | | | | --On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: | | | | I'm getting an AssertionError on this!!! | | After having used objectValues() for so many many years without much | | trouble my confidence in it just dropped. | | | | After years of Zope, you know that we need a full traceback :-) | | | | | | The traceback will just say that the AssertionError happened. It | | doesn't mention anything interesting. | | | | Traceback (innermost last): | | Module ZPublisher.Publish, line 101, in publish | | Module ZPublisher.mapply, line 88, in mapply | | Module ZPublisher.Publish, line 39, in call_object | | Module Products.ExternalMethod.ExternalMethod, line 225, in __call__ | |- __traceback_info__: ((CMFSite instance at 9b924a0,), {}, None) | | Module /home/peterbe/virtualenvzope/Gosh/zope278/Products/SeamusCore/Extensions/wf_script_despatcher.py, | | line 41, in workflowscripts2methods | | AssertionError | | It does tell you the script and the line of the External Method that is | causing the problem when being instanced via _getOb() (from | objectValues) | | I know. That was the script I showed and the code only has one obvious | assert statement. That was my code the assert statement. The point is, | it shouldn't happen, ever. Except CMFSite is calling it according to your traceback.. -- Andrew Milton [EMAIL PROTECTED] ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Annoying when objectValues() doesn't work
2008/7/3 Andrew Milton [EMAIL PROTECTED]: +---[ Peter Bengtsson ]-- | 2008/7/3 Andrew Milton [EMAIL PROTECTED]: | +---[ Peter Bengtsson ]-- | | 2008/7/3 Andreas Jung [EMAIL PROTECTED]: | | | | | | --On 3. Juli 2008 15:41:52 +0100 Peter Bengtsson [EMAIL PROTECTED] wrote: | | | | I'm getting an AssertionError on this!!! | | After having used objectValues() for so many many years without much | | trouble my confidence in it just dropped. | | | | After years of Zope, you know that we need a full traceback :-) | | | | | | The traceback will just say that the AssertionError happened. It | | doesn't mention anything interesting. | | | | Traceback (innermost last): | | Module ZPublisher.Publish, line 101, in publish | | Module ZPublisher.mapply, line 88, in mapply | | Module ZPublisher.Publish, line 39, in call_object | | Module Products.ExternalMethod.ExternalMethod, line 225, in __call__ | |- __traceback_info__: ((CMFSite instance at 9b924a0,), {}, None) | | Module /home/peterbe/virtualenvzope/Gosh/zope278/Products/SeamusCore/Extensions/wf_script_despatcher.py, | | line 41, in workflowscripts2methods | | AssertionError | | It does tell you the script and the line of the External Method that is | causing the problem when being instanced via _getOb() (from | objectValues) | | I know. That was the script I showed and the code only has one obvious | assert statement. That was my code the assert statement. The point is, | it shouldn't happen, ever. Except CMFSite is calling it according to your traceback.. CMFSite is the type of the container that holds the External Method that runs that stuff. -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zope stops responding
Following a hard disk crash 2 weeks ago we have installed Zope onto a new server and all was fine until yesterday morning when Zope stopped responding and required a restart to get it working. It did the same at 3-30pm today. We are using Zope 2.9.0 on Centos 5.1 on a quad Zeon server with 4gb. Ram. At the moment the server is in quite heavy use with teachers trying to write reports for a deadline tomorrow. These are sent to a Postgres DB via psycopg. There is nothing in Z2.log or event.log to point me to the problem. I am therefore asking for advice on the sorts of things that can cause Zope to stop responding and whether there is anything we can do to mitigate against such an event. Thanks and regards Garry ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] zope stops responding
Have you tried looking at it with the deadlock debugger? That should give you an idea of what its actually running (and if the deadlock debugger is usually responsive even when zope isn't and if it isn't that's also information). Kevin -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Garry Saddington Sent: Thursday, July 03, 2008 9:00 AM To: zope list user Subject: [Zope] zope stops responding Following a hard disk crash 2 weeks ago we have installed Zope onto a new server and all was fine until yesterday morning when Zope stopped responding and required a restart to get it working. It did the same at 3-30pm today. We are using Zope 2.9.0 on Centos 5.1 on a quad Zeon server with 4gb. Ram. At the moment the server is in quite heavy use with teachers trying to write reports for a deadline tomorrow. These are sent to a Postgres DB via psycopg. There is nothing in Z2.log or event.log to point me to the problem. I am therefore asking for advice on the sorts of things that can cause Zope to stop responding and whether there is anything we can do to mitigate against such an event. Thanks and regards Garry ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zope stops responding
Yes, couldn't agree more with kevin. DeadlockDebugger is crucial. It won't change anything other than that you'll have information about what really needs to change. As an intermediate solution you can start to increase the number of Zope threads from 4 to 6 or 8 and see if that kills the CPU of your beefy server. Too many threads can hurt performance especially when working with badly coded external connectors that selfishly hook itself to the python GIL too much. A lightweight alternative to DeadlockDebugger is the table at the bottom of the Debug Information page in Control_Panel. If any of the dates in the first column in that table shows times longer than a second you have a problem with those requests. Peter 2008/7/3 Garry Saddington [EMAIL PROTECTED]: Following a hard disk crash 2 weeks ago we have installed Zope onto a new server and all was fine until yesterday morning when Zope stopped responding and required a restart to get it working. It did the same at 3-30pm today. We are using Zope 2.9.0 on Centos 5.1 on a quad Zeon server with 4gb. Ram. At the moment the server is in quite heavy use with teachers trying to write reports for a deadline tomorrow. These are sent to a Postgres DB via psycopg. There is nothing in Z2.log or event.log to point me to the problem. I am therefore asking for advice on the sorts of things that can cause Zope to stop responding and whether there is anything we can do to mitigate against such an event. Thanks and regards Garry ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zope stops responding
I was same problems with new IBM hardware 2 yrs ago and etc at past. Wait, please, your HW will be good after 2..3 months - it needs burning process. It is known problem with very fresh silicon chips about diffusion of non-homogenities - look at mfg date at asphalt squares. But my english is not as good to detailed explain whats happen. - Original Message - From: Garry Saddington [EMAIL PROTECTED] Following a hard disk crash 2 weeks ago we have installed Zope onto a new server and all was fine until yesterday morning when Zope stopped responding and required a restart to get it working. It did the same at 3-30pm today. We are using Zope 2.9.0 on Centos 5.1 on a quad Zeon server with 4gb. Ram. At the moment the server is in quite heavy use with teachers trying to write reports for a deadline tomorrow. These are sent to a Postgres DB via psycopg. There is nothing in Z2.log or event.log to point me to the problem. I am therefore asking for advice on the sorts of things that can cause Zope to stop responding and whether there is anything we can do to mitigate against such an event. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] zope stops responding
My experience is that increasing the number of threads past 4 does absolutely nothing to assist performance. Multi threads will help with the occasional long running request, but if its more than occasional then the thread list will eventually clog up no matter how many threads you have (and the more threads you have, the more they fight over the processor). The server sounds beefy, but since Zope is only going to be using one of those four cores, that might be deceptive. Kevin -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Peter Bengtsson Sent: Thursday, July 03, 2008 9:19 AM To: Garry Saddington Cc: zope list user Subject: Re: [Zope] zope stops responding Yes, couldn't agree more with kevin. DeadlockDebugger is crucial. It won't change anything other than that you'll have information about what really needs to change. As an intermediate solution you can start to increase the number of Zope threads from 4 to 6 or 8 and see if that kills the CPU of your beefy server. Too many threads can hurt performance especially when working with badly coded external connectors that selfishly hook itself to the python GIL too much. A lightweight alternative to DeadlockDebugger is the table at the bottom of the Debug Information page in Control_Panel. If any of the dates in the first column in that table shows times longer than a second you have a problem with those requests. Peter 2008/7/3 Garry Saddington [EMAIL PROTECTED]: Following a hard disk crash 2 weeks ago we have installed Zope onto a new server and all was fine until yesterday morning when Zope stopped responding and required a restart to get it working. It did the same at 3-30pm today. We are using Zope 2.9.0 on Centos 5.1 on a quad Zeon server with 4gb. Ram. At the moment the server is in quite heavy use with teachers trying to write reports for a deadline tomorrow. These are sent to a Postgres DB via psycopg. There is nothing in Z2.log or event.log to point me to the problem. I am therefore asking for advice on the sorts of things that can cause Zope to stop responding and whether there is anything we can do to mitigate against such an event. Thanks and regards Garry ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zope stops responding
Hi, I does not agree with You, because my servers runs with 20..150 threads, because lower number of threads makes for somebody infinity waiting and Zope under load responding very slow (with external database access). After increasing threads, response under load is more quickly and more paralel users should access to the web. - Original Message - From: Sours, Kevin [EMAIL PROTECTED] My experience is that increasing the number of threads past 4 does absolutely nothing to assist performance. Multi threads will help with the occasional long running request, but if its more than occasional then the thread list will eventually clog up no matter how many threads you have (and the more threads you have, the more they fight over the processor). The server sounds beefy, but since Zope is only going to be using one of those four cores, that might be deceptive. Kevin ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Problems with Zope + Apahe and exUserFolder logins
Hi everyone. I am migrating my web application to a new server and I am trying to configure Zope + Apache as described in http://wiki.zope.org/zope2/ZopeAndApache . I previously used fast-cgi to communicate Apache with Zope and it worked just fine. I use exUserFolder to store my users in a postgresql database. Everything seems to work fine except when I try to login to a restricted area through port 80. After debugging a bit (unfortunately I'm not any Zope guru) I've noticed that the username and password variables which are posted from the login page are located at the HTTPRequest? http://wiki.zope.org/zope2/ZopeAndApache/createform?page=HTTPRequest.stdin property when the page is accessed through the 8080 port, but they are not present when the page is accessed though apache. Anyone has any clue about what might be going on? Thanks a lot. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Z2.log has Anonymous users
Andreas Jung wrote at 2008-7-1 08:31 +0200: ... My plone instance doesn't log user names into the Z2.log access log file. Instead I always get user names as Anonymous. Is there any setting that I need to change to get the user name correctly? I am on the following versions of Plone/Zope: *Plone 3.0.5 *CMF-2.1.0 *Zope (Zope 2.10.5-final, python 2.4.4, linux2) *Python 2.4.4 (#1, Jun 30 2008, 14:50:35) [GCC 3.4.6 20060404 (Red Hat 3.4.6-3)] *PIL 1.1.6 Credentials are only logged inside the Z2.log for basic authentication. However Zope and Plone use cookie-based authentication. That's why you see 'Anonymous'. When I remember right, there is a ZServer patch (ZServer writes Z2.log) to make correct user logging available even for non HTTP-authentication: It's very easy: instead of emulating its own authentication method, ZServer should use the Zope API to determine the current user. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] CMF security error in DCWorkflow that doesn't make sense
Peter Bengtsson wrote at 2008-7-1 12:15 +0100: ... The owner of the executing script is defined outside the context of the object being accessed. The script has proxy roles, but they do not apply in this context.. Access to 'workflow' of (Products.DCWorkflow.Expression.StateChangeInfo instance at 0xddffb48) denied. Access requires Manage_portal_Permission, granted to the following roles: ['Manager']. The executing script is (PythonScript at /intranet/portal_workflow/intranet_workflow/scripts/wf_script_despatcher), owned by oliver. The message tells you that the problem is the so called executable owner of the script. The executable owner is a mechanism to protect against Trojan horse attacks: a user with low priviledges creates an executable object that does bad things when executed by a user with high priviledges. The protection restricts the effective permissions to the intersection of the current user permissions and the executable owner permission. Probably, the executable owner of this script does not exist in your test instance (but it exists in the other instances). Use take ownership to change the executable owner (or create the executable owner). -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )