Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
On Sun, Mar 27, 2011 at 10:54 AM, Uli Fouquet u...@gnufix.de wrote: - `assert` works like you would expect it to work in tests. No need to use `self.assertEqual()`` and friends (but you can if you prefer). How do they deal with the fact that assert statements are dropped when Python is run with -O? -- Benji York ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi there, Benji York wrote: On Sun, Mar 27, 2011 at 10:54 AM, Uli Fouquet u...@gnufix.de wrote: - `assert` works like you would expect it to work in tests. No need to use `self.assertEqual()`` and friends (but you can if you prefer). How do they deal with the fact that assert statements are dropped when Python is run with -O? I think they don't. You get a warning if you do and, of course, reports might become useless. On the other hand you have distributed-testing options (I haven't tried yet) where you should be able to finetune the enviroments (yes, plural) nicely in which your tests run. It is said that distributed testing with py.test allows parallel usage of several CPUs (if the machine has several CPUs), parallel testing on remote machines and more. AFAICS you can also run the same tests with different Python interpreters in parallel. In sum all that might give better speedups than -O, but -- I haven't tried it yet. Best regards, -- Uli signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
On Sun, Mar 27, 2011 at 11:13 AM, Martin Aspeli optilude+li...@gmail.com wrote: Hi, On 27 March 2011 15:54, Uli Fouquet u...@gnufix.de wrote: The (limited) experiences with py.test, however, were awesome. Some points that are quite cool IMHO: - Easy finding of tests: just write some ``test_function`` in a ``test_module`` and it will be found and executed. That also makes py.test tests more readable and maybe more intuitive. I'm not sure this is always a good idea. It feels a bit implicit, and having a base class isn't really a big problem, IMHO. It seems a bit like the kind of thing that sounds cool (look, it's even easier!), but in practice makes little difference. +1 +1 +1 This is especially important for doctests (and manuel) or any situation where setup is important and where you can't really guess. - py.test is more widespread in the Python community (that's my impression; I can't proof it) What about nose? It looks to me like a layerish mechanism might be possible in nose, or at least like zope.testing layers could be integrated with nose. - Support of unittest/unittest2: you can write standard lib setups (defining TestCases; no need to also write testsuite-setup stuff) and they will be found/executed. zope.testrunner for instance does not support the new `setUpClass`/`tearDownClass` concept of unittest2 (yes, you would use layers in that case; but it might be nice if zope.testrunner would support also class-wide fixtures in unittest2-style; people from other worlds might expect that to work). zope.testing should definitely gain support for the new unittest2 hooks. That wouldn't be very hard, though. ;-) I assume you mean zope.testrunner. Main drawbacks I see on py.test side are: - Lack of layer support (yet). Maybe we can do something about that in `zope.pytest` based on `plone.testing.layer`. - Limited doctest support. It is quite difficult (AFAIK) to define fixtures for doctests or to even set the usual doctest options (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests are simply collected and executed and not much finetuning is possible. With zope.testrunner, you *do* need a test_suite method to run doctests. I think that's a good thing. Look at plone.testing's README for examples. Again, +1 If I were to use nose or py.tests, I would want to adopt an explicit style, which I believe is possible w nose. ... FWIW, I think we should stop using .txt doctests for unit tests. I disagree, of course. Doctests should be used to test *documentation* (the examples are valid). Manuel is *much* better for that. (Of course, manuel is arguably a form of doctest.) For actual unit tests, writing tests in a unittest class is almost always better in the long run. doctests don't scale well and discourage the kind of ad-hoc this seems broken, I'll just write a quick test or I just fixed a bug, better add a regression test testing. You're just not using them correctly. :) One thing I hate about unittest is the javiotic ceremony it involves. Doctests can cut down on the clutter a lot. I believe that that is py.test's strength as well.. More generally, I'd love to see us adopt another test runner so that we can stop maintianing zope.testrunner. When it was written at the turn of the century, there weren't good alternatives. Personally, I think maintaining it is boring. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
On Mon, Mar 28, 2011 at 9:33 AM, Uli Fouquet u...@gnufix.de wrote: Hi there, Benji York wrote: On Sun, Mar 27, 2011 at 10:54 AM, Uli Fouquet u...@gnufix.de wrote: - `assert` works like you would expect it to work in tests. No need to use `self.assertEqual()`` and friends (but you can if you prefer). How do they deal with the fact that assert statements are dropped when Python is run with -O? I think they don't. You get a warning if you do and, of course, reports might become useless. I don't think they're that foolish. At least I hope they're not. I think they use a custom compiler. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi there, I first would like to thank everybody very much for the interesting and elaborated answers to my last questions! I learned a lot about registries and think that for now `plone.testing` (which in fact we hadn't had on the screen) will help to solve some of the most urgent issues we have with zope.pytest currently. So, thank you very much! A sidenote: IMO it would be nice to have proper deletion of registrations in the long run as well and I am willing to help as good as I can with that, even if that would mean much work. The 'just-setup-your-layers-more-carefully'-solution is a bit tricky in testing-frameworks without layers ;-) Wolfgang Schnerring wrote: [snip] Are there already possibilities to mimic testlayer-like behaviour in `pytest` which we simply overlooked? An honest counter question: Why not use zope.testrunner? What advantages does py.test offer? Due to my limited experience with py.test I'm certainly not in a position to answer that comprehensively. I personally use zope.testrunner and layers happily in most of my projects. zope.pytest is currently certainly more a try to see what's possible with other approaches than zope.testrunner and some features of py.test are really promising IMO. The (limited) experiences with py.test, however, were awesome. Some points that are quite cool IMHO: - Easy finding of tests: just write some ``test_function`` in a ``test_module`` and it will be found and executed. That also makes py.test tests more readable and maybe more intuitive. - Lots of setup code (unrelated to fixtures) can simply be skipped. No need to do the ``testsuite = complex-testcase-collecting`` over and over again. Maybe the main point of py.test. - py.test is more widespread in the Python community (that's my impression; I can't proof it) - Support of unittest/unittest2: you can write standard lib setups (defining TestCases; no need to also write testsuite-setup stuff) and they will be found/executed. zope.testrunner for instance does not support the new `setUpClass`/`tearDownClass` concept of unittest2 (yes, you would use layers in that case; but it might be nice if zope.testrunner would support also class-wide fixtures in unittest2-style; people from other worlds might expect that to work). - `assert` works like you would expect it to work in tests. No need to use `self.assertEqual()`` and friends (but you can if you prefer). - It was very easy to run py.test from the Sphinx doctest builder which made it possible to test documentation on a quite complex level. That might be possible with zope.testrunner as well (but I never tried that; wait - I tried once and it worked, but was more complex to setup IIRC). I see that this is a very special usecase. That said I'd like to stress that I would not say that py.test is superior to zope.testrunner in general. I only enjoyed the first experiences with py.test very much and could imagine that it would be a pleasure for others too. Other things, however, meant less fun. Main drawbacks I see on py.test side are: - Lack of layer support (yet). Maybe we can do something about that in `zope.pytest` based on `plone.testing.layer`. - Limited doctest support. It is quite difficult (AFAIK) to define fixtures for doctests or to even set the usual doctest options (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests are simply collected and executed and not much finetuning is possible. My very own concern about the latter point is: if this cannot be solved satisfactorily (in terms of readability and ease of use at least), py.test will not become a really hot candidate in the testing framework game on the Zope side at all. There are too many doctests used already and whether you like them or not: every testing framework used should offer at least minimal support for doctest fixtures and doctest options. But I might just have missed these features in py.test and they are provided already. For now I think that there is absolutely no need to think about a general move to py.test for the ztk. I guess Martijn or others with more py.test experience could say more concerning the pros and cons of py.test from daily work use. [snip] Best regards, -- Uli signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi, On 27 March 2011 15:54, Uli Fouquet u...@gnufix.de wrote: The (limited) experiences with py.test, however, were awesome. Some points that are quite cool IMHO: - Easy finding of tests: just write some ``test_function`` in a ``test_module`` and it will be found and executed. That also makes py.test tests more readable and maybe more intuitive. I'm not sure this is always a good idea. It feels a bit implicit, and having a base class isn't really a big problem, IMHO. It seems a bit like the kind of thing that sounds cool (look, it's even easier!), but in practice makes little difference. - Lots of setup code (unrelated to fixtures) can simply be skipped. No need to do the ``testsuite = complex-testcase-collecting`` over and over again. Maybe the main point of py.test. You don't need that for zope.testrunner either, of course, at least not when using unittest base classes. - py.test is more widespread in the Python community (that's my impression; I can't proof it) What about nose? - Support of unittest/unittest2: you can write standard lib setups (defining TestCases; no need to also write testsuite-setup stuff) and they will be found/executed. zope.testrunner for instance does not support the new `setUpClass`/`tearDownClass` concept of unittest2 (yes, you would use layers in that case; but it might be nice if zope.testrunner would support also class-wide fixtures in unittest2-style; people from other worlds might expect that to work). zope.testing should definitely gain support for the new unittest2 hooks. That wouldn't be very hard, though. ;-) Main drawbacks I see on py.test side are: - Lack of layer support (yet). Maybe we can do something about that in `zope.pytest` based on `plone.testing.layer`. - Limited doctest support. It is quite difficult (AFAIK) to define fixtures for doctests or to even set the usual doctest options (``ELLIPSIS``, ``NORMALIZE_WHITESPACE``, ...) at setup time. Doctests are simply collected and executed and not much finetuning is possible. With zope.testrunner, you *do* need a test_suite method to run doctests. I think that's a good thing. Look at plone.testing's README for examples. My very own concern about the latter point is: if this cannot be solved satisfactorily (in terms of readability and ease of use at least), py.test will not become a really hot candidate in the testing framework game on the Zope side at all. There are too many doctests used already and whether you like them or not: every testing framework used should offer at least minimal support for doctest fixtures and doctest options. But I might just have missed these features in py.test and they are provided already. FWIW, I think we should stop using .txt doctests for unit tests. Doctests should be used to test *documentation* (the examples are valid). For actual unit tests, writing tests in a unittest class is almost always better in the long run. doctests don't scale well and discourage the kind of ad-hoc this seems broken, I'll just write a quick test or I just fixed a bug, better add a regression test testing. For now I think that there is absolutely no need to think about a general move to py.test for the ztk. I think there's benefit in unifying the concepts and support for concepts like layers so that people can use the test runner they prefer. Maritn ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hi Jim, On 25 March 2011 14:12, Jim Fulton j...@zope.com wrote: Agree. There is a problem in that provideAdapter() and friends don't use getSiteManager() - the always use the global site manager. And there are parts of zope.component that use module level variables directly, ignoring hooks. These are meant to work this way. If you want to do local configuration, you should explictly select the relevent registry/manager or call getSiteManager and use the result. I sortof understand, but it makes it impossible to let people use provideAdapter() co in test setup and still retain some kind of layered test setup, without the kind of hacks we do in plone.testing. -- but anyone could at any time call getSiteManager.sethook to change it! Seriously? Nobody calls that but deep infrastructure code. People do call zope.site.hooks.setHooks() sometimes, though, e.g. upon traversal. This was never meant to be an application-level feature. I find the notion that people would call these dureing traversal to be disturbing. Are you sure you're not confusing this with setSite? Sorry, I meant setSite() above yes. Although sometimes people call setHooks() and then setSite(site) in test setup, because setSite() doesn't work until setHooks() has been called once. I think this may sometimes just be cargo-cult, though. Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hello, part two. :) * Uli Fouquet u...@gnufix.de [2011-03-24 01:05]: A big advantage of test layers over `pytest` testing scopes might be that you can spread your tests associated to a certain layer over many files/modules/packages as you like and the setup/teardown will nevertheless only be performed once for each layer Compared to Zope test layers I came to the conclusion that there is not much like this concept already in `pytest` and the behaviour of test layers can not easily be faked. I fully agree, the test layer concept of zope.testrunner provides some very nice properties, most notably: 1) Shared fixture setup. It is shared across many TestCases and created only once per test run. (And it is independent of how the tests are organized into modules or whatever.) 2) Stacking of fixture setups. My canonical example: First, we initialize the application, load ZCML, prepare databases, etc. and talk to it with zope.testbrowser. That's the first layer. Then we add a HTTP port with gocept.selenium on another layer and do the rest with a real browser -- but the whole, possibly expensive, application setup *does not have to be done again*. 3) Orthogonal composition, i.e. you can have small, well defined fixture units that you can then combine in a separate step to form the actual setup for your tests. This enables for example that library packages like zope.component could provide test infrastructure for themselves, and application packages could pull all the test infrastructure together that they need -- not like zope.app.testing for example, which is a giant monolith that sets up everything at once. I think that these properties are very valuable, and I haven't seen them anywhere else so far. I haven't looked at py.test closely, yet, but from what I've seen, and what you wrote, it doesn't seem to offer these things right now. Would it make sense to bring the Zope layer concept into `pytest`? Are there already possibilities to mimic testlayer-like behaviour in `pytest` which we simply overlooked? An honest counter question: Why not use zope.testrunner? What advantages does py.test offer? But your email brings into focus a question that I've been thinking about lately: What's the future direction for test fixture setup? Because, while test layers are nice because they have the above properties, I'm not too happy with the current implementation, namely the use (or is it abuse? ;-) of __bases__ and __name__, which leads very naturally to not-so-helpful implementation patterns. As a concrete example, some of the test layers of ZTK packages, most notably the successors of zope.app.testing.functional in zope.component, zope.app.appsetup, zope.app.wsgi, are implemented right now in a way that does not have properties 2+3. The most straightforward way to improve that situation would be to use plone.testing, since that provides a layer implementation that is both sane and easy to use, and has all the nice properties. (Maybe even fold some of it into zope.testrunner itself.) But then I realized, that's a major undertaking, so we better think about what the actual goals are before changing lots of stuff. And then I heard people expressing interest in other test runners. (py.test and nose seem to be the biggest players, stdlib's unittest/unittest2 seems to be woefully behind in terms of test selection and other features like coverage or debugging. I for one happen to like zope.testrunner, but if there's a good alternative, why not combine our efforts?) I also found that it is rather hard to explain how to use test layers in a way so you actually get the above properties, so I started wondering whether there was a cleaner way to get them. So I guess I have three questions: 1. (The starting point:) How can we improve the test infrastructure situation of the ZTK packages? 2. (The main thing:) Are test layers, both the concept and the implementation as epitomized by plone.testing, the way to go or is there a cleaner alternative? 3. (But only tangentially, I don't want to sidetrack the discussion at hand nor start a flamewar:) Is there a compelling alternative to zope.testrunner that would make it worthwile to think about either generalizing the fixture support or even migrating away from zope.testrunner? Wolfgang -- Wolfgang Schnerring · w...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Test fixture concepts (was: Zope test layers, pytest, and test isolation)
Hi, On 26 March 2011 14:18, Wolfgang Schnerring w...@gocept.com wrote: Because, while test layers are nice because they have the above properties, I'm not too happy with the current implementation, namely the use (or is it abuse? ;-) of __bases__ and __name__, which leads very naturally to not-so-helpful implementation patterns. As a concrete example, some of the test layers of ZTK packages, most notably the successors of zope.app.testing.functional in zope.component, zope.app.appsetup, zope.app.wsgi, are implemented right now in a way that does not have properties 2+3. The most straightforward way to improve that situation would be to use plone.testing, since that provides a layer implementation that is both sane and easy to use, and has all the nice properties. (Maybe even fold some of it into zope.testrunner itself.) I'd be happy to move layer.py from plone.testing to zope.testrunner for sure. But then I realized, that's a major undertaking, so we better think about what the actual goals are before changing lots of stuff. And then I heard people expressing interest in other test runners. (py.test and nose seem to be the biggest players, stdlib's unittest/unittest2 seems to be woefully behind in terms of test selection and other features like coverage or debugging. I for one happen to like zope.testrunner, but if there's a good alternative, why not combine our efforts?) I also found that it is rather hard to explain how to use test layers in a way so you actually get the above properties, so I started wondering whether there was a cleaner way to get them. Personally, I rely a lot on the layer mechanism and would not want to lose it. So I guess I have three questions: 1. (The starting point:) How can we improve the test infrastructure situation of the ZTK packages? I'd say having more reusable test layers would help. 2. (The main thing:) Are test layers, both the concept and the implementation as epitomized by plone.testing, the way to go or is there a cleaner alternative? When we looked at how to improve Plone's testing story, we couldn't find anything better. One thing I would like is a better way to support python setup.py test. 3. (But only tangentially, I don't want to sidetrack the discussion at hand nor start a flamewar:) Is there a compelling alternative to zope.testrunner that would make it worthwile to think about either generalizing the fixture support or even migrating away from zope.testrunner? It's a good question to ask, though I think this would be a really big undertaking for maybe only marginal gain (if any). Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hello Uli, I've spent quite some time thinking (and partly coding) about the same issues you mention (but didn't feel ready to talk about it here, yet), so I'm glad that maybe we can start thinking about them together I think your email addresses two quite different topics, so I'll split my reply. First up: test support for zope.component. Second part: about the concept of test layers. * Uli Fouquet u...@gnufix.de [2011-03-24 01:05]: Right now we have a problem with pytest integration when it comes to ZCA setups [...] all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation. Exactly. This issue has bitten me too in various places, and as far as I know there are no solutions for it, yet. What I envision to solve this issue is that test support for zope.component should work the same way as with the ZODB. There, we have a *stack* of Databases (DemoStorages, to be precise) that wrap each other, where each one delegates reads downwards, but keeps writes for itself. So you might have one database for the layer that provides the baseline, and each test (in its setUp) gets its own database where it can do whatever it wants, because it is thrown away in its tearDown. In principle, quite a few of the mechanics to do the same thing with zope.component registries are already there (since a registry keeps a list of base registries it will delegate to when something can not be found in itself). And as Hanno and Godefroid mentioned, plone.testing does something in this direction already. (And, it bears repeating, in its core has no dependencies on Plone or Zope2.) But as far as I see, there are issues that plone.testing does not address: 1. I've been going over this stuff with my colleague Thomas Lotze, and we realized that just squeezing in a new registry and bending its bases to the previously active one is not enough for full isolation, since this does not cover *deleting* registrations (one, you can only delete the registration from the precise registry it was created in, and two, in the just-bend-the-bases approach, once you delete a registration, it's gone forever). I think to provide full isolation, we need to make *copies*. And since zope.component in general supports a chain of registries, we probably need to make copies of each of them. 2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry. I think to provide test support for zope.component (i. e. generally, at the library level), we need to support both entry points. The global one is not hard, but the getSiteManager one gets nasty really fast, because of course we have to rely on bending getSiteManager to return the current test registry -- but anyone could at any time call getSiteManager.sethook to change it! Which means we need to intercept that and a) prevent our hook from being replaced and b) inject the new registry into the test stack somehow. As far as I understand, plone.testing sidesteps these issues by only dealing with the global registry, and specially munging the two known cases in the Zope world where getSiteManager is changed (zope.site and five.something). ** I'd like to know what people think about this plan. Thomas and I have been over this quite a bit and think it's sound, not overly complicated, and (after we did some experiments) definitely doable. Please do point out stuff we missed. :-) I'd very much like to put this functionality into zope.component itself, which of course raises backwards compatibility issues galore, but any code for this definitely isn't wasted since we can always package it separately if we don't find a way to integrate it. Thomas and I taken up implementing this, but we can't devote a lot of time to it (about one session per week), so realistically I'm afraid I guess it will take a few months until we have something of substance. So if there are people who want to pitch in, that'd be great. I definitely could write up a more detailed plan and maybe even formulate smaller chunks so we could go at this with more people. Wolfgang -- Wolfgang Schnerring · w...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring w...@gocept.com wrote: Hello Uli, I've spent quite some time thinking (and partly coding) about the same issues you mention (but didn't feel ready to talk about it here, yet), so I'm glad that maybe we can start thinking about them together I think your email addresses two quite different topics, so I'll split my reply. First up: test support for zope.component. Second part: about the concept of test layers. * Uli Fouquet u...@gnufix.de [2011-03-24 01:05]: Right now we have a problem with pytest integration when it comes to ZCA setups [...] all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation. Exactly. This issue has bitten me too in various places, and as far as I know there are no solutions for it, yet. The classic solution is to start tests with empty registries, or, if you're using layers, with some baseline registries. What I envision to solve this issue is that test support for zope.component should work the same way as with the ZODB. There, we have a *stack* of Databases (DemoStorages, to be precise) that wrap each other, where each one delegates reads downwards, but keeps writes for itself. So you might have one database for the layer that provides the baseline, and each test (in its setUp) gets its own database where it can do whatever it wants, because it is thrown away in its tearDown. In principle, quite a few of the mechanics to do the same thing with zope.component registries are already there (since a registry keeps a list of base registries it will delegate to when something can not be found in itself). And as Hanno and Godefroid mentioned, plone.testing does something in this direction already. (And, it bears repeating, in its core has no dependencies on Plone or Zope2.) I like the idea of stacking registries. But as far as I see, there are issues that plone.testing does not address: 1. I've been going over this stuff with my colleague Thomas Lotze, and we realized that just squeezing in a new registry and bending its bases to the previously active one is not enough for full isolation, since this does not cover *deleting* registrations (one, you can only delete the registration from the precise registry it was created in, and two, in the just-bend-the-bases approach, once you delete a registration, it's gone forever). I think to provide full isolation, we need to make *copies*. And since zope.component in general supports a chain of registries, we probably need to make copies of each of them. Is deleting registrations important? This seems like an odd use case. If it's needed, I would suggest starting with a baseline (e.g. stack) that doesn't include the component you want to test deleting, then adding in setup. 2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry. I think to provide test support for zope.component (i. e. generally, at the library level), we need to support both entry points. Why? Why would someone care about anything other than the current effective configuration. The global one is not hard, but the getSiteManager one gets nasty really fast, because of course we have to rely on bending getSiteManager to return the current test registry But as you point out, there's a hook for that. -- but anyone could at any time call getSiteManager.sethook to change it! Seriously? Nobody calls that but deep infrastructure code. Which means we need to intercept that and a) prevent our hook from being replaced and b) inject the new registry into the test stack somehow. I think you're making this more complicated than it needs to be. As far as I understand, plone.testing sidesteps these issues by only dealing with the global registry, and specially munging the two known cases in the Zope world where getSiteManager is changed (zope.site and five.something). ** I'd like to know what people think about this plan. Thomas and I have been over this quite a bit and think it's sound, not overly complicated, and (after we did some experiments) definitely doable. Please do point out stuff we missed. :-) I think a stack-based approach is very appealing. I think anything more complex is likely to cause more problems than it solves. I'd very much like to put this functionality into zope.component itself, which of course raises backwards compatibility issues galore, Not sure why this would have to be backward incompatible, but I'm unconvinced that the complexity comes close to being justified by the benefit. but any code for this definitely isn't wasted since we can always package it separately if we
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
Hi, On 25 March 2011 13:17, Jim Fulton j...@zope.com wrote: On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring w...@gocept.com wrote: Hello Uli, I've spent quite some time thinking (and partly coding) about the same issues you mention (but didn't feel ready to talk about it here, yet), so I'm glad that maybe we can start thinking about them together I think your email addresses two quite different topics, so I'll split my reply. First up: test support for zope.component. Second part: about the concept of test layers. * Uli Fouquet u...@gnufix.de [2011-03-24 01:05]: Right now we have a problem with pytest integration when it comes to ZCA setups [...] all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation. Exactly. This issue has bitten me too in various places, and as far as I know there are no solutions for it, yet. The classic solution is to start tests with empty registries, or, if you're using layers, with some baseline registries. plone.testing (which is Plone non-specific and will shortly be BSD licensed) allows for stacking of ZCA registries. It has to do some ugly hacking to achieve this, since zope.component stores handles to the global registry in *three* different modules and update them in weird ways depending on the registry hooking, but it's well tested and robust now. See http://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.py and http://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.txt. What I envision to solve this issue is that test support for zope.component should work the same way as with the ZODB. There, we have a *stack* of Databases (DemoStorages, to be precise) that wrap each other, where each one delegates reads downwards, but keeps writes for itself. So you might have one database for the layer that provides the baseline, and each test (in its setUp) gets its own database where it can do whatever it wants, because it is thrown away in its tearDown. In principle, quite a few of the mechanics to do the same thing with zope.component registries are already there (since a registry keeps a list of base registries it will delegate to when something can not be found in itself). And as Hanno and Godefroid mentioned, plone.testing does something in this direction already. (And, it bears repeating, in its core has no dependencies on Plone or Zope2.) I like the idea of stacking registries. plone.testing implements this. If we could fix zope.component to make the implementation less ugly, that'd be a big win. But as far as I see, there are issues that plone.testing does not address: 1. I've been going over this stuff with my colleague Thomas Lotze, and we realized that just squeezing in a new registry and bending its bases to the previously active one is not enough for full isolation, since this does not cover *deleting* registrations (one, you can only delete the registration from the precise registry it was created in, and two, in the just-bend-the-bases approach, once you delete a registration, it's gone forever). Correct, although this is in practice extremely rare. I'd say it's much better to control setup more carefully and not have to undo in a child layer. I think to provide full isolation, we need to make *copies*. And since zope.component in general supports a chain of registries, we probably need to make copies of each of them. Copying is very hard. It was my first attempt in plone.testing and didn't work out well. You need to support pickling of registries for local/persistent component registries. I cannot begin to tell you how many weird pickling errors I found and had to work around. Is deleting registrations important? This seems like an odd use case. If it's needed, I would suggest starting with a baseline (e.g. stack) that doesn't include the component you want to test deleting, then adding in setup. +1 2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry. I think to provide test support for zope.component (i. e. generally, at the library level), we need to support both entry points. Why? Why would someone care about anything other than the current effective configuration. Agree. There is a problem in that provideAdapter() and friends don't use getSiteManager() - the always use the global site manager. And there are parts of zope.component that use module level variables directly, ignoring hooks. The global one is not hard, but the getSiteManager one gets nasty really fast, because of course we have to rely on bending getSiteManager to return the current test registry But as you point
Re: [Zope-dev] zope.component test isolation (was: Zope test layers, pytest, and test isolation)
On Fri, Mar 25, 2011 at 9:58 AM, Martin Aspeli optilude+li...@gmail.com wrote: On 25 March 2011 13:17, Jim Fulton j...@zope.com wrote: On Fri, Mar 25, 2011 at 4:24 AM, Wolfgang Schnerring w...@gocept.com wrote: ... 2. zope.component has two entry points, the global site registry and the current registry (getGlobalSiteManager and getSiteManager). The current registry can be anything, or more precisely, you can call getSiteManager.sethook(callable) and provide a callable that returns the current registry. I think to provide test support for zope.component (i. e. generally, at the library level), we need to support both entry points. Why? Why would someone care about anything other than the current effective configuration. Agree. There is a problem in that provideAdapter() and friends don't use getSiteManager() - the always use the global site manager. And there are parts of zope.component that use module level variables directly, ignoring hooks. These are meant to work this way. If you want to do local configuration, you should explictly select the relevent registry/manager or call getSiteManager and use the result. -- but anyone could at any time call getSiteManager.sethook to change it! Seriously? Nobody calls that but deep infrastructure code. People do call zope.site.hooks.setHooks() sometimes, though, e.g. upon traversal. This was never meant to be an application-level feature. I find the notion that people would call these dureing traversal to be disturbing. Are you sure you're not confusing this with setSite? Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope test layers, pytest, and test isolation
Hi. Disclaimer: I don't know anything about pytest. On Thu, Mar 24, 2011 at 1:05 AM, Uli Fouquet u...@gnufix.de wrote: Are there cheap/fast ways to cache/restore registry setups we hadn't had on the screen? Really fast setups/cache-restores could make even function-wise registrations a considerable thing. The approach we take in the Plone world (though plone.testing has no Plone or Zope 2 dependencies) is: http://pypi.python.org/pypi/plone.testing/4.0a3#configuration-registry-sandboxing As part of plone.testing we have proper layer based ZCA isolation, by doing stacked component registries. This approach is extremely fast, as you only need to push/pop a registry to/from the stack for each test. If you want to use layers, you can use this. Otherwise you might be able to take the ideas and code. Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope test layers, pytest, and test isolation
Hi there, [extensive introduction by Uli of zope.pytest] Thanks Uli for that introduction to the package and its problems! I've been using zope.pytest within 2 Grok projects so far to test through webob some JSON webservices, and it's worked well for me. I hadn't had to deal with the issue of multiple configurations in those tests though, as they're basically functional/integration tests. I've also been happily using plain py.test in several other non-Zope projects, and I must say it has many nice features that make life easier, and makes a lot of test cruft go away. I'm particularly hoping that Wolfgang Schnerring will give feedback: I'm curious how his investigations on improving the layer story might also benefit py.test integration. Regards, Martijn P.S. Uli did a lot of work himself to make this package and help document it, by the way, he's doing himself injustice in his introduction. :) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope test layers, pytest, and test isolation
Hi Uli, I am happy that some people started to look what is needed to make modular component registrations with py.test. The long explanation that you wrote made it easy for me to jump into funcargs. Those last months, I have been diving into the great plone.testing package written by Martin Aspeli. Among others, plone.testing has a very nice module zca.py to ease building zope.testing layers that need ZCA. See https://dev.plone.org/plone/browser/plone.testing/trunk/src/plone/testing/zca.py Specifically, pushGlobalRegistry and popGlobalRegistry are a pair of functions that allow to make registrations step by step, while easily throwing away the latest registrations when they are not needed anymore. plone.testing does not depend on Zope2 even if it comes with support for building Zope2 layers. Le 24/03/11 01:05, Uli Fouquet a écrit : Compared to Zope test layers I came to the conclusion that there is not much like this concept already in `pytest` and the behaviour of test layers can not easily be faked. `pytest` provides only the three mentioned scopes as kind of 'natural' layers. Spreading fixtures over many modules (as layers easily do) might contradict with the basic design goals of pytest and I am pretty sure that Holge Krekel wouldn't like it. I do not agree with that last statement : the fact that resources can be cached during a session allows them to be reused over many modules. IOW, it does not contradict pytest design. Digging into funcargs gives me the feeling that they are much richer than layers. Because funcargs are functions that get access to a lot of context, they allow for more flexibility than layers that are only static resources. Overall, we're now looking for a satisfying solution in terms of runtime and isolation. Some of the questions that arise: My feeling is that the extrakey argument of cached_setup can be used in combination with code similar to plone.testing ZCA support to build something satisfying in terms of runtime and isolation. Would it make sense to bring the Zope layer concept into `pytest`? I think what is already available in py.test might avoid the need of layers. Are there already possibilities to mimic testlayer-like behaviour in `pytest` which we simply overlooked? See above. Are there cheap/fast ways to cache/restore registry setups we hadn't had on the screen? Really fast setups/cache-restores could make even function-wise registrations a considerable thing. Would it simply be okay to use the 'module' scope for registration setups? Or do you have completely different ideas how to solve that issue? Any comments are really appreciated! Best regards, -- Uli -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope test layers, pytest, and test isolation
Hi there, Some of you might have noticed that some time ago the first version of `zope.pytest` was released: http://pypi.python.org/pypi/zope.pytest It's a try to make usage of `pytest` more comfortable in Zope-based environments. `zope.pytest` is mainly based on Martijns, Jan-Wijbrands, and Christian Klingers efforts. I put in some bits too. Right now we have a problem with pytest integration when it comes to ZCA setups: among other things `zope.pytest` offers a pytest-funcarg__ compatible function to automate ZCML based fixtures, i.e. it automatically adds setup and teardown functionality before/after test functions if requested by the test writer like this:: import my.project from zope.pytest import configure from zope.component import queryUtility def pytest_funcarg__config(request): return configure(request, my.project, 'ftesting.zcml') def test_myutil_registered(config): util = queryUtility( my.project.interfaces.ISomeIface, name = 'myutil', default = None) assert util is not None Here for test functions requiring an argument named ``config`` it is guaranteed that before the test function is run a ZCA initialization is performed based on the ZCML configuration given in the ``ftesting.zcml`` of `my.project`. This configuration is cached for the whole test session (the complete test run including all tests). If we assume that the used `ftesting.zcml` provides a named utility named ``myutil`` then the call to ``configure`` in the funcarg__-function should perform that registration and the 'myutil_registered' test should succeed. All that works very well -- in principle. The problem is: the `configure` function calls the registry setup before any real test is run and performs a teardown only after the last of all tests was run. And that leads to side-effects. In other words: all tests share the same global ZCA registrations and changes to the registrations in one test will affect other tests run thereafter. We have a lack of test isolation. Example:: import my.project from zope.pytest import configure from zope.component import queryUtility, provideUtility from zope.interface import Interface def pytest_funcarg__config(request): return configure(request, my.project, 'ftesting.zcml') def test_1(config): util = object() provideUtility(util, provides=Interface, name='a_util') def test_2(config): util = queryUtility(Interface, name='a_util', default=None) assert util is None def test_3(): util = queryUtility(Interface, name='a_util', default=None) assert util is None Here the second test will fail. Even worse, also the third test will fail, which does not require a ZCA setup at all. That's because the global registry is setup before all tests and not torn down before the last test was run. This behaviour is partly intentional: we do not want to setup/teardown the ZCA before/after single test functions as this would increase time consumed by test runs dramatically. Instead we want to reduce the number of ZCA setups/teardown as far as possible. We _could_ do a per testfunction setup/teardown by simply using a different test scope in `configure`. `pytest` offers three kinds of scope for that purpose: `session`, `module`, and `function`. `session` is the current default. Using `function` as scope we would get complete test isolation but pretty long lasting test runs. That could stop people from writing tests. Something we would like to avoid by all means. We could instead use `module` as default scope, so that all test functions inside a module would share one ZCA configuration (and for different test modules the setup would be performed from scratch). This might be a reasonable compromise. Test isolation of single tests inside a test module would of course be broken but one could say: if you want a fresh registry, just put your tests inside a new test module. With Zope test layers there might be a similar problem (each layer can create a global ZCA configuration that then will be shared amongst all applied test methods) but I think people are used to it and can cope with that specific test isolation breakage. A big advantage of test layers over `pytest` testing scopes might be that you can spread your tests associated to a certain layer over many files/modules/packages as you like and the setup/teardown will nevertheless only be performed once for each layer (well, normally at least). If you have one layer and that's enough for you, you will only have one ZCA-setup/teardown in the whole testrun. That's fast and certainly fits many real-world use-cases. Compared to Zope test layers I came to the conclusion that there is not much like this concept already in `pytest` and the behaviour of test layers can not easily be faked. `pytest` provides only the three mentioned scopes as kind of 'natural' layers. Spreading fixtures over many modules (as layers easily do) might
[Zope] Test mail, please ignore
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Quick test, please ignore -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.8 (Darwin) iEYEARECAAYFAkv/WJkACgkQRAx5nvEhZLJ4oACgm3AmtfWAwvrhVvB83AxcbHfJ EdkAoLTWC5DeBS3xsls8FTCPU5UEZIgu =q806 -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] test, please ignore
I haven't received any mail from zope.org in quite while... -- http://www.slinkp.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope-dev] Re: Zope Test Summaries 2006-04-05
On Thu, Apr 06, 2006 at 06:04:11PM -0400, Tres Seaver wrote: +1 for getting rid of *anything* in Testing which does something as Utterly Evil (tm) as scribbling on 'os.environ'. Any tests which break on that account should be fixed, not covered over. You don't risk breaking production code, only tests, and those *need* to be fixed, even when running against the stable tree (I would even think backporting to 2.8 was worthwhile, just for the testing bugs it would expose). I agree with the sentiment, but thinking of it from a user's p.o.v. I would be mighty annoyed if I upgraded from 2.9.2 and 2.9.3 and found that tests in package foo were broken by tests in package bar only because of changes in package qux. Test pollution bugs can be real maddening time-wasters. Upgrading to 2.10 is another story ;-) I may try eradicating os.environ usage from Testing on my trunk sandbox and see what else breaks... -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope Test Summaries 2006-04-05
On Fri, Apr 07, 2006 at 10:04:36AM -0400, Paul Winkler wrote: I may try eradicating os.environ usage from Testing on my trunk sandbox and see what else breaks... Answer: nothing. Committed on the trunk. I did a bit of grepping around too. There are a few tests that twiddle os.environ, but these are all things that really rely on environ. For example, the tests for zdaemon/tests/testzdoptions.py, which uses a little monkeypatching in setUp() and unpatching in tearDown() so the environ should be restored after every test. However, there are some tests around that scribble on the REQUEST.environ as created by Testing.makerequest. So these would have inadvertently twiddled the real os.environ before my latest commit. The test authors might not even know they were doing that. Blech. Fixed :) -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope Test Summaries 2006-04-05
On Fri, Apr 07, 2006 at 12:52:29PM -0400, Paul Winkler wrote: On Fri, Apr 07, 2006 at 10:04:36AM -0400, Paul Winkler wrote: I may try eradicating os.environ usage from Testing on my trunk sandbox and see what else breaks... Answer: nothing. Committed on the trunk. To clarify, I just removed os.environ from Testing.makerequest. I didn't touch anything in Testing.ZopeTestCase. There's a bunch of os.environ usage in there and I'm not up for looking through that right now. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Test Summaries 2006-04-05
[We seem to have lost Steve Alexander's test summarizer. I will contact him in a separate mail.] The switch to Python 2.3.4 caused tests for Zope 2.7 and 2.8 to experience a funny problem with the logging module. Anybody want to take a guess what's up here? I know these are not a truly supported configurations, but still. http://mail.zope.org/pipermail/zope-tests/2006-April/004645.html http://mail.zope.org/pipermail/zope-tests/2006-April/004647.html The VHM tests in Zope 2.9 and trunk broke last night: http://mail.zope.org/pipermail/zope-tests/2006-April/004648.html http://mail.zope.org/pipermail/zope-tests/2006-April/004649.html Stefan -- Anything that happens, happens. --Douglas Adams ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Test Summaries 2006-04-05
On Thu, Apr 06, 2006 at 10:45:11AM +0200, Stefan H. Holek wrote: The VHM tests in Zope 2.9 and trunk broke last night: http://mail.zope.org/pipermail/zope-tests/2006-April/004648.html http://mail.zope.org/pipermail/zope-tests/2006-April/004649.html I'm assuming this was me, but I ran test.py -q -all before checking in so I'm not sure how that happened. Investigating. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Test Summaries 2006-04-05
--On 6. April 2006 12:06:25 -0400 Paul Winkler [EMAIL PROTECTED] wrote: On Thu, Apr 06, 2006 at 10:45:11AM +0200, Stefan H. Holek wrote: The VHM tests in Zope 2.9 and trunk broke last night: http://mail.zope.org/pipermail/zope-tests/2006-April/004648.html http://mail.zope.org/pipermail/zope-tests/2006-April/004649.html zope-tests? I've never seen that list :-) But this reminds of that I am missing mails from the buildbotnobody broke the the trunk/branches lately? Unbelieveble :-) Andreas pgpJkV028Jox3.pgp Description: PGP signature ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope Test Summaries 2006-04-05
On Thu, Apr 06, 2006 at 12:06:25PM -0400, Paul Winkler wrote: On Thu, Apr 06, 2006 at 10:45:11AM +0200, Stefan H. Holek wrote: The VHM tests in Zope 2.9 and trunk broke last night: http://mail.zope.org/pipermail/zope-tests/2006-April/004648.html http://mail.zope.org/pipermail/zope-tests/2006-April/004649.html I'm assuming this was me, but I ran test.py -q -all before checking in so I'm not sure how that happened. Investigating. Must've run the tests in the wrong sandbox :-\ This failure was a side-effect of the changes I made: - The unit tests in Products/Session were polluting os.environ with SERVER_NAME and SERVER_PORT keys. - The unit tests in Products/SiteAccess were relying on the old behavior of Testing.makerequest to unconditionally set those keys in the request. - My change allowed the values from the Sessions tests to show up in the SiteAccess tests. I've fixed the Sessions tests to use a dict instead of os.environ. But I'm also concerned that similarly indirect breakage might happen in other projects if they use os.environ carelessly. This part of my change was an effort to remove redundancy, not a bugfix; so on second thought, given the demonstrated potential for breakage in other test suites, it doesn't belong on the stable 2.9 branch and I'll revert that change there. But I'll leave it on the trunk. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: Zope Test Summaries 2006-04-05
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul Winkler wrote: On Thu, Apr 06, 2006 at 12:06:25PM -0400, Paul Winkler wrote: On Thu, Apr 06, 2006 at 10:45:11AM +0200, Stefan H. Holek wrote: The VHM tests in Zope 2.9 and trunk broke last night: http://mail.zope.org/pipermail/zope-tests/2006-April/004648.html http://mail.zope.org/pipermail/zope-tests/2006-April/004649.html I'm assuming this was me, but I ran test.py -q -all before checking in so I'm not sure how that happened. Investigating. Must've run the tests in the wrong sandbox :-\ This failure was a side-effect of the changes I made: - The unit tests in Products/Session were polluting os.environ with SERVER_NAME and SERVER_PORT keys. - The unit tests in Products/SiteAccess were relying on the old behavior of Testing.makerequest to unconditionally set those keys in the request. - My change allowed the values from the Sessions tests to show up in the SiteAccess tests. I've fixed the Sessions tests to use a dict instead of os.environ. But I'm also concerned that similarly indirect breakage might happen in other projects if they use os.environ carelessly. This part of my change was an effort to remove redundancy, not a bugfix; so on second thought, given the demonstrated potential for breakage in other test suites, it doesn't belong on the stable 2.9 branch and I'll revert that change there. But I'll leave it on the trunk. +1 for getting rid of *anything* in Testing which does something as Utterly Evil (tm) as scribbling on 'os.environ'. Any tests which break on that account should be fixed, not covered over. You don't risk breaking production code, only tests, and those *need* to be fixed, even when running against the stable tree (I would even think backporting to 2.8 was worthwhile, just for the testing bugs it would expose). Tres. - -- === Tres Seaver +1 202-558-7113 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFENZBb+gerLs4ltQ4RAntpAJ9iRnL8zZPMtaNGrQrKxAhxD7L7JgCgmC3O K/mIjFr++qV6xpo2L+x60vg= =1xn+ -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Fwd: Re: [Zope] test for the existence of a nodeValue]
Many thanks Dieter, works a treat. I'm working through the python tutorial now. -Original Message- From: Dieter Maurer [mailto:[EMAIL PROTECTED] Sent: 04 August 2005 18:58 To: [EMAIL PROTECTED] Cc: zope@zope.org Subject: Re: [Fwd: Re: [Zope] test for the existence of a nodeValue] Jon Bowlas wrote at 2005-8-4 10:31 +0100: ... But I still cant get it to work. I've changed my get_attributes.py script to this: if not attobject: return '' version = attobject.get_viewable() nodes = version.content.documentElement.getElementsByTagName(attname) if not nodes: return '' nodeValue = nodes[0].childNodes[0].nodeValue if not nodeValue: return none return nodeValue ... But I get the following error: Error Type: IndexError error Value: list index is out of range IndexError -- list index is out of range tells you that this is not a nodeValue problem. Almost surely, you got a node without childNodes. Then node.childNodes[0] will result in an IndexError. Thus, you will need to check for this situation (in a way similar to checks for an empty nodes list). and the traceback says Module None, line 7 in get_attributes is the cause. This should tell you in which line of get_attributes the problem was. As your mail agent decided to reformat the code (oh this Outlook :-( ), and you did not number the line, I cannot tell you -- but I guessed above (and probably not too bad)... -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Fwd: Re: [Zope] test for the existence of a nodeValue]
Thanks Dieter, But I still cant get it to work. I've changed my get_attributes.py script to this: if not attobject: return '' version = attobject.get_viewable() nodes = version.content.documentElement.getElementsByTagName(attname) if not nodes: return '' nodeValue = nodes[0].childNodes[0].nodeValue if not nodeValue: return none return nodeValue and my zpt is like this: span id=section_subheader tal:define=isadditional python:here.get_uclattribute(attobject, 'ucl_additional_name') tal:condition=python: isadditional !='none' tal:attributes=id python:here.get_uclattribute(attobject, 'ucl_additional_name_color')span tal:content=python:here.get_uclattribute(attobject, 'ucl_additional_name') or nothing tal:omit-tag=Website Subheader/span/span But I get the following error: Error Type: IndexError error Value: list index is out of range and the traceback says Module None, line 7 in get_attributes is the cause. Hope you can help. Jon Jon Bowlas wrote at 2005-8-2 19:08 +0100: ... I'm pretty new to python and zope but would like to know how I can adapt this script below to test to see if nodeValue actually contains a value and if it doesn't then I would like to return a default value I can check for using my zpt that references this script: You probably should work through the Python tutorial you will find on http://python.org;. ... if not nodes: return '' return nodes[0].childNodes[0].nodeValue Try: nodeValue = nodes[0].childNodes[0].nodeValue if not nodeValue: return someFault return nodeValue -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Fwd: Re: [Zope] test for the existence of a nodeValue]
Jon Bowlas wrote at 2005-8-4 10:31 +0100: ... But I still cant get it to work. I've changed my get_attributes.py script to this: if not attobject: return '' version = attobject.get_viewable() nodes = version.content.documentElement.getElementsByTagName(attname) if not nodes: return '' nodeValue = nodes[0].childNodes[0].nodeValue if not nodeValue: return none return nodeValue ... But I get the following error: Error Type: IndexError error Value: list index is out of range IndexError -- list index is out of range tells you that this is not a nodeValue problem. Almost surely, you got a node without childNodes. Then node.childNodes[0] will result in an IndexError. Thus, you will need to check for this situation (in a way similar to checks for an empty nodes list). and the traceback says Module None, line 7 in get_attributes is the cause. This should tell you in which line of get_attributes the problem was. As your mail agent decided to reformat the code (oh this Outlook :-( ), and you did not number the line, I cannot tell you -- but I guessed above (and probably not too bad)... -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] test for the existence of a nodeValue
Jon Bowlas wrote at 2005-8-2 19:08 +0100: ... I'm pretty new to python and zope but would like to know how I can adapt this script below to test to see if nodeValue actually contains a value and if it doesn't then I would like to return a default value I can check for using my zpt that references this script: You probably should work through the Python tutorial you will find on http://python.org;. ... if not nodes: return '' return nodes[0].childNodes[0].nodeValue Try: nodeValue = nodes[0].childNodes[0].nodeValue if not nodeValue: return someFault return nodeValue -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] test for the existence of a nodeValue
Hi All, Im pretty new to python and zope but would like to know how I can adapt this script below to test to see if nodeValue actually contains a value and if it doesnt then I would like to return a default value I can check for using my zpt that references this script: if not attobject: return '' version = attobject.get_viewable() nodes = version.content.documentElement.getElementsByTagName(attname) if not nodes: return '' return nodes[0].childNodes[0].nodeValue Two parameters are passed to this script attobject, attname. attobject is an object id and attname is the childNode with the nodeValue I wish to return. Hope someone can help, and I hope this makes sense. Thanks Jon ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] test
test -- - Matthew Burleigh - Systems Administrator, Digital Creations - publishers of Zope. - (888)344-4332 -- http://www.digicool.com - http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
test -- - Matthew Burleigh - Systems Administrator, Digital Creations - publishers of Zope. - (888)344-4332 -- http://www.digicool.com - http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
test -- - Matthew Burleigh - Systems Administrator, Digital Creations - publishers of Zope. - (888)344-4332 -- http://www.digicool.com - http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] Test - Is both the maillist and www.zope.org down?
??? ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Test - Is both the maillist and www.zope.org down?
Max Møller Rasmussen wrote: ??? Hm, the list seems to be up, but not so zope.org ... I've bee trying now for about 4 hours or so ... regards, ta -- smartferret it-consulting Dipl.-Ing. Linzbichler KEG Dipl.-Ing. Takashi Veikko Linzbichler Tannhofweg 28/3 A-8044 Graz, Austria Tel.: 0316 / 39 89 40 -0 Fax:0316 / 39 89 40 -20 Mobil: 0676 / 31 26 286 eMail: [EMAIL PROTECTED] WWW:http://www.smartferret.com -- ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
test -- Matt Burleigh Systems Administrator, Digital Creations - publishers of Zope. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
[Zope] test
ignore this -- - Matthew Burleigh - Systems Administrator, Digital Creations - publishers of Zope. - (888)344-4332 -- http://www.digicool.com - http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
ignore -- - Matthew Burleigh - Systems Administrator, Digital Creations - publishers of Zope. - (888)344-4332 -- http://www.digicool.com - http://www.zope.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
test, please ignore ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] TEST (please discard)
just a test, Sorry! Jack Morgan Debian GNU/Linux Enthusiast [EMAIL PROTECTED] http://www.mandinka.org ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
if it is dead of just everybody went on vacations? :) Vitaly. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] test
On Wed, 31 May 2000, Vitaly Osipov wrote: if it is dead of just everybody went on vacations? :) We are here, sure! (Russian?) Oleg.(All opinions are mine and not of my employer) Oleg Broytmann Foundation for Effective Policies [EMAIL PROTECTED] Programmers don't die, they just GOSUB without RETURN. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] test
if it is dead of just everybody went on vacations? :) :-) I think the list had some downtime for assorted reasons. Looks like it's back up again now... though if you don't read this then it's back down again ;-) John -- John Chandler / Software Developer / New Information Paradigms Ltd [ Linux in the office, AmigaOS in the home, PalmOS in the pocket ] The opinions above aren't those of my company... ...but then, they aren't really mine either. ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
[Zope] test
ignore please ___ Zope maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope-dev )
