Re: [Zope-dev] Circular dependency hell.
On Sat, Apr 17, 2010 at 06:34, Martin Aspeli optilude+li...@gmail.com wrote: Hi Lennart co, On 17 April 2010 02:38, Lennart Regebro rege...@gmail.com wrote: On Fri, Apr 16, 2010 at 19:53, Jonathan Lange j...@mumak.net wrote: As the author of one of those other testrunners, I can tell you that if you do this you'll find that your number one biggest problem is getting layers to work. Ah, right, layers are in zope.testing too. To actually get widespread movement from zope.testing we would have to make some other layer support. Hm... As you may know, I've been working on plone.testing. This is mainly to make it easier for people working on Plone packages to write good tests (and a lot of it is just about patterns, rather than code), but in fact it doesn't depend on Plone (and only soft-depends on Zope 2). I'm certainly hoping to use it for my plain Zope 3/Toolkit packages in the future. plone.testing makes very heavy use of layers. I think layers are a great feature, when done properly, and I haven't seen any better approach. The setUpClass/setUpModule stuff in unittest2 is nice, but doesn't really solve the same problem. For integration testing with something as complex as Zope 2 (or, arguably, the ZODB, various bits of the ZTK, and so on) layers allow us the framework authors to make life much easier for those people who are not experts in the framework. Anyway, a few high level observations: - In neither plone.testing (apart from its own tests), nor in code that uses it, would I imagine actually depending on zope.testing via an import. We use unittest(2) and doctest from the standard library. - We do depend on a zope.testing-compatible test runner, in that we expect layers to work. In reality, we depend on zc.recipe.testrunner, though I would *love* to be able to do 'python setup.py test' as well (and have that execute tests with layers). I have no idea how that works or whether it'd be possible. - The way zope.testing promotes writing layers is actually pretty evil. It overloads the 'class' keyword, essentially, making you spell base layers as base classes. This has a few problems: - If your base layer has a testTearDown method, say, and your layer doesn't, the base class version will actually inherit into the child layer. zope.testing will then run the same code twice, once for the base layer and once for the child layer. - You can't use OOP inheritance to re-use layer conveniences. - People get quite confused. :) - You can't have two copies of a layer - all layers are module-level global singletons. This becomes somewhat important when layers manage and expose resources (like database connections). Anyway, based on Ross Patterson's work in collective.testcaselayer, I think we have something better in plone.testing's layer module. Aha, interesting. General info and patterns: http://svn.plone.org/svn/plone/plone.testing/trunk/README.txt Layer module: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.py Layer doctests: http://svn.plone.org/svn/plone/plone.testing/trunk/src/plone/testing/layer.txt If I could have my cake and eat it too, I'd like: - A test runner that supports layers - A test runner that supports the new fixture methods in unittest2/Python 2.7 - setUpClass and setUpModule - A test runner that properly reports skipped tests (pretty easy) - A test runner that supports the above begin run from 'setup.py test' - A test runner installable via a buildout part just like zc.recipe.testrunner We could quite possibly factor layer.py out of plone.testing and push it into something else if that was desirable. It's self-contained and has no dependencies. All this sounds good and not to complicated compared to extracting the testrunner itself. :) Which in turn is less complicated than the work you've already done. :) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On Sat, Apr 17, 2010 at 03:41, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This kind of goes with Lennart's frustration about trying to port the ZTK packages, or a core subset, to Python 3. I would like to see the ZTK packages have really excellent documentation, as well as 100% test coverage. I think we need to look at refactoring how the testing story is done to get both needs satisfied: trying to achieve both with a single set of doctests is pretty near impossible: - - Testing setup and teardown code obscures the narrative flow / intent of the documentation. - - Edge case testing *really* obscures the same. - - Test coverage suffers, because it is hard to write edge-case tests in doctest. - - Porting the doctests where people have tried to juggle both goals across Python versions is a mess. The refactoring I would like to see happen is to move the main narrative documentation into a separate, Sphinx-driven 'docs' directory in each ZTK package. As part of this move, we can start adding some of the really nice Sphinx features (cross references, indexing, auto-generation of API reference information, etc.). We can still leave the main line of the code illustrated using doctest-style blocks: Sphinx has nice support for testing those blocks *while building the docs*. At the end of this part of the change, each package would have a much improved documentation set, with tested examples. At the end of the process, we would be able to put the docs for each package into an intersphinx tree underneath the main ZTK docs, which could serve as the gateway into them. The trickier testing bits we would re-write as super thorough, no shortcuts-taken unit tests: one testcase class per class (or API function) under test, at least one method per class-under-test method, with more preferred to get all code paths / preconditions covered. The goal here would be to comment out the doctests from the test_suites and get the code to 100% coverage using pure unit tests. Meanwhile, the doctests would be refactored into the Sphinx documentation, losing all the bits which obscure their intent as documentation. I would say that the risks here are slight, given that aiming for 100% test coverage is likely to catch any oversights made during porting to the new style. The amount of work for any given package is fairly-well contained, I think: most of the effort will be in reverse-engineering the intent of the edge-case tests. I made a sketch at what these changes would look for the zope.event package like on a branch in Subversion: http://svn.zope.org/zope.event/branches/tseaver-new_style_docs I select zope.event largely because it is small enough that the scope of the structural changes wouldn't be lost in the diffs for the actual tests (and also because I didn't want to do the work for a bigger package before getting feedback). The rendered docs from that branch are here: http://palladion.com/static/zope.event-docs/ I pushed the bzr branch to Launchpad, to let you see the kinds of changes I made to get there (our viewcvs story is not nearly as easy to review as the Launchpad changeset view): https://code.launchpad.net/~tseaver/zope.event/new_style_docs I am attaching an annotated diff for those who would prefer to see the changes in that format. I would say that this cleanup effort should start at the bottom layers of the ZTK, dependency-wise (zope.interface, zope.component, etc.) and move upwards (in the depenency sense) over time. Thoughts? +lots. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
(Although possible we could call it developing instead of hacking but that's not a big issue. ;) ) -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 2010-4-17 03:41, Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This kind of goes with Lennart's frustration about trying to port the ZTK packages, or a core subset, to Python 3. I would like to see the ZTK packages have really excellent documentation, as well as 100% test coverage. I think we need to look at refactoring how the testing story is done to get both needs satisfied: trying to achieve both with a single set of doctests is pretty near impossible: - - Testing setup and teardown code obscures the narrative flow / intent of the documentation. - - Edge case testing *really* obscures the same. - - Test coverage suffers, because it is hard to write edge-case tests in doctest. - - Porting the doctests where people have tried to juggle both goals across Python versions is a mess. The refactoring I would like to see happen is to move the main narrative documentation into a separate, Sphinx-driven 'docs' directory in each ZTK package. As part of this move, we can start adding some of the really nice Sphinx features (cross references, indexing, auto-generation of API reference information, etc.). We can still leave the main line of the code illustrated using doctest-style blocks: Sphinx has nice support for testing those blocks *while building the docs*. At the end of this part of the change, each package would have a much improved documentation set, with tested examples. At the end of the process, we would be able to put the docs for each package into an intersphinx tree underneath the main ZTK docs, which could serve as the gateway into them. The trickier testing bits we would re-write as super thorough, no shortcuts-taken unit tests: one testcase class per class (or API function) under test, at least one method per class-under-test method, with more preferred to get all code paths / preconditions covered. The goal here would be to comment out the doctests from the test_suites and get the code to 100% coverage using pure unit tests. Meanwhile, the doctests would be refactored into the Sphinx documentation, losing all the bits which obscure their intent as documentation. I would say that the risks here are slight, given that aiming for 100% test coverage is likely to catch any oversights made during porting to the new style. The amount of work for any given package is fairly-well contained, I think: most of the effort will be in reverse-engineering the intent of the edge-case tests. I made a sketch at what these changes would look for the zope.event package like on a branch in Subversion: http://svn.zope.org/zope.event/branches/tseaver-new_style_docs I select zope.event largely because it is small enough that the scope of the structural changes wouldn't be lost in the diffs for the actual tests (and also because I didn't want to do the work for a bigger package before getting feedback). The rendered docs from that branch are here: http://palladion.com/static/zope.event-docs/ I pushed the bzr branch to Launchpad, to let you see the kinds of changes I made to get there (our viewcvs story is not nearly as easy to review as the Launchpad changeset view): https://code.launchpad.net/~tseaver/zope.event/new_style_docs I am attaching an annotated diff for those who would prefer to see the changes in that format. I would say that this cleanup effort should start at the bottom layers of the ZTK, dependency-wise (zope.interface, zope.component, etc.) and move upwards (in the depenency sense) over time. Thoughts? +1 Wichert. -- Wichert Akkerman wich...@wiggy.net It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
Hi! Tres Seaver wrote: The refactoring I would like to see happen is to move the main narrative documentation into a separate, Sphinx-driven 'docs' directory in each ZTK package. As part of this move, we can start adding some of the really nice Sphinx features (cross references, indexing, auto-generation of API reference information, etc.). We can still leave the main line of the code illustrated using doctest-style blocks: Sphinx has nice support for testing those blocks *while building the docs*. [...] Thoughts? +1 in general, but I'm concerned about one thing: How can we make sure docs and code don't get out of sync? Do we have to run unittests *and* build the docs before each checkin? Will someone make sure buildbots and nightly tests report broken docs as well? Cheers, Yuppie ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On Sat, Apr 17, 2010 at 11:18, yuppie y.2...@wcm-solutions.de wrote: How can we make sure docs and code don't get out of sync? Do we have to run unittests *and* build the docs before each checkin? Will someone make sure buildbots and nightly tests report broken docs as well? Hm... As long as you use the syntax and none of the ..test-block:: syntax of sphinx, the sphinx docs can be tested as a part of the standard testruns, but I guess part of this proposal was so we could easily use the testblock syntax as it has the possibility to hide setup etc easier. And then I don't know the best way to integrate it, but it would definitely be a drawback if you need two different testruns to run all the tests... Also I think it should be the responsibility of the one who does a release to make sure that we can build HTML docs and that the formatting isn't *too* broken. Preferably these docs should also be uploaded to packages.python.org, maybe there can be a makefile step for that in the docs? -- Lennart Regebro: Python, Zope, Plone, Grok http://regebro.wordpress.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 10 OK, 4 Failed
Summary of messages to the zope-tests list. Period Fri Apr 16 12:00:00 2010 UTC to Sat Apr 17 12:00:00 2010 UTC. There were 14 messages: 6 from Zope Tests, 7 from ccomb at free.fr, 1 from ct at gocept.com. Test failures - Subject: FAILED: Repository policy check found errors in 668 projects From: ct at gocept.com Date: Fri Apr 16 21:14:57 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013985.html Subject: FAILED : ZTK 1.0dev / Python2.4.6 Linux 32bit From: ccomb at free.fr Date: Fri Apr 16 23:56:01 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013996.html Subject: FAILED : ZTK 1.0dev / Python2.5.2 Linux 32bit From: ccomb at free.fr Date: Fri Apr 16 23:57:19 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013997.html Subject: FAILED : ZTK 1.0dev / Python2.6.4 Linux 32bit From: ccomb at free.fr Date: Fri Apr 16 23:57:31 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013998.html Tests passed OK --- Subject: OK : Zope-2.10 Python-2.4.6 : Linux From: Zope Tests Date: Fri Apr 16 21:29:29 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013986.html Subject: OK : Zope-2.11 Python-2.4.6 : Linux From: Zope Tests Date: Fri Apr 16 21:31:29 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013987.html Subject: OK : Zope-2.12 Python-2.6.4 : Linux From: Zope Tests Date: Fri Apr 16 21:33:29 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013988.html Subject: OK : Zope-2.12-alltests Python-2.6.4 : Linux From: Zope Tests Date: Fri Apr 16 21:35:29 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013989.html Subject: OK : Zope-trunk Python-2.6.4 : Linux From: Zope Tests Date: Fri Apr 16 21:37:29 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013990.html Subject: OK : Zope-trunk-alltests Python-2.6.4 : Linux From: Zope Tests Date: Fri Apr 16 21:39:29 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013991.html Subject: OK : BlueBream template / Python2.5.2 32bit linux From: ccomb at free.fr Date: Fri Apr 16 22:00:47 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013992.html Subject: OK : BlueBream template / Python2.4.6 32bit linux From: ccomb at free.fr Date: Fri Apr 16 22:00:47 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013993.html Subject: OK : BlueBream template / Python2.6.4 32bit linux From: ccomb at free.fr Date: Fri Apr 16 22:00:49 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013995.html Subject: OK : BlueBream template / Python2.7b1 32bit linux From: ccomb at free.fr Date: Fri Apr 16 22:00:49 EDT 2010 URL: http://mail.zope.org/pipermail/zope-tests/2010-April/013994.html ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 17 April 2010 10:41, Lennart Regebro rege...@gmail.com wrote: On Sat, Apr 17, 2010 at 11:18, yuppie y.2...@wcm-solutions.de wrote: How can we make sure docs and code don't get out of sync? Do we have to run unittests *and* build the docs before each checkin? Will someone make sure buildbots and nightly tests report broken docs as well? Hm... As long as you use the syntax and none of the ..test-block:: syntax of sphinx, the sphinx docs can be tested as a part of the standard testruns, but I guess part of this proposal was so we could easily use the testblock syntax as it has the possibility to hide setup etc easier. And then I don't know the best way to integrate it, but it would definitely be a drawback if you need two different testruns to run all the tests... Also I think it should be the responsibility of the one who does a release to make sure that we can build HTML docs and that the formatting isn't *too* broken. Preferably these docs should also be uploaded to packages.python.org, maybe there can be a makefile step for that in the docs? It's important that documentation is tested as part of the standard test run, that means when you change something you know to update the docs. repoze.bfg seemed to make an attempt at this, though it is currently disabled. http://svn.repoze.org/repoze.bfg/trunk/repoze/bfg/tests/test_docs.py Laurence ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On Sat, Apr 17, 2010 at 16:05, Laurence Rowe l...@lrowe.co.uk wrote: It's important that documentation is tested as part of the standard test run, that means when you change something you know to update the docs. repoze.bfg seemed to make an attempt at this, though it is currently disabled. http://svn.repoze.org/repoze.bfg/trunk/repoze/bfg/tests/test_docs.py How could I forget about Manuel. *facepalm*. I've written Manuel extensions to test sphinx tests, but Benji hasn't had time to review it yet: http://svn.zope.org/manuel/branches/sphinx/ So it's perfectly possible to run all types of Sphinx tests even with codeblocks, as a part of the standard testrun. -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Lennart Regebro wrote: On Sat, Apr 17, 2010 at 16:05, Laurence Rowe l...@lrowe.co.uk wrote: It's important that documentation is tested as part of the standard test run, that means when you change something you know to update the docs. repoze.bfg seemed to make an attempt at this, though it is currently disabled. http://svn.repoze.org/repoze.bfg/trunk/repoze/bfg/tests/test_docs.py How could I forget about Manuel. *facepalm*. I've written Manuel extensions to test sphinx tests, but Benji hasn't had time to review it yet: http://svn.zope.org/manuel/branches/sphinx/ So it's perfectly possible to run all types of Sphinx tests even with codeblocks, as a part of the standard testrun. I'm ambivalent about testing the Sphinx code snippets on each test run. I want those snippets to be *much* less comprehensive than they are currently, and am pretty sure that drift in the non-executable bits is at least as important a problem as drift in the snippets. Basically, bugfix changeses shouldn't need to run the documentation tests, because the visible / main-line behavior isn't supposed to change at all. Feature changes should *start* by writing the narrative docs about the added feature: feature branches which don't include updates to the docs shouldn't be merged. Running the snippet tests as part of making a release (or even via some sort of '--all' argument for nightly runners) seems pretty adequate to me. If we upgrade our testing style to keep each testcase tightly focused, with meaningful names for the methods and local varioables, it should be much easier to figure out what, if anything, got overlooked in the documentaion. +0 on running them every time IFF we can find an easy way to run them during 'setup.py test' as well as buildout, and we continue to be mindful of the distinction between docs that work primarily as docs and tests that cover every code path. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvJ7Q0ACgkQ+gerLs4ltQ7QAwCfUZfetKf3ZBdCa4AZ/G+02jab w5AAn2ZkjA5eevc+clc4Jre/IHSk7SH0 =rdQf -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On Sat, Apr 17, 2010 at 19:17, Tres Seaver tsea...@palladion.com wrote: I'm ambivalent about testing the Sphinx code snippets on each test run. I want those snippets to be *much* less comprehensive than they are currently, and am pretty sure that drift in the non-executable bits is at least as important a problem as drift in the snippets. Well, unless they take of time, but they shouldn't, really. I think it's positive to get a quick feedback if you are breaking the documentation. +0 on running them every time IFF we can find an easy way to run them during 'setup.py test' as well as buildout, and we continue to be mindful of the distinction between docs that work primarily as docs and tests that cover every code path. I think manuel should solve that, we'll see, I'll have to test. I need to get manuel ported to Python 3 then too. :) -- Lennart Regebro: http://regebro.wordpress.com/ Python 3 Porting: http://python-incompatibility.googlecode.com/ +33 661 58 14 64 ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Lennart Regebro wrote: On Sat, Apr 17, 2010 at 19:17, Tres Seaver tsea...@palladion.com wrote: I'm ambivalent about testing the Sphinx code snippets on each test run. I want those snippets to be *much* less comprehensive than they are currently, and am pretty sure that drift in the non-executable bits is at least as important a problem as drift in the snippets. Well, unless they take of time, but they shouldn't, really. I think it's positive to get a quick feedback if you are breaking the documentation. Again, I don't mind that part, but I want to break the cycle of jamming crap (for documentation purposes) into the docs for purposes of getting test coverage. +0 on running them every time IFF we can find an easy way to run them during 'setup.py test' as well as buildout, and we continue to be mindful of the distinction between docs that work primarily as docs and tests that cover every code path. I think manuel should solve that, we'll see, I'll have to test. I need to get manuel ported to Python 3 then too. :) Turtles all the way down. ;) Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvKIHgACgkQ+gerLs4ltQ7foQCfTzBh8IGTr8beH/H+vdSLrAI5 TdEAoJWNJ1w/GQaVoMITw1siOxKaRJZj =ns67 -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On Fri, Apr 16, 2010 at 9:41 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This kind of goes with Lennart's frustration about trying to port the ZTK packages, or a core subset, to Python 3. I would like to see the ZTK packages have really excellent documentation, as well as 100% test coverage. I think we need to look at refactoring how the testing story is done to get both needs satisfied: trying to achieve both with a single set of doctests is pretty near impossible: - - Testing setup and teardown code obscures the narrative flow / intent of the documentation. - - Edge case testing *really* obscures the same. - - Test coverage suffers, because it is hard to write edge-case tests in doctest. - - Porting the doctests where people have tried to juggle both goals across Python versions is a mess. The refactoring I would like to see happen is to move the main narrative documentation into a separate, Sphinx-driven 'docs' directory in each ZTK package. As part of this move, we can start adding some of the really nice Sphinx features (cross references, indexing, auto-generation of API reference information, etc.). We can still leave the main line of the code illustrated using doctest-style blocks: Sphinx has nice support for testing those blocks *while building the docs*. At the end of this part of the change, each package would have a much improved documentation set, with tested examples. At the end of the process, we would be able to put the docs for each package into an intersphinx tree underneath the main ZTK docs, which could serve as the gateway into them. The trickier testing bits we would re-write as super thorough, no shortcuts-taken unit tests: one testcase class per class (or API function) under test, at least one method per class-under-test method, with more preferred to get all code paths / preconditions covered. The goal here would be to comment out the doctests from the test_suites and get the code to 100% coverage using pure unit tests. Meanwhile, the doctests would be refactored into the Sphinx documentation, losing all the bits which obscure their intent as documentation. I would say that the risks here are slight, given that aiming for 100% test coverage is likely to catch any oversights made during porting to the new style. The amount of work for any given package is fairly-well contained, I think: most of the effort will be in reverse-engineering the intent of the edge-case tests. I made a sketch at what these changes would look for the zope.event package like on a branch in Subversion: http://svn.zope.org/zope.event/branches/tseaver-new_style_docs I select zope.event largely because it is small enough that the scope of the structural changes wouldn't be lost in the diffs for the actual tests (and also because I didn't want to do the work for a bigger package before getting feedback). The rendered docs from that branch are here: http://palladion.com/static/zope.event-docs/ I pushed the bzr branch to Launchpad, to let you see the kinds of changes I made to get there (our viewcvs story is not nearly as easy to review as the Launchpad changeset view): https://code.launchpad.net/~tseaver/zope.event/new_style_docs I am attaching an annotated diff for those who would prefer to see the changes in that format. I would say that this cleanup effort should start at the bottom layers of the ZTK, dependency-wise (zope.interface, zope.component, etc.) and move upwards (in the depenency sense) over time. Thoughts? I agree with the main thrust. (Of course, I disagree with some of your judgments of doctest, but that's mostly beside the point.) - Documentation should be written for documentation's sake. The emphasis should be on helping people understand what the software is for and how to use it, *not* on coverage. - Documentation should be executable. Manuel helps a lot for this. - I can almost guarantee that any examples that aren't tested will be wrong. I tried to do a good job on the bobo docs. I made almost all of the examples executable, and the ones I didn't had an amazing tendency to have bugs. - I agree that tests should be separate from documentation. You can get some of your coverage from the docs, but you'll need tests for edge cases and details not addressed by the docs. Jim -- Jim Fulton ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jim Fulton wrote: On Fri, Apr 16, 2010 at 9:41 PM, Tres Seaver tsea...@palladion.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 This kind of goes with Lennart's frustration about trying to port the ZTK packages, or a core subset, to Python 3. I would like to see the ZTK packages have really excellent documentation, as well as 100% test coverage. I think we need to look at refactoring how the testing story is done to get both needs satisfied: trying to achieve both with a single set of doctests is pretty near impossible: - - Testing setup and teardown code obscures the narrative flow / intent of the documentation. - - Edge case testing *really* obscures the same. - - Test coverage suffers, because it is hard to write edge-case tests in doctest. - - Porting the doctests where people have tried to juggle both goals across Python versions is a mess. The refactoring I would like to see happen is to move the main narrative documentation into a separate, Sphinx-driven 'docs' directory in each ZTK package. As part of this move, we can start adding some of the really nice Sphinx features (cross references, indexing, auto-generation of API reference information, etc.). We can still leave the main line of the code illustrated using doctest-style blocks: Sphinx has nice support for testing those blocks *while building the docs*. At the end of this part of the change, each package would have a much improved documentation set, with tested examples. At the end of the process, we would be able to put the docs for each package into an intersphinx tree underneath the main ZTK docs, which could serve as the gateway into them. The trickier testing bits we would re-write as super thorough, no shortcuts-taken unit tests: one testcase class per class (or API function) under test, at least one method per class-under-test method, with more preferred to get all code paths / preconditions covered. The goal here would be to comment out the doctests from the test_suites and get the code to 100% coverage using pure unit tests. Meanwhile, the doctests would be refactored into the Sphinx documentation, losing all the bits which obscure their intent as documentation. I would say that the risks here are slight, given that aiming for 100% test coverage is likely to catch any oversights made during porting to the new style. The amount of work for any given package is fairly-well contained, I think: most of the effort will be in reverse-engineering the intent of the edge-case tests. I made a sketch at what these changes would look for the zope.event package like on a branch in Subversion: http://svn.zope.org/zope.event/branches/tseaver-new_style_docs I select zope.event largely because it is small enough that the scope of the structural changes wouldn't be lost in the diffs for the actual tests (and also because I didn't want to do the work for a bigger package before getting feedback). The rendered docs from that branch are here: http://palladion.com/static/zope.event-docs/ I pushed the bzr branch to Launchpad, to let you see the kinds of changes I made to get there (our viewcvs story is not nearly as easy to review as the Launchpad changeset view): https://code.launchpad.net/~tseaver/zope.event/new_style_docs I am attaching an annotated diff for those who would prefer to see the changes in that format. I would say that this cleanup effort should start at the bottom layers of the ZTK, dependency-wise (zope.interface, zope.component, etc.) and move upwards (in the depenency sense) over time. Thoughts? I agree with the main thrust. (Of course, I disagree with some of your judgments of doctest, but that's mostly beside the point.) I'm trying hard to keep my prejudices out of the discussion;). - Documentation should be written for documentation's sake. The emphasis should be on helping people understand what the software is for and how to use it, *not* on coverage. Amen. - Documentation should be executable. Manuel helps a lot for this. Amen. The Sphinx tools for this are pretty neat, too. - I can almost guarantee that any examples that aren't tested will be wrong. I tried to do a good job on the bobo docs. I made almost all of the examples executable, and the ones I didn't had an amazing tendency to have bugs. I'm not against having the snippets be executable, because I *do* want them to work. I just don't want to encourage anyone to think that they are testing the software when they write the snippets, or execute them. Executing the snippets is testing the documentation, not the software. - I agree that tests should be separate from documentation. You can get some of your coverage from the docs, but you'll need tests for edge cases and details not addressed by the docs. I would actually prefer to measure coverage (objectively) without reference to the snippets
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 4/17/10 5:20 PM, Tres Seaver wrote: - Documentation should be written for documentation's sake. The emphasis should be on helping people understand what the software is for and how to use it, *not* on coverage. Amen. - Documentation should be executable. Manuel helps a lot for this. Amen. The Sphinx tools for this are pretty neat, too. Documentation needn't be executable. Documentation should be right. But sometimes documentation is just documentation and automating the checking of its rightness isn't reasonable, especially if making it executable takes the slightest bit of meaning away from it as documentation. Sorry. No free lunch here. - C ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] KGS 3.4.1 versions
Christophe Combelles a écrit : Roger a écrit : Hi Betreff: Re: [Zope-dev] KGS 3.4.1 versions Adam GROSZER a écrit : Hello, There is a sheet with versions for KGS 3.4.1 http://spreadsheets.google.com/pub?key=tUE5Q72d4Kg1FXaacCA3EKQoutput= html Anyone for/against those versions? The open questions that remain: * What about pytz 2010g? * Which lxml version to take? 1.3.6? * What about zope.app.container 3.6.2? * Would be nice to have zope.testbrowser 3.5.1 Comments are welcome. z3c.layer has a major security issue, because of trusted traversing adapters that removes the security proxy everywhere. yes and no, only miss use could end in security issues It's not really a security issue, it's the only concept which allows to use nested sites with more then one IAuthentication utility and allows to authenticate on objects behind the first site. But since this was such a rare use case, we decided to split the package in different packages which also supports a non trusted setup. This makes the packages more general usable without to run into security issues based on trusted confirgurations where non trusted is needed. This package has been retired and splitted into its 3 subpackages : z3c.layer.minimal z3c.layer.pagelet Both package above should not use trusted traverser z3c.layer.trusted This package should still use trusted traverser There is no problem upgrading to branch 1.0 of these packages, as they don't have any significant changes, excepted the splitting. However: z3c.layer.pagelet should be in version 1.0.2. Nothing below. z3c.layer.minimal has no corrected 1.0 branch. A new maintenance release 1.0.2 of this package should be released. z3c.layer.trusted is OK, since this is trusted in purpose. (I think) Yes Ok thanks, I'll release z3c.layer.minimal during the WE. I've released z3c.layer.minimal 1.0.2 with the fix, and z3c.layer 0.2.4 with the same fix. For the KGS 3.4.1, we just have to upgrade z3c.layer to 0.2.4. No need to add z3c.layer.[pagelet|minimal|trusted] Christophe Regards Roger Ineichen Christophe ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Proposed new style for documenting and testing ZTK packages
On 18 April 2010 05:20, Tres Seaver tsea...@palladion.com wrote: I'm not against having the snippets be executable, because I *do* want them to work. I just don't want to encourage anyone to think that they are testing the software when they write the snippets, or execute them. Executing the snippets is testing the documentation, not the software. Everyone who's ever written a doctest, or is thinking about doing so: Please frame this and hang it above your monitor. :-) Martin ___ Zope-Dev maillist - Zope-Dev@zope.org https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )
