Re: [Zope-dev] Dependencies for ZCML
Hi I would like to chime in here on the zcml I have managed to get a core stack of zope3 running on gae had to hack a lot of zope.security and zope.proxy to get it there, but it all works quite well I found I had to ignore zope.configuration to get most of the base stack working because the default configure.zcml in most packages introduced a lot of dependancies that weren't always necessary at least for me. In most cases I manually and selectively setup a whole lot of provideAdapter setups to get a minimal working set, Later I found I went back to get a minimal zope.configuration going, becuase it became way to much effort to register all of the base z3c.form components (widgets, dataconverters and terms) that I wanted registered. How is this relevant to to discussion, I found reading the unit tests didn't really enlighten me how many of the packages should be registered in the broader framework and had to rely on the zcml to work it out, however the zcml often pulled in dependancies that where not obvious from other packages because the various registrations and order of registrations that are performed in a normal zope deployment. This means the zcml is pretty important description of how the components are used in the broader context, it also means I think tests that are dependant on the zcml registration working is important, however the downside is the zcml does usually bring in things like browser views, which you may not want and so need to then gut some of the zml. Just my 2c worth T P.S. Sorry about the top posting, I though I was subscribed to the list, but wasn't so didn't have the original posting to reply to ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Tres Seaver wrote: [snip] Please don't put words in my mouth. I *do* care that the mega-frameworks succeed. My apologies, I got this impression because you very much want to frame the debate in terms of libraries, but I understand that is also a way to try to improve the whole. However, I don't think that blessing the current usage is going to help them succeed in the long run. I think that moving the shared configuration bits out of library packages ought to be a fairly high-priority, near-term goal, in order to increase the quality / reusability of the library packages, which will also increase the quality / stability of the mega-frameworks, and reduce the burden of maintaining both. I think the tight coupling of scattered + required ZCML has turned out to be a net loss over time, because it forces people to make an all-or-nothing choice about adopting Zope early on in their evaluation. Making it attracive and easy to use pieces of Zope, while deferring that all-in bet, is a big driver for the purism you are describing. I agree that going into a more library-like direction makes sense. I have caveats as I can think of packages that behave like libraries from one perspective but like frameworks from another. But I do agree we should look into getting more library-like reuse out of our packages. [snip] Here you sketch out a program: For instance, if we provided for each mega-framework a single everything {Grok,Zope2,Zope3ZMI} needs from the Zope framework package, which named all the appropriate dependencies *and* provided the shared ZCML, and then switched each mega-framework and its applications to use that package, we could remove the ZCML from all the other packages (except for BBB). In fact that single package would *be* the mega-framework at that point. Once we had such packages, we could look at whether factoring out some of the common dependencies / ZCML from each of them into one or more shared Zope Framework ZCML package would be worthwhile. The choice to do such a refactoring would be transparent to existing applications built on the mega-frameworks. New applications might choose to target one or more of the subset packages, or not. I'm for such a program in principle. We can't do it in one big step; we'll have to review each package and rewrite tests and do it bit by bit and see where it makes sense and what issues we run into. We also very much don't want these ZCML packages to be like zope.app.zcmlfiles, which is cluttering up the dependency works. I'll add again my caveat about losing abstraction if all test writers have to use fairly low level component configuration for some high-level registrations, and we require such knowledge from the writers of tests. I wish there were a higher-level API that could be used to do common registrations for views, subscribers, etc. Especially views can be configured in non-trivial ways by browser:page and I'm worried people will need to learn a bit much about the details of this. We should simplify that of course. But in general I have a concern about losing an abstraction layer. I wish we had some form of answer to this. With grokcore.component, you can in your tests simply decide to grok a class or not to configure it. If that class is a grokcore.component.Adapter (for instance), it'll be registered. This introduces a dependency on grokcore.component. This would add a dependency everywhere but it is higher level. Once the dependencies for grokcore.view are rooted out, you could do the same for views. I'm not proposing we rewrite all code to use grokcore.component or grokcore.view; I'm just indicating that having an easy way to register a component can be easy. But that shouldn't block the effort; I'm sure many packages that use ZCML in their tests can be rewritten today to use zope.component registrations in their tests without too much effort and if people want to do that, we should go ahead. I've heard your purist opinions in this area before. It's not very helpful in the way presented. If you want us to buy into your opinions you'll have to make them more attractive to us, and you know about our concerns as a community. Hmm, I'm not sure how to reply to that. I'll keep trying to clarify my positions, including removing any unintended heat, and supplying any extra considerations to address your concerns. What I'm trying to get across is that besides positions we need a way forward to improve the situation. This approach isn't obvious to all of us, you know. :) You started out sketching such an approach, and that's good. I think a next step is to try modifying a package in the way you envision and discuss the consequences and results here. We'd need a place to store the ZCML that's pulled out too. Thanks for sticking with it! Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org
Re: [Zope-dev] Dependencies for ZCML
Hey, Carsten Senger wrote: [snip] zcml contains many useful informations and I often use it as documentation how things fit together. It would be a loss to detach all zcml from the implementations into one/few big zcml packages. Moving them into one dedicated zcml for every package leaves them logically related to the implementation. It's also easier to maintain: - The zcml for an implementation has the same release cycle as the implementation. - Every relevant change in an implementation would need changes by a number of zcml package maintainers (Grok, Zope2, Zope3ZMI) that don't know the package nearly as good as the package maintainer. I would prefer to find them inside the implementation packages where possible. Where it's intended to reduce dependencies a dedicated zcml package like zope.i18n_zcml is at least more clear. You make good points in favor of keeping this information close together. I'll note that with grok-style packages it's actually not *possible* to remove the information out completely, as it's in the Python code. You can choose not to configure this code at all, but that's it. I think whether ZCML is useful depends, in part, on much on how much ZCML a package defines today. If a package defines a little bit of ZCML only, I think it's less of a risk to pull it out. If a package defines a *lot* of ZCML, we will have to wonder about the purpose of the package (is this really a library-like package or more like an application defining a UI or something?), and we'll have to think about another strategy. I want to move this discussion to concrete examples next and see what a treatment of moving out ZCML would entail. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
On Tuesday 17 March 2009, Martijn Faassen wrote: If a package defines a *lot* of ZCML, we will have to wonder about the purpose of the package (is this really a library-like package or more like an application defining a UI or something?), and we'll have to think about another strategy. I want to move this discussion to concrete examples next and see what a treatment of moving out ZCML would entail. A common complaint I am hearing about z3c.form, for example, is that I should use the ZCML more in my doctests, so that the doctests contain more example on how to use ZCML to create z3c.form pages. At work I have started taking this approach more and more and it makes tests actually more readable and provides better documentation to consumers of the libraries/code. Regards, Stephan -- Stephan Richter Web Software Design, Development and Training Google me. Zope Stephan Richter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Hey Tim, Tim Hoffman wrote: I would like to chime in here on the zcml [snip] Thanks for this balanced view which gives points that can be used to support both sides in the discussion: * today, the ZCML is very useful to understand how a package is supposed to be put together. Removing the ZCML will make this harder. * today, the ZCML pulls in too many dependencies that you don't really need; you want to be able to supply your own configuration for particular setups. [snip] This means the zcml is pretty important description of how the components are used in the broader context, it also means I think tests that are dependant on the zcml registration working is important, however the downside is the zcml does usually bring in things like browser views, which you may not want and so need to then gut some of the zml. I'll note that our dependency refactoring project will hopefully allow you to reuse more of a package's ZCML, as each package itself has a more focused goal (and no UI). It's my opinion that the dependency refactoring project is the most important we have going on right now. Whether we should be pulling out ZCML or not from these smaller packages will need a few experiments to work out the full consequences of such a procedure. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Stephan Richter wrote: On Tuesday 17 March 2009, Martijn Faassen wrote: If a package defines a *lot* of ZCML, we will have to wonder about the purpose of the package (is this really a library-like package or more like an application defining a UI or something?), and we'll have to think about another strategy. I want to move this discussion to concrete examples next and see what a treatment of moving out ZCML would entail. A common complaint I am hearing about z3c.form, for example, is that I should use the ZCML more in my doctests, so that the doctests contain more example on how to use ZCML to create z3c.form pages. At work I have started taking this approach more and more and it makes tests actually more readable and provides better documentation to consumers of the libraries/code. I have had the same realisation. One possible compromise if you don't want to actually use the XML configuration language, is to show an example ZCML file, but to actually register with the zope.component API. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Hey, Tres Seaver wrote: [snip] Doesn't that in some cases make tests harder to understand, as lower-level APIs are in use that are not as recognizable as the equivalent ZCML directives? (say, registering an event) Don't we place a burden on the test writers to learn these APIs while they could use the ones abstracted away behind ZCML instead? No. Relying on real ZCML in testing, as is using the real components is an anti-pattern: it makes tests fragile, couples the packages tightly, etc. Huh? You can't be serious saying there is not extra burden on the test developers if you require them to learn about the implementation of various ZCML directives in order to write a test. The burden is in learning how a view is registered, or how a subscriber is registered, in order to write a test. You need them to break through the abstraction that ZCML provides in order to write a test. It will also make it harder to read the tests as the reader will need to share this understanding as well. You can't just go and say it's an anti-pattern without giving an alternative, otherwise people will continue to use the higher-level of abstraction for registration (i.e. ZCML). [snip] I don't think library packages have ZCML in them at all, except as examples, because trying to reusing ZCML doesn't actually win unambiguosly over copying it. Here's my position on this: You take a hard-line purist opinion. We may want to take an attitude like this for the Zope Framework libraries, eventually. We cannot just do this by throwing out all ZCML from our packages. Why not? Because the Zope community is in the business of providing an integrated experience too (Zope 2, Zope 3 and Grok), like it or not. (hold on, I know you don't care about this. I'm getting to this) This means that we, as a community on zope-dev care about configuration (no matter where it is maintained). Since we do, we should maintain and test it. Since we're a community and care about compatibility it's good to share the burden of maintaining and distributing this configuration, instead of just ignoring this and deferring it to the individual projects. Today, the shared configuration project is scattered over the individual packages in individual zcml files that refer to each other. If we want this project elsewhere, we need to take realistic, active evolutionary steps to get there, package by package. We can't just drop the ball on this, as we have projects depending on this information. I know you don't care one bit about such projects yourself. You just care about the libraries. Fine, but you'll have to acknowledge that other people *do* care about this project. They have frameworks and applications to maintain that need the configuration scattered through the Zope Framework code base right now. I've heard your purist opinions in this area before. It's not very helpful in the way presented. If you want us to buy into your opinions you'll have to make them more attractive to us, and you know about our concerns as a community. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Am 12.03.2009 um 19:25 schrieb Tres Seaver: [...] Now when testing these libraries you could do three things: * not use ZCML at all and recreate the effect of these registrations in Python code. +1. * use the ZCML in the package's configure.zcml. (perhaps through ftesting.zcml) - -lots. I disagree: if there is ZCML inside the package it must be tested inside the package. If there is a syntax errors in the zcml, tests do not find it. It is even possible to make a release without noticing this defect. I think that's really bad. Otherwise we should we require to run the compat-tests before releasing a package of the zope framework. This might find the errors in zcml, maybe. Yours sincerely, -- Michael Howitz · m...@gocept.com · software developer gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martijn Faassen wrote: Hey, Tres Seaver wrote: [snip] Doesn't that in some cases make tests harder to understand, as lower-level APIs are in use that are not as recognizable as the equivalent ZCML directives? (say, registering an event) Don't we place a burden on the test writers to learn these APIs while they could use the ones abstracted away behind ZCML instead? No. Relying on real ZCML in testing, as is using the real components is an anti-pattern: it makes tests fragile, couples the packages tightly, etc. Huh? You can't be serious saying there is not extra burden on the test developers if you require them to learn about the implementation of various ZCML directives in order to write a test. The burden is in learning how a view is registered, or how a subscriber is registered, in order to write a test. You need them to break through the abstraction that ZCML provides in order to write a test. It will also make it harder to read the tests as the reader will need to share this understanding as well. You can't just go and say it's an anti-pattern without giving an alternative, otherwise people will continue to use the higher-level of abstraction for registration (i.e. ZCML). The alternative is that developers use 'zope.component.provide*' rather than loading ZCML. The upsides are: - - They can register stub implementations, some of which may not even be importable (e.g., local closures). - - They can avoid testing dependencies on 'zope.configuration' and friends. - - The tests run faster. - - The tests run cleaner. [snip] I don't think library packages have ZCML in them at all, except as examples, because trying to reusing ZCML doesn't actually win unambiguosly over copying it. Here's my position on this: You take a hard-line purist opinion. We may want to take an attitude like this for the Zope Framework libraries, eventually. We cannot just do this by throwing out all ZCML from our packages. Why not? Because the Zope community is in the business of providing an integrated experience too (Zope 2, Zope 3 and Grok), like it or not. (hold on, I know you don't care about this. I'm getting to this) This means that we, as a community on zope-dev care about configuration (no matter where it is maintained). Since we do, we should maintain and test it. Since we're a community and care about compatibility it's good to share the burden of maintaining and distributing this configuration, instead of just ignoring this and deferring it to the individual projects. Today, the shared configuration project is scattered over the individual packages in individual zcml files that refer to each other. If we want this project elsewhere, we need to take realistic, active evolutionary steps to get there, package by package. We can't just drop the ball on this, as we have projects depending on this information. I know you don't care one bit about such projects yourself. You just care about the libraries. Please don't put words in my mouth. I *do* care that the mega-frameworks succeed. However, I don't think that blessing the current usage is going to help them succeed in the long run. I think that moving the shared configuration bits out of library packages ought to be a fairly high-priority, near-term goal, in order to increase the quality / reusability of the library packages, which will also increase the quality / stability of the mega-frameworks, and reduce the burden of maintaining both. I think the tight coupling of scattered + required ZCML has turned out to be a net loss over time, because it forces people to make an all-or-nothing choice about adopting Zope early on in their evaluation. Making it attracive and easy to use pieces of Zope, while deferring that all-in bet, is a big driver for the purism you are describing. Fine, but you'll have to acknowledge that other people *do* care about this project. They have frameworks and applications to maintain that need the configuration scattered through the Zope Framework code base right now. I'll draw a semantic distinction, based on the grammatical ambiguity in that sentence: those applications need the configuration, but they don't need the configuration to be scattered through the code base, except for BBB purposes. For instance, if we provided for each mega-framework a single everything {Grok,Zope2,Zope3ZMI} needs from the Zope framework package, which named all the appropriate dependencies *and* provided the shared ZCML, and then switched each mega-framework and its applications to use that package, we could remove the ZCML from all the other packages (except for BBB). In fact that single package would *be* the mega-framework at that point. Once we had such packages, we could look at whether factoring out some of the common dependencies / ZCML from each of them into one or more shared Zope Framework ZCML
Re: [Zope-dev] Dependencies for ZCML
Am 11.03.2009 um 21:26 schrieb Dan Korostelev: 2009/3/11 Martijn Faassen faas...@startifact.com: Oh, and on the topic, one more time: can we have a steering group decision on the package requirements for zcml statements? Are we doing extras for them or simply skipping them? Sorry, I wasn't clear that there was an open question and I'm afraid I don't understand this one. :) Could you point me to the appropriate thread that was left in the middle, or could you start a new thread with a description of the open question? I'm too lazy now to search in archives, so I'll just describe again. For example, the zope.password package only requires zope.interface to be functional. But it's configure.zcml contains directives that need zope.component (or repoze.zcml) and zope.security. Also, the zcml thing itself needs zope.configure as well. Should we mention it in extra dependencies somehow or just document it, saying that zcml is intented to be used in more zope3-ish environment that already has needed packages, so others can simply ignore these files. zope.container has a similar problem: its configure.zcml uses zope:view directives. When I'd like to use zope.container in a Zope 3 the application server environment I have to know that zope:view is defined in zope.app..component or I have to find it out. There is no dependency, not test and no documentation mentioning this inside zope.container. I think that's bad as it makes it more difficult to learn Zope for new developers. Yours sincerely, -- Michael Howitz · m...@gocept.com · software developer gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 8 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Hi Tres, Tres Seaver schrieb: For instance, if we provided for each mega-framework a single everything {Grok,Zope2,Zope3ZMI} needs from the Zope framework package, which named all the appropriate dependencies *and* provided the shared ZCML, and then switched each mega-framework and its applications to use that package, we could remove the ZCML from all the other packages (except for BBB). In fact that single package would *be* the mega-framework at that point. zcml contains many useful informations and I often use it as documentation how things fit together. It would be a loss to detach all zcml from the implementations into one/few big zcml packages. Moving them into one dedicated zcml for every package leaves them logically related to the implementation. It's also easier to maintain: - The zcml for an implementation has the same release cycle as the implementation. - Every relevant change in an implementation would need changes by a number of zcml package maintainers (Grok, Zope2, Zope3ZMI) that don't know the package nearly as good as the package maintainer. I would prefer to find them inside the implementation packages where possible. Where it's intended to reduce dependencies a dedicated zcml package like zope.i18n_zcml is at least more clear. ..Carsten ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Tres Seaver wrote at 2009-3-11 21:27 -0400: ... In packages that don't load their own ZCML during the tests, it's harder to say. One reaction could be that this package doesn't have enough tests then! Of course another would argue that this is configuration information only that can be overridden, but it is rather special configuration... - -1 on having configuration dependecies, including having mandatory tests that ZCML will load: mandatory configuration is a contradiction in terms. I therefore don't believe that tests which try to load ZCML are useful, at least for library pacakges (as opposed to applications). I do not agree with you here. *IF* a package contains ZCML files (e.g. in order to facilitate a standard integration into a larger Zope 3 system), then these ZCML files should of course be tested -- even when there are conceivable package uses that do not use the ZCML files. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Tres Seaver wrote at 2009-3-12 14:25 -0400: ... Sorry, I meant mandatory tests which load ZCML. I'm actually against ever loading ZCML in tests at all. If you ship ZCML, you should test it, no? You will not ship ZCML, but this may not apply to everybody -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dieter Maurer wrote: Tres Seaver wrote at 2009-3-12 14:25 -0400: ... Sorry, I meant mandatory tests which load ZCML. I'm actually against ever loading ZCML in tests at all. If you ship ZCML, you should test it, no? Not necessarily: in fact, if testing it means that users of my package have to accept a big dogpile of dependencies that they otherwise wouldn't need to, then no. I can easily imagine shipping ZCML in the form I outline below in a package which itself doesn't depend on zope.configuration at all, for instance. You will not ship ZCML, but this may not apply to everybody I likely won't ship ZCML inside a reusable library (as opposed to a deployable application): if I do, it will probably be not a convenience that I make everybody include (not my definition of convenience, anyway) but an example which I expect they can either include or *copy and change*. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJusAW+gerLs4ltQ4RAv0wAKCJg83M2fgdaunXS9b8PZysJDHLrgCg0MHQ 6+VxRild3DMB5y8PYJ7RK1s= =a3Ax -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Tres Seaver wrote at 2009-3-13 16:20 -0400: ... Dieter Maurer wrote: Tres Seaver wrote at 2009-3-12 14:25 -0400: ... Sorry, I meant mandatory tests which load ZCML. I'm actually against ever loading ZCML in tests at all. If you ship ZCML, you should test it, no? Not necessarily: in fact, if testing it means that users of my package have to accept a big dogpile of dependencies that they otherwise wouldn't need to, then no. Your and my quality principles diverge: If I release a package, then its tests have the verify that the package contents work correctly. This implies: the tests should cover everything the package delivers including delivered ZCML files and optional features. The tests are my tool (as developper of the package) to help me find and fix errors before the release. I am completely uninterested to facilitate testing of reduced or otherwise special use of my packages. If the full tests pass then a reduced use should work as well (provided the integrator did everything right). If the user is interested to verify for his own that the tests pass, I expect of him to test the full functionality -- or not use the package at all. To stress it: the above just describes test requirements -- not install requirements. I am ready to support loose install requirements (and use extras to support optional features) but I am not ready to invest in loose test requirements. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Dieter Maurer wrote: Tres Seaver wrote at 2009-3-13 16:20 -0400: ... Dieter Maurer wrote: Tres Seaver wrote at 2009-3-12 14:25 -0400: ... Sorry, I meant mandatory tests which load ZCML. I'm actually against ever loading ZCML in tests at all. If you ship ZCML, you should test it, no? Not necessarily: in fact, if testing it means that users of my package have to accept a big dogpile of dependencies that they otherwise wouldn't need to, then no. Your and my quality principles diverge: If I release a package, then its tests have the verify that the package contents work correctly. This implies: the tests should cover everything the package delivers including delivered ZCML files and optional features. The tests are my tool (as developper of the package) to help me find and fix errors before the release. I am completely uninterested to facilitate testing of reduced or otherwise special use of my packages. If the full tests pass then a reduced use should work as well (provided the integrator did everything right). If the user is interested to verify for his own that the tests pass, I expect of him to test the full functionality -- or not use the package at all. To stress it: the above just describes test requirements -- not install requirements. I am ready to support loose install requirements (and use extras to support optional features) but I am not ready to invest in loose test requirements. As a frequent recipient of Tres' wrath when things go untested, I can vouch that Tres is definitely interested in comprehensive test coverage. ;-) I think the point is that some ZCML he ships might be example ZCML which he has previously tested interactively. Like, say, a configuration snippet within a README.txt. I think maybe the lesson here is: think very carefully before you release a package with any ZCML in it that you want to be a library; if users *need* to load the ZCML, it's probably really not a library, it's probably an application. If you're OK with it being an application, then by all means, you should have automated end-to-end tests. But if you really mean it to be a library, the ZCML should be advisory (close to documentation), I don't see a problem with not doing any automated testing of it if that means the package doesn't have inappropriate dependencies. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Dependencies for ZCML
Hey, Tres Seaver wrote: [snip] - -1 on having configuration dependecies, including having mandatory tests that ZCML will load: You mean mandatory ZCML loaded by tests? You seem to write the reverse here, and I don't understand what that could mean. mandatory configuration is a contradiction in terms. I therefore don't believe that tests which try to load ZCML are useful, at least for library pacakges (as opposed to applications). Yeah, I know you're a purist on this topic. Here are some observations on configuration, mandatory or not, and ZCML. Sometimes you can only test a library when a particular utility or adapter or event handler is registered. The library uses this utility or adapter in its own logic and while the utility or adapter is intended to be replaceable (to make the library pluggable), it is mandatory to actually register a component that fulfills the interface requirements in order to use the library at all. As a convenience to the users of this library, that library's configure.zcml will provide default implementations (which may be the right ones in most cases). This is a useful pattern. I'll note another pattern. A library could define an interface, and a few event handlers that react on this interface. Now if you use that interface on your own objects, your object will trigger those event handlers. Your interest as a library user is not directly in those event handlers and you may be unaware of their existence, but they do maintain something that is important to you as a user (an index, for example). So this is an example of registrations that should be loaded in order to use the library, and there's not even much interest here in overriding those registrations - mandatory configuration. Now when testing these libraries you could do three things: * not use ZCML at all and recreate the effect of these registrations in Python code. * use the ZCML in the package's configure.zcml. (perhaps through ftesting.zcml) * construct ZCML in the tests itself and load it. In fact there's a fourth way you could go and use martian-style patterns, where you can manually 'grok' a component in tests that inherits a particular base class. We could argue that all ZCML in use in tests should be rewritten to manual registrations from Python code. Is this indeed a useful exercise? Doesn't that in some cases make tests harder to understand, as lower-level APIs are in use that are not as recognizable as the equivalent ZCML directives? (say, registering an event) Don't we place a burden on the test writers to learn these APIs while they could use the ones abstracted away behind ZCML instead? I will also repeat my observation that if a package has ZCML in it that never gets loaded by the tests of that package, that means that there are no automatic tests for this ZCML. There is something in this package that is not tested and can only be tested indirectly. Isn't that something we try to avoid? What do we gain dependency-wise by avoiding the loading of ZCML during tests but do manual registrations instead? We *may* get rid of dependencies on zope.configuration. If the definition of ZCML directives were always strictly separated from the functionality that these directives manipulate, then this would often be the case. In reality this is frequently not so, however. We may also get rid of such ZCML directive definition only packages, for instance zope.componentzcml. Any ZCML file which needs something like zope.component or zope.security to be present should signalt that by either including the meta.zcml (e.g., to define directives) or nesting the dependent directives inside a conditional block, whose predicate documents the requirement. You're saying that we should be an include of the zope.component 'meta.zcml' in *all* ZCML files that register an adapter? This is certainly not happening always now. That's like import before usage in Python modules, right? I'll note that martian-based code actually does use that pattern - any package based on grokcore.* for instance automatically follows this pattern. I'll also note that with martian-based configuration the situation is clear for setup.py: the dependencies needed for configuration are always normal dependencies of a package. With ZCML-based configuration the situation is far less clear. So what does all of this mean for Dan's question? I don't know yet. I think we should observe some packages. We strive for library-like packages. More library-like packages should likely not have to do a lot of work in their configure.zcml, but the amount of work is not always zero. Dan, do you have any examples of packages where you are wondering about what to do? Let's examine then and reason about how they could be organized. Regards, Martijn ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or
Re: [Zope-dev] Dependencies for ZCML
Martijn Faassen wrote: mandatory configuration is a contradiction in terms. I therefore don't believe that tests which try to load ZCML are useful, at least for library pacakges (as opposed to applications). Yeah, I know you're a purist on this topic. Here are some observations on configuration, mandatory or not, and ZCML. Sometimes you can only test a library when a particular utility or adapter or event handler is registered. The library uses this utility or adapter in its own logic and while the utility or adapter is intended to be replaceable (to make the library pluggable), it is mandatory to actually register a component that fulfills the interface requirements in order to use the library at all. Can you provide an example? In my own libraries, very rarely does this happen; I tend to only depend on components like this at the edges; eg. in I only try to allow pluggability via the CA within an application or a framework, very rarely in a straight library. But if somehow it does happen, and I need to test the library, and the library relies on some utility external to itself being registered, I'll register a dummy utility for purposes of testing. I'll never use a real implementation of the component during *unit* testing (although during integration testing I'll use the real one instead of the dummy one of course). As a convenience to the users of this library, that library's configure.zcml will provide default implementations (which may be the right ones in most cases). This is a useful pattern. Minor nit: the configure.zcml won't *provide* any default implementation, but might point at one. This implementation may or may not live in the package that the configure.zcml lives in. I'll note another pattern. A library could define an interface, and a few event handlers that react on this interface. Now if you use that interface on your own objects, your object will trigger those event handlers. Your interest as a library user is not directly in those event handlers and you may be unaware of their existence, but they do maintain something that is important to you as a user (an index, for example). So this is an example of registrations that should be loaded in order to use the library, and there's not even much interest here in overriding those registrations - mandatory configuration. If some library comes along that thinks it knows enough to register mandatory arbitrary CA configuration, I'll classify it along with the Python stdlib logging module (which registers atexit handlers) or asyncore dispatchers (whose constructors register the dispatcher with a socket map); these libraries tend to be wrong about the configuration they register under some circumstance and it probably would have been better for their authors to do less, pushing the configuration more into some glue package or the application that uses them rather than down into the library itself. That would offer maximum configurability. In particular, I'm pretty sure that to be maximally useful outside a Zope context (which may or may not be desireable for the maintainer), library packages should probably be broken up into two pieces: a piece that is a straight Python library that contains no ZCML, then some glue package that provides both an API to use the library in a Zope context along with (possibly) some ZCML that is meant to be loaded by an application. An example of this pattern today is in Chameleon: chameleon.zpt and chameleon.core have no dependency on ZCML; however, the z3c.pt package contains ZCML and an API that allows Chameleon to be used in a Zope context. It depends on chameleon.zpt, but chameleon.zpt can be used in, say, Turbogears without the TG folks needing to understand anything about ZCML, which feels quite right. Now when testing these libraries you could do three things: * not use ZCML at all and recreate the effect of these registrations in Python code. This is exactly what I do always (at least when unit testing; integration testing is a different story). * use the ZCML in the package's configure.zcml. (perhaps through ftesting.zcml) * construct ZCML in the tests itself and load it. In fact there's a fourth way you could go and use martian-style patterns, where you can manually 'grok' a component in tests that inherits a particular base class. We could argue that all ZCML in use in tests should be rewritten to manual registrations from Python code. Is this indeed a useful exercise? Doesn't that in some cases make tests harder to understand, as lower-level APIs are in use that are not as recognizable as the equivalent ZCML directives? (say, registering an event) Don't we place a burden on the test writers to learn these APIs while they could use the ones abstracted away behind ZCML instead? Folks who write *unit* tests (as opposed to integration tests) need to supply mock implementations of registered components anyway..
Re: [Zope-dev] Dependencies for ZCML
Chris McDonough wrote: Martijn Faassen wrote: [snip] Sometimes you can only test a library when a particular utility or adapter or event handler is registered. The library uses this utility or adapter in its own logic and while the utility or adapter is intended to be replaceable (to make the library pluggable), it is mandatory to actually register a component that fulfills the interface requirements in order to use the library at all. Can you provide an example? In my own libraries, very rarely does this happen; I tend to only depend on components like this at the edges; eg. in I only try to allow pluggability via the CA within an application or a framework, very rarely in a straight library. Maybe because unlike you I don't know the difference between a framework and a library very well. :) z3c.saconfig Allows you to set up a utility that integrates with SQLAlchemy's scoped session so you can have one database configuration per Zope site. This is explicitly to support local configuration, but in order to use SQLAlchemy at all in this way the utility has got to be there in some form. http://svn.zope.org/z3c.saconfig/trunk/ hurry.resource Javascript/CSS resource management. Intended to be web-framework neutral. Requires a utility ICurrentNeededInclusions to exist that somehow maintains the resource inclusions that are needed (for instance on the request object, but that's up to the utility). Also needs a ILibraryUrl adapter to be defined on a resource in order to render the URL for a resource. http://svn.zope.org/hurry.resource/trunk/ z3c.relationfield A 'Relation' field for zope.schema that gets tracked by the zc.relation catalog. Sets up a bunch of event handlers on IHasOutgoingRelations and IHasIncomingRelations. If you define those interfaces on your objects and use the Relation field in your schema, those relations are tracked. http://svn.zope.org/z3c.relationfield/trunk z3c.schema2xml Exporting zope.schema-driven content as XML and importing XML into an object again. Offers a simple API to do this. Registers a whole bunch of adapters for particular schema fields that know how to import and export attributes on objects that implement this schema. http://svn.zope.org/z3c.schema2xml/trunk/ But if somehow it does happen, and I need to test the library, and the library relies on some utility external to itself being registered, I'll register a dummy utility for purposes of testing. I'll never use a real implementation of the component during *unit* testing (although during integration testing I'll use the real one instead of the dummy one of course). Yes, I frequently do this, and frequently I load the ZCML (or grok things directly). Anyway, we're not so interested in that distinction here, right? If the library does integration testing (testing the real component also defined by that library), this will need to be reflected in the dependencies just like anything else. As a convenience to the users of this library, that library's configure.zcml will provide default implementations (which may be the right ones in most cases). This is a useful pattern. Minor nit: the configure.zcml won't *provide* any default implementation, but might point at one. This implementation may or may not live in the package that the configure.zcml lives in. Sure, yes, it registers it. In case the case of grokked libraries, it provides *and* indicates the desire to do registration in the same place (typically the python class). [snip] If some library comes along that thinks it knows enough to register mandatory arbitrary CA configuration, I'll classify it along with the Python stdlib logging module (which registers atexit handlers) or asyncore dispatchers (whose constructors register the dispatcher with a socket map); these libraries tend to be wrong about the configuration they register under some circumstance and it probably would have been better for their authors to do less, pushing the configuration more into some glue package or the application that uses them rather than down into the library itself. That would offer maximum configurability. This argument works for hurry.resource and z3c.saconfig, where people are expected to do their own registrations. It doesn't work for z3c.relationfield: If my library informs people that if they make their schema inherit IOutgoingRelations they can use Relation fields on it and any objects that implement that schema be tracked by a relation index automatically, I can't really say that the event handlers are somehow wrong and someone else would like to override them. People could hook up the event handlers again for their own interfaces, sure, but the original ones aren't in the way. It also doesn't work for z3c.schema2xml: The library's adapter registrations that know how to export a Text field aren't somehow in the way and it'd be a major pain to request people to
Re: [Zope-dev] Dependencies for ZCML
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martijn Faassen wrote: Tres Seaver wrote: [snip] - -1 on having configuration dependecies, including having mandatory tests that ZCML will load: You mean mandatory ZCML loaded by tests? You seem to write the reverse here, and I don't understand what that could mean. Sorry, I meant mandatory tests which load ZCML. I'm actually against ever loading ZCML in tests at all. mandatory configuration is a contradiction in terms. I therefore don't believe that tests which try to load ZCML are useful, at least for library pacakges (as opposed to applications). Yeah, I know you're a purist on this topic. Here are some observations on configuration, mandatory or not, and ZCML. Sometimes you can only test a library when a particular utility or adapter or event handler is registered. The library uses this utility or adapter in its own logic and while the utility or adapter is intended to be replaceable (to make the library pluggable), it is mandatory to actually register a component that fulfills the interface requirements in order to use the library at all. Such tests should supply a stub version of the component: they should *never* rely on a real version. As a convenience to the users of this library, that library's configure.zcml will provide default implementations (which may be the right ones in most cases). This is a useful pattern. That pattern does not lead to good testing. I'll note another pattern. A library could define an interface, and a few event handlers that react on this interface. Now if you use that interface on your own objects, your object will trigger those event handlers. Not if I don't register the handlers. Choosing to register them is *not* the purview of a library writer. Your interest as a library user is not directly in those event handlers and you may be unaware of their existence, but they do maintain something that is important to you as a user (an index, for example). Registering such machinery is the job of the application writer, not the library writer: it is *policy*. So this is an example of registrations that should be loaded in order to use the library, and there's not even much interest here in overriding those registrations - mandatory configuration. That would defat the whole point of the events machinery, which is to decouple things. If you can't use a package at all without reigstering some event handler, then the package isn't a library, it is an application. Now when testing these libraries you could do three things: * not use ZCML at all and recreate the effect of these registrations in Python code. +1. * use the ZCML in the package's configure.zcml. (perhaps through ftesting.zcml) - -lots. * construct ZCML in the tests itself and load it. - -0 (I don't see a win over just doing it in Python). In fact there's a fourth way you could go and use martian-style patterns, where you can manually 'grok' a component in tests that inherits a particular base class. We could argue that all ZCML in use in tests should be rewritten to manual registrations from Python code. Is this indeed a useful exercise? Yes. Doesn't that in some cases make tests harder to understand, as lower-level APIs are in use that are not as recognizable as the equivalent ZCML directives? (say, registering an event) Don't we place a burden on the test writers to learn these APIs while they could use the ones abstracted away behind ZCML instead? No. Relying on real ZCML in testing, as is using the real components is an anti-pattern: it makes tests fragile, couples the packages tightly, etc. I will also repeat my observation that if a package has ZCML in it that never gets loaded by the tests of that package, that means that there are no automatic tests for this ZCML. There is something in this package that is not tested and can only be tested indirectly. Isn't that something we try to avoid? *I* don't care about testing ZCML at the package level: I don't think it is useful, and I find that it actively screws up real tests. Testing the real configuration is something which makes sense for applications, not reusable library packages. What do we gain dependency-wise by avoiding the loading of ZCML during tests but do manual registrations instead? We don't try to re-use the real components which that ZCMl would register, and therefore don't depend on the other package at all. We *may* get rid of dependencies on zope.configuration. If the definition of ZCML directives were always strictly separated from the functionality that these directives manipulate, then this would often be the case. In reality this is frequently not so, however. We may also get rid of such ZCML directive definition only packages, for instance zope.componentzcml. Any ZCML file which needs something like zope.component or zope.security to be present should
Re: [Zope-dev] Dependencies for ZCML
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martijn Faassen wrote: Dan Korostelev wrote: 2009/3/11 Martijn Faassen faas...@startifact.com: Oh, and on the topic, one more time: can we have a steering group decision on the package requirements for zcml statements? Are we doing extras for them or simply skipping them? Sorry, I wasn't clear that there was an open question and I'm afraid I don't understand this one. :) Could you point me to the appropriate thread that was left in the middle, or could you start a new thread with a description of the open question? I'm too lazy now to search in archives, so I'll just describe again. For example, the zope.password package only requires zope.interface to be functional. But it's configure.zcml contains directives that need zope.component (or repoze.zcml) and zope.security. Also, the zcml thing itself needs zope.configure as well. Should we mention it in extra dependencies somehow or just document it, saying that zcml is intented to be used in more zope3-ish environment that already has needed packages, so others can simply ignore these files. Good question. In packages where the *tests* load the ZCML, they will definitely need to be described as test extra dependencies at the least. In packages that don't load their own ZCML during the tests, it's harder to say. One reaction could be that this package doesn't have enough tests then! Of course another would argue that this is configuration information only that can be overridden, but it is rather special configuration... - -1 on having configuration dependecies, including having mandatory tests that ZCML will load: mandatory configuration is a contradiction in terms. I therefore don't believe that tests which try to load ZCML are useful, at least for library pacakges (as opposed to applications). Any ZCML file which needs something like zope.component or zope.security to be present should signalt that by either including the meta.zcml (e.g., to define directives) or nesting the dependent directives inside a conditional block, whose predicate documents the requirement. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJuGTz+gerLs4ltQ4RAitlAKDXU6+YEGsF5ftyZim+8LYPVTilOACcC09i dh9HsIp2qk1jLbeee0KVvpw= =HzIJ -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )