Re: [Zope3-dev] zope.configuration
Jim Fulton wrote: I'd have no problem with generating actions in Python. It would allow greater control and would probably make action generation much faster. If we did this, We'd probably want to improve the action-generation API. We'd also need to think about how action info should be generated, especially wrt error reporting. Perhaps we should support both ZCML and python action generation. Cool. This is exactly what I was hoping to hear. I now realize that if I write Zope 3 software using Python code for configuration or action generation, I won't be swimming upstream. Thanks. Shane ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
Jim Fulton wrote: [snip very clear explanation of how zope.configuration works] Thanks for putting this all in one place! The bit on conflict resolution was helpful to my understanding - I hadn't seen it explained so clearly before. (Note that a flaw in this model is that we have no good way to undo actions. We don't need this for normal execution, but it would be very helpful for testing.) Presumably you have thought about the registration of an equal and opposite action whenever we register an action that can undo the current registration. What are your thoughts about that? It would still need to work with order and I assume the component architecture has sufficient infrastructure to deregister registrations here. Actions need not be generated by executing ZCML. They can be generated by processing configuration files in other formats. They could even be generated by Python code. I'm told that that's what Launchpad does. It'd be nice if Steve could talk about what Launchpad does a bit more often... I was talking to Philipp last night about providing an equivalent API that looks exactly like ZCML but is in Python. The motivation was that this would help in making testing easier - if you want to accomplish the equivalent effect of ZCML, you would write Python code that's the same. My idea however was that it would execute the actions immediately, which is different from actually generating actions as you suggest here. Perhaps we need both, in a layered approach: ZCML | v Python API for generating actions (zcpy, for zope configuration python) | v Python API for doing what actions do now, using same names and arguments as ZCML (zrpy, for zope registration python) If zrpy and zcpy essentially implement the same interface, which is morally equivalent to ZCML's, this might be a step in the right direction. On the other hand, it might turn out to be incredibly confusing. :) An alternative way to accomplish immediacy in action execution is to have a special configuration engine which simply executes actions as they come in. It would have no support for conflict handling, and might want to just simply bail out if it runs into what would be an override in ZCML. Or perhaps a testing API doesn't actually need this immediacy at all. I'd have no problem with generating actions in Python. It would allow greater control and would probably make action generation much faster. If we did this, We'd probably want to improve the action-generation API. We'd also need to think about how action info should be generated, especially wrt error reporting. Perhaps we should support both ZCML and python action generation. I think that this would be a useful direction to go towards, if only to make writing tests easier (only one API to learn instead of the underlying APIs that are used by ZCML). Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
Martijn Faassen wrote: Jim Fulton wrote: ... (Note that a flaw in this model is that we have no good way to undo actions. We don't need this for normal execution, but it would be very helpful for testing.) Presumably you have thought about the registration of an equal and opposite action whenever we register an action that can undo the current registration. What are your thoughts about that? I haven't though that much about it. I just mention it as something we'll probably want to come to terms with if we expect people to start writing actions more directly. I would just give actions both forward and reverse callables and arguments. It would still need to work with order The idea is that after applying configuration, you'd keep the resolved sequence of actions around so that you could call their undo methods later. and I assume the component architecture has sufficient infrastructure to deregister registrations here. It does now. Actions need not be generated by executing ZCML. They can be generated by processing configuration files in other formats. They could even be generated by Python code. I'm told that that's what Launchpad does. It'd be nice if Steve could talk about what Launchpad does a bit more often... I was talking to Philipp last night about providing an equivalent API that looks exactly like ZCML but is in Python. The motivation was that this would help in making testing easier - if you want to accomplish the equivalent effect of ZCML, you would write Python code that's the same. My idea however was that it would execute the actions immediately, which is different from actually generating actions as you suggest here. I'll note that we typically write 3 kinds of tests: 1, Unit tests. For these we go out of our way to avoid configuration. We make agressive use of skaffolding, if necessary, to avoid dependence on other systems. 2, Component functional tests. These are tests of a package using the default configuration use by the package. This allows us to test things like configuration and UI without depending on a particular application. This was the primary motivation for layers in the new test runner. 3. System tests, in which we test a very specific configured application. For 1, we don't need configuration support. For 2 and 3 it's important to use exactly the ZCML provided by the package or by the application respectively. Fly what you test, and test what you fly. If you use ZCML for configurating, use it for functional and system tests. ... I'd have no problem with generating actions in Python. It would allow greater control and would probably make action generation much faster. If we did this, We'd probably want to improve the action-generation API. We'd also need to think about how action info should be generated, especially wrt error reporting. Perhaps we should support both ZCML and python action generation. I think that this would be a useful direction to go towards, if only to make writing tests easier (only one API to learn instead of the underlying APIs that are used by ZCML). I strongly oppose and discourage using a separate API for tests, for the reasons given above. The main reasons, IMO, for providing a Python API are: - Provide greater control for those who want it. For example, to allow more abstract configuration tools, avoid repetition, etc. - Provide an alternative for people who just can't stand pointy brackets. (Who can blame them? ;) Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
Jim Fulton wrote: Martijn Faassen wrote: Jim Fulton wrote: [snip] The idea is that after applying configuration, you'd keep the resolved sequence of actions around so that you could call their undo methods later. Yes, that's what I was thinking of too. Good. :) [snip] I was talking to Philipp last night about providing an equivalent API that looks exactly like ZCML but is in Python. The motivation was that this would help in making testing easier - if you want to accomplish the equivalent effect of ZCML, you would write Python code that's the same. My idea however was that it would execute the actions immediately, which is different from actually generating actions as you suggest here. I'll note that we typically write 3 kinds of tests: 1, Unit tests. For these we go out of our way to avoid configuration. We make agressive use of skaffolding, if necessary, to avoid dependence on other systems. 2, Component functional tests. These are tests of a package using the default configuration use by the package. This allows us to test things like configuration and UI without depending on a particular application. This was the primary motivation for layers in the new test runner. 3. System tests, in which we test a very specific configured application. For 1, we don't need configuration support. For 2 and 3 it's important to use exactly the ZCML provided by the package or by the application respectively. Fly what you test, and test what you fly. If you use ZCML for configurating, use it for functional and system tests. Agreed. However... For 1. we don't need configuration support, but what I think could be improved is the skaffolding. Often in unit tests we do register something with the component architecture. Right now we need to know two systems whenever we write code that needs unit tests: * ZCML for when the code needs to actually run * component architecture APIs when we need to set up some stuff to support our tests. What I'm suggesting is having a Python API that looks much like ZCML but does the component architecture registration bit of what ZCML does. Less things to learn for the person writing (or reading) the scaffolding for the tests. Since we also were talking about a Python API to generate ZCML actions, it seems to make sense that the same API could also have an implementation that *does* generate actions. I'd have no problem with generating actions in Python. It would allow greater control and would probably make action generation much faster. If we did this, We'd probably want to improve the action-generation API. We'd also need to think about how action info should be generated, especially wrt error reporting. Perhaps we should support both ZCML and python action generation. I think that this would be a useful direction to go towards, if only to make writing tests easier (only one API to learn instead of the underlying APIs that are used by ZCML). I strongly oppose and discourage using a separate API for tests, for the reasons given above. The main reasons, IMO, for providing a Python API are: - Provide greater control for those who want it. For example, to allow more abstract configuration tools, avoid repetition, etc. - Provide an alternative for people who just can't stand pointy brackets. (Who can blame them? ;) Perhaps I'm wrong, but I think you might not have understood my reasons for wanting ZCML-similar API for unit tests. I absolutely agree that for functional tests, the same configuration should be used as when the code actually runs. I wasn't talking about that however - I hope I explained more clearly now why this might be a good idea for *unit* tests, where effectively a separate API is already being used to set up the various components (skaffolding). If we are to present a Python-style API for ZCML anyway, reusing it (with a different implementation) seems like low-hanging consistency fruit. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
Jim Fulton wrote: Stephan Richter wrote: On Friday 17 March 2006 06:34, Jim Fulton wrote: The idea is that after applying configuration, you'd keep the resolved sequence of actions around so that you could call their undo methods later. Of course, the undo feature has other benefits, such as reloading functionality without restarting the server. Or at least this is one required step. I'm skeptical of this. Jim any plan to have something like component hotplugging, i.e. registering and unregistering a set of directives without restarting the server? /JM ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
On Friday 17 March 2006 10:46, Jim Fulton wrote: I'll note, however, that I think there will be lots of interesting small Python applications that don't use zope.configuration. zope.configuration is helpful for providing flexible configuration of an application server. I think and hope that there will be lots of interesting applications that don't use an application server and that do use Zope components. These components should not depend on zope.configuration and their unit tests should not introduce such a dependency or be made more complex just to provide some sort of consistency. I think this is a really good point that we should keep in mind. I, personally, love the new zope.component registration functions. OT: I also used ``zope.interface.implementer`` and ``zope.component.adapter`` as decorators for the first time yesterday and I absolutely love them! I am intrigued to write a ``zope.app.apidoc.factory`` decorator now that does the apidoc helper dance now. For all that are wondering what this is about: If the adapter is a function (for example), apidoc has no way of telling what type of object is being returned and thus the documentation is much less useful. By setting the ``factory`` attribute to the object factory on the function, apidoc will have the data it needs. This even works for nested functions. Regards, Stephan -- Stephan Richter CBU Physics Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
Jim Fulton wrote: Martijn Faassen wrote: Jim Fulton wrote: Martijn Faassen wrote: Jim Fulton wrote: [snip] The idea is that after applying configuration, you'd keep the resolved sequence of actions around so that you could call their undo methods later. Yes, that's what I was thinking of too. Good. :) I'll note that this is a half baked idea and may not work. In particular, as successful undo would need to restore configuration that the action overrode. I'm not sure that the benefit is worth the bother of getting this right. That's true, I hadn't considered that. [skaffolding in unit tests] For 1. we don't need configuration support, but what I think could be improved is the skaffolding. Often in unit tests we do register something with the component architecture. This can be avoided more often than you realize. It's a good thing you know what I realize. :) Anyway, I've seen lots of testing code that does not avoid this. The people who wrote this test might have done it unnecessarily, but that points to doing it the right way apparently being harder than doing it the wrong way. Are you saying this is an education/documentation issue? Also, there are simple APIs that make this straightforward when it is necessary. I know. I'm looking for ways to make it even simpler, as I've seen people actually run into this issue. Right now we need to know two systems whenever we write code that needs unit tests: * ZCML for when the code needs to actually run * component architecture APIs when we need to set up some stuff to support our tests. What I'm suggesting is having a Python API that looks much like ZCML but does the component architecture registration bit of what ZCML does. Less things to learn for the person writing (or reading) the scaffolding for the tests. OK, I think we'll be better served by different APIs. An API necessary to create actions will, I think, be too combersome for simple registration needed by tests. Yes, two APIs, same interface. One creates actions, the other one doesn't. The one creating the actions could hopefully be implemented in terms of the other. Since we also were talking about a Python API to generate ZCML actions, it seems to make sense that the same API could also have an implementation that *does* generate actions. ZCML actions require information that isn't necessary when just registering components for tests. I won't argue. If we create an API for configuration, and you want to use it for setting up tests, be my guest. I'll stick to the simpler APIs myself. Sounds good. I'll note, however, that I think there will be lots of interesting small Python applications that don't use zope.configuration. zope.configuration is helpful for providing flexible configuration of an application server. I think and hope that there will be lots of interesting applications that don't use an application server and that do use Zope components. These components should not depend on zope.configuration and their unit tests should not introduce such a dependency or be made more complex just to provide some sort of consistency. That's a good point. Perhaps the API I'm talking about belongs in zope.component (and anything else that ZCML statements exist for, in the end). It's likely a very thin layer over what's already there. Then zope.configuration can take the interfaces and implement an action-based and ZCML based system for them. Anyway, it's just an idea. Maybe I'll find some time to look into this eventually. I know enough of the innards of zope.configuration by now to also be potentially helpful in trying to pull it apart into a Python API (the action-generating one) and the ZCML bit. Regards, Martijn ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] zope.configuration
Martijn Faassen wrote: Jim Fulton wrote: ... Anyway, I've seen lots of testing code that does not avoid this. The people who wrote this test might have done it unnecessarily, but that points to doing it the right way apparently being harder than doing it the wrong way. Are you saying this is an education/documentation issue? Yes, to a large degree. We have lots of grody tests, espectally ones written earlier or ones written after looking at ones written earlier. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] zope.configuration
Lets step back and look at why we created ZCML. A primary goal of the component architecture was to make it easier to reuse packages. A major problem with Zope 2 products is that they are hard to customize. The CMF made this a bit better with skins. You could override selected scripts or templates provided by a product by providing overriding versions in your own products or by customizing through the web. You don't have to modify the original product. With the component architecture, you can override at the level of components. As with the CMF, we need to be able to override components without modifying the original package. The zope.configuration package provides a framework for defining and overriding configurations. It *also* supports an XML syntax. It supports other syntaxes too. (This support may have atrophied somewhat from lack of use. :) In fact, the original syntax for zope.configuration was a simple text-based syntax. The configuration framework is based on computing sequences of actions. When a ZCML file is processed, the individual handlers compute lists of actions. These actions are accumulated as different files are processed. An action includes: - a discriminator used to detect conflicting actions. Two or more actions conflict of they have the same discriminator. An action may have None as it's discriminator, in which case, it cannot conflict with another action. - callable, positional, and keyword arguments, - include path used to resolve conflicts. If there is an action in a set of conflicting actions whose include path is a proper prefix of the include paths of all of the other conflicting actions, then the conflict is resolved by keeping this action and discarding the other actions. This is why a file that includes another file can have a directive that overrides directives in the included file. - info that documents the action. - order (lame) that can be used to affect the order in which an action executes. A very high number can be used to make an action execute relatively last. Configuration processing proceeds in 3 steps: 1. Actions are generated, typically by processing ZCML files. (We'll ignore meta-configuration in this discussion. :) 2. Actions generated in step 1 are analyzed for conflicts. If any conflicts can't be resolved, they are reported as errors. The action info is used in the generation of these errors. 3. Actions are executed by calling their callables with their positional and keyword arguments. The resolution of conflicts in stage 2 is critical for achieving the goal of the configuration framework, which is overriding. The way actions are generated is not critical. (Note that a flaw in this model is that we have no good way to undo actions. We don't need this for normal execution, but it would be very helpful for testing.) Actions need not be generated by executing ZCML. They can be generated by processing configuration files in other formats. They could even be generated by Python code. I'm told that that's what Launchpad does. I'd have no problem with generating actions in Python. It would allow greater control and would probably make action generation much faster. If we did this, We'd probably want to improve the action-generation API. We'd also need to think about how action info should be generated, especially wrt error reporting. Perhaps we should support both ZCML and python action generation. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
[Zope3-dev] zope.configuration and ImportErrors
Today it's very hard to debug ImportErrors raise inside modules
referenced through a zcml directive, because the ImportError is
swallowed an no traceback comes back to the user. Just a
ConfigurationError: ('Invalid value for', 'class', Couldn't
import some.class, cannot import name somename)
which doesn't point anywhere useful.
I'd like to modify zope.configuration.config to something like:
try:
mod = __import__(mname, *_import_chickens)
except ImportError, v:
+if sys.exc_info()[2].tb_next is not None:
+# ImportError was caused deeper
+raise
raise ConfigurationError, (
Couldn't import %s, %s % (mname, v)
), sys.exc_info()[2]
Instead of the simpler code that was there (this nice trick is by
PJE, and is used ).
Also I need to fix zope.configuration.xmlconfig, today it has a bare
except:
try:
self.context.begin(name, data, info)
except:
if self.testing:
raise
raise ZopeXMLConfigurationError, (
info, sys.exc_info()[0], sys.exc_info()[1]
), sys.exc_info()[2]
Shouldn't it just catch ConfigurationError?
With both changes I get much nicer error report: just a traceback
with the correct info.
Florent
--
Florent Guillaume, Nuxeo (Paris, France) Director of RD
+33 1 40 33 71 59 http://nuxeo.com [EMAIL PROTECTED]
___
Zope3-dev mailing list
Zope3-dev@zope.org
Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
