Re: [Zope3-dev] Re: Autoconfiguring a site.zcml
On Apr 23, 2007, at 2:43 PM, Martijn Faassen wrote: ... It'd be a lot easier. You'd still have to list it for all eggs that you depend on directly. It would be nice if this could be automated as well, as being in two places to add a single dependency is more work than being in one place. Could be done with a special recipe, I think. Except that the information belongs in your application package's configure.zcml. site.zcml should, IMO, be a very short file that include's your application package's configure.zcml. If you want to write a tool that writes your configure.zcml, go for it. I wouldn't object to a zcml configuration directive that took a project name and included zcml files provided by it's dependencies at run time. Package-level ZCML include dependencies would also mean a kind of duplication, too. You list dependencies in setup.py and then again in the package's ZCML. You could argue that the dependencies in setup.py duplicate information in your Python import statements -- if you wanted to. I also don't understand David's problem with buildout times. The option -N helps a lot. Jim, is there anything to say for actually making -N the default behavior of buildout? No, But I can say something against it. :) Making it the default would make buildout's default behavior less deterministic. I don't understand. I can see how -o is the most deterministic buildout behavior, as it will upgrade nothing. The next step in my mind would be -N, as it only installs what's not there yet. Finally there's "no option", which is least predictable it might start updating stuff, depending on what package restrictions you have. Obviously I'm not understanding what you mean by "deterministic". The default behavior is to update all packages to the most recent version that satisfies your requirements. This means that running buildout in 2 different buildouts with the same configuration should produce the same results, regardless of their history. I think this is an important property that you lose with -N. It is easy for people to change the default for their own use by putting: [buildout] newest = false in their ~/.buildout/default.cfg file. Ah, a trick I wasn't aware of. I will do that then in my buildout.cfgs, though this will indeed might cause a few surprises to people they might get used to it and be surprised when they need -N in other buildouts. That's why I suggested putting it in: ~/.buildout/default.cfg, which would effect only you. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Autoconfiguring a site.zcml
On Apr 23, 2007, at 2:14 PM, Martijn Faassen wrote: David Pratt wrote: Hi Jim. I guess I missed the boat on app construction which I have clarified hopefully below. I am speaking of using zc.zope3recipes. As far as site.zcml. I think I'll need to experiment. At this point I see a strong incentive to automate. Too many packages - too little time and too many errors without. I want z3 development to be fast and enjoyable and I don't want to hate configuration but I honestly hate trial and error approaches. This is where I feel I am at with the site.zcml - particularly reconstituting zope and apps from packages and eggs in this context (particularly with the ordering of includes). Many thanks. David and Jim, As far as I can determine from this thread (I have some difficulty following parts of David's long mail as well) you two are agreeing. :) Jim, is it correct that you'd like the zope 3 recipe to grow an option to auto-construct a site.zcml based on egg dependencies? No. I suggested that packages that have ZCML and that depend on other packages that have ZCML should include the ZCML of the packages they depend on. If this was common practice, then I don't think that constructing a site ZCML would be that hard. Tres's argument is that since there is no system to override ZCML auto-inclusion of configuration should not be implemented yet. But Jim seems to disagree with this (and I do too). There isn't a good way to cancel configuration done by included packages. I've found that in practice this usually doesn't matter, but I agree with Tres that it is a significant hole. I also don't understand David's problem with buildout times. The option -N helps a lot. Jim, is there anything to say for actually making -N the default behavior of buildout? No, But I can say something against it. :) Making it the default would make buildout's default behavior less deterministic. It is easy for people to change the default for their own use by putting: [buildout] newest = false in their ~/.buildout/default.cfg file. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope3-dev mailing list Zope3-dev@zope.org Unsub: http://mail.zope.org/mailman/options/zope3-dev/archive%40mail-archive.com
Re: [Zope3-dev] Re: Autoconfiguring a site.zcml
Hi Jim. I guess I missed the boat on app construction which I have clarified hopefully below. I am speaking of using zc.zope3recipes. As far as site.zcml. I think I'll need to experiment. At this point I see a strong incentive to automate. Too many packages - too little time and too many errors without. I want z3 development to be fast and enjoyable and I don't want to hate configuration but I honestly hate trial and error approaches. This is where I feel I am at with the site.zcml - particularly reconstituting zope and apps from packages and eggs in this context (particularly with the ordering of includes). Many thanks. Regards David - The recipe: zc.zope3recipes The app: An app configured using the main setup.py for an application. The app part: an app configured using recipe = zc.zope3recipes:app The contrast: 1. An app as glue for functionality bin bootstrap.py buildout.cfg develop-eggs eggs parts setup.py src --namespace --app-package Advantages: * I dont care about grouping packages under main app src - I don't care about heirarchy - let eggs find paths - each egg contains package with functionality. * Each package of functionality is an egg with its own dependencies * Each egg is listed as dependency to the app in the main setup.py * The setup.py dependencies are explicit * Buildout checks dependencies so I don't have to. * I develop a package, I list dependencies in egg for each small package. This is consistent with package development and package reuse * I am concerned about app as glue. It is small and also an egg. Disadvantages: * Larger list of dependencies in main setup.py * Buildout takes longer. Buildout performance: Takes more time. Why? Because package functionality in an egg with its own dependencies. Dependencies do not overlap in the main setup because they are explicit for each egg. Every package to app is an egg that is treated as an 'external' dependency that requires checks by buildout. 2. An app as a grouping functionality of several packages bin bootstrap.py buildout.cfg develop-eggs eggs parts setup.py src --namespace --app-package --my-pkg1 --my-pkg2 --my-pkg3 --my-pkg4 --my-pkg5 -- ... Advantages: * I care about app package grouping - the heirachy of the app is intuitive to the app * The app setup.py only identifies external dependencies to the source of my app * Buildout is faster Disadvantages: * Dependencies are not explicit. I need to determine what they are on the collection of packages that are my app to include in setup.py * This is not a good scenario for package reuse. * Buildout checks dependencies but I need to spend time ensuring I have got it right for my collection of packages in app to include in the setup.py for my app * I develop a package for my app, but since dependencies only come into the app setup.py for the app egg, I have not been explicit in identifying dependencies for each package. Buildout performance: Buildout takes less time. Why? The setup.py is checking 'external' dependencies only. Relationship of packages in app is implicit Jim Fulton wrote: On Apr 22, 2007, at 10:32 AM, David Pratt wrote: ... In looking at the way I am developing, my goals are package reuse and thin glue in an app package (that is also the app egg) to bind packages to make an application consisting mostly of installation, security, testing and perhaps skinning. That said, I do not really want or care about organizing packages neatly under a single src folder to pull them together into a larger egg. To keep it simple, I just want to list the eggs I use in app part of my buildout. I wonder what you are trying to contrast here. Your goal of being able to just list eggs sounds reasonable. Who would disagree with it? You are penalized for this approach in app buildouts since more small independent eggs (holding some generic functionality) with their own dependencies equals more checks. What checks are you referring to? Being explicit like this Like what? I don't understand what the "this" is that you are referring to. increases the amount of time for the buildout to run, particularly if there are plenty of eggs with many dependencies. Think in numbers like 150 - 200+ eggs with zope and zope app - no exaggeration. Are you referring to Buildout's checking for new egg versions? Judicious use of the -N option can cut down on this a lot. ... An approach that that leads to shorter app buildouts is to group packages under the main app setup.py. This is currently what fits with the development approach spelled out in the app recipe docs but it has downsides also. I don't understand what you are referring to. Which app recipe are you referring to? This is nice for speed in running a buildout but don't like the fact that the main setup has got to capture dependencies for the collection as a whole because you just a bigge
Re: [Zope3-dev] Re: Autoconfiguring a site.zcml
On Apr 22, 2007, at 10:32 AM, David Pratt wrote: ... In looking at the way I am developing, my goals are package reuse and thin glue in an app package (that is also the app egg) to bind packages to make an application consisting mostly of installation, security, testing and perhaps skinning. That said, I do not really want or care about organizing packages neatly under a single src folder to pull them together into a larger egg. To keep it simple, I just want to list the eggs I use in app part of my buildout. I wonder what you are trying to contrast here. Your goal of being able to just list eggs sounds reasonable. Who would disagree with it? ... You are penalized for this approach in app buildouts since more small independent eggs (holding some generic functionality) with their own dependencies equals more checks. What checks are you referring to? Being explicit like this Like what? I don't understand what the "this" is that you are referring to. increases the amount of time for the buildout to run, particularly if there are plenty of eggs with many dependencies. Think in numbers like 150 - 200+ eggs with zope and zope app - no exaggeration. Are you referring to Buildout's checking for new egg versions? Judicious use of the -N option can cut down on this a lot. ... An approach that that leads to shorter app buildouts is to group packages under the main app setup.py. This is currently what fits with the development approach spelled out in the app recipe docs but it has downsides also. I don't understand what you are referring to. Which app recipe are you referring to? This is nice for speed in running a buildout but don't like the fact that the main setup has got to capture dependencies for the collection as a whole because you just a bigger egg with multiple packages where you still need to track down dependencies. True, some of these may be packages from external eggs you need to list, but probably more are packages that you have assembled within your app (that may be desirable to re-use) but where you have not been being explicit about the dependencies (since these are determined in an egg not a package). Could you give some examples of what you're talking about. I'm trying to follow you but can't. You seem to be discussing whether dependencies should be listed in a setup file and suggesting that it somehow affects buildout performance. buildout performance isn't affected by *where* dependencies are defined. If your package has dependencies, then list them. This is othogonal to performance. Personally, I don't want to take the time to try to pull the dependencies together in an app that is a collection of packages like this - since it is a chore not worthy of a human.A machine can do this better than I can (and it is welcome to the task!). Also, apps are prone to change - each change equals more time re- examining this over this again and again amidst a constantly changing landscape of package revisions - yuk. Are you referring to Python dependencies or ZCML dependencies? If you are the author of a package, you should know what other packages you're using. You should know when this changes. In theory, determining these dependencies could be automated. No one has automated it yet. When you add that as a zope3/python developer you are mostly working at a package level, you realize that spelling dependencies with more granularity in an egg is much more attractive and just easier overall than figuring this out for an app with a cluster of packages afterwards (on top of how this all goes together with other external/namespace packages like zope, zope.app, z3c, zc etc, etc). I wish I knew what you were trying to say here. Are you saying that if A depends on B and B depends on C and A doesn't use C directly that A should not have to list C as a dependency? If so, I don't think anyone would disagree with you. I feel the same way about the site.zcml. In the initial docs on the app recipe, What app recipe. Please be specific. a smaller site.zcml is recommended since you can tuck most of the meat in your app configuration. In experimenting, the bottom line is it doesn't really matter where it is - because is really the same size regardless of where the pieces reside. So I have been considering - what am I really trying to do here - and how much grief do I want to encumber putting it all together. The semantics of having a large site.zcml that takes care of app configuration is not inconsistent with the fact that I am using the buildout to configure and build my app. So in the end I would be happy to consider a larger site.zcml that does the work of organizing includes to all packages in the app. And at the package level being left with just making the includes within it (such as including a browser or an ftest sub package) to complete the configu
Re: [Zope3-dev] Re: Autoconfiguring a site.zcml
Hi Tres and Martijn. I apologize for the length of this post in advance. I have been spending a bit of time on app buildouts over the past days. What I have learned about some of this is that there are a couple of ways to configure apps with pluses and minuses. There are also major differences in time running buildouts depending on how you construct an app (since buildout checks against eggs in the main setup, dev eggs etc). To get a good sense of the acrobatics it is good to increase the verbosity of the output generated in buildout to -vv (you can add more v's to see more). In looking at the way I am developing, my goals are package reuse and thin glue in an app package (that is also the app egg) to bind packages to make an application consisting mostly of installation, security, testing and perhaps skinning. That said, I do not really want or care about organizing packages neatly under a single src folder to pull them together into a larger egg. To keep it simple, I just want to list the eggs I use in app part of my buildout. In the end it all about making sure that the code is able to connect. You are penalized for this approach in app buildouts since more small independent eggs (holding some generic functionality) with their own dependencies equals more checks. Being explicit like this increases the amount of time for the buildout to run, particularly if there are plenty of eggs with many dependencies. Think in numbers like 150 - 200+ eggs with zope and zope app - no exaggeration. Ok - now think again - that could be your time and not machine time working out that all dependencies are met. You start saying I like what computers can do and I can be patient while the checks are made. An approach that that leads to shorter app buildouts is to group packages under the main app setup.py. This is currently what fits with the development approach spelled out in the app recipe docs but it has downsides also. This is nice for speed in running a buildout but don't like the fact that the main setup has got to capture dependencies for the collection as a whole because you just a bigger egg with multiple packages where you still need to track down dependencies. True, some of these may be packages from external eggs you need to list, but probably more are packages that you have assembled within your app (that may be desirable to re-use) but where you have not been being explicit about the dependencies (since these are determined in an egg not a package). Personally, I don't want to take the time to try to pull the dependencies together in an app that is a collection of packages like this - since it is a chore not worthy of a human. A machine can do this better than I can (and it is welcome to the task!). Also, apps are prone to change - each change equals more time re-examining this over this again and again amidst a constantly changing landscape of package revisions - yuk. When you add that as a zope3/python developer you are mostly working at a package level, you realize that spelling dependencies with more granularity in an egg is much more attractive and just easier overall than figuring this out for an app with a cluster of packages afterwards (on top of how this all goes together with other external/namespace packages like zope, zope.app, z3c, zc etc, etc). I feel the same way about the site.zcml. In the initial docs on the app recipe, a smaller site.zcml is recommended since you can tuck most of the meat in your app configuration. In experimenting, the bottom line is it doesn't really matter where it is - because is really the same size regardless of where the pieces reside. So I have been considering - what am I really trying to do here - and how much grief do I want to encumber putting it all together. The semantics of having a large site.zcml that takes care of app configuration is not inconsistent with the fact that I am using the buildout to configure and build my app. So in the end I would be happy to consider a larger site.zcml that does the work of organizing includes to all packages in the app. And at the package level being left with just making the includes within it (such as including a browser or an ftest sub package) to complete the configuration tree. That said, I don't relish to time it will take to looking at ordering all of the includes in a site.zcml. It is tedious work and not fun at all. Here, this is still about dependency first. We need to make sure that as configuration is read, the app already knows enough for the next bit of configuration so it does not choke when run. Put it in the wrong order and your app will not understand what an adapter is, or a provider or one of your custom components. Unlike the comfort we had in automatically having all of the zope or zope app packages when we use zope we are now in the drivers seat to determine what we need by spelling it out. This provides power and flexibili
