[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
whit wrote: > Jim Fulton wrote: >> whit wrote: >> ... Specific use cases would help to guide this. >>> the main usecase for me is the following... hanno writes a recipe for >>> plone, and I want to use that recipe as part of setting up a >>> openplans development environment (for example inside my workingenv >>> that I've been developing w/out plone). >> Does this recipe create something in particular? A Plone instance? >> A Plone software install? > (correct me if I'm wrong hanno) currently it would create a zope > instance w/ all the packages needed to create a plone instance(via the zmi.) I have written a couple of recipes that used together can get you an environment where you can create a Plone instance in the ZMI. So far there are no Plone specific recipes. The only Plone specific thing at the moment is the buildout.cfg found in ploneout. Currently there's: z2c.recipe.zope2install -- It will download either a tarball or checkout a SVN branch, put it into parts and run 'setup.py build_ext -i' on it. z2c.recipe.zope2instance -- This creates a Zope2 instance for you. You give it a path to a Zope2 version, which it will call mkzopeinstance.py from. There's a bunch of options that allow you to configure everything from various zope.conf options, multiple products directories, ... and it will create a zopectl like starter script in the buildout root bin folder that'll let you run tests from it and start/stop the instance. This recipe reuses zc.recipe.egg and is the only controversial in regard to workingenv. z2c.recipe.bundlecheckout -- You give this recipe an url and tell it which vcs system it is (defaulting to svn, with support for cvs as well) and it will checkout the url and put it into /parts. z2c.recipe.distros -- And another one which you give an url to and it will download an unpack a tarball into /parts. You can tell the recipe that the tarball has a nested structure (so it will go down one level and unpack that, this is useful for the CMF tarball for instance) or that the folder inside the tarball has a version suffix (like PluginRegistry does it for example). Hanno ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: RFC: Eggifying Zope's extension mechanism ("Products")
Philipp von Weitershausen wrote:
Rocky Burt wrote:
On Thu, 2007-25-01 at 05:07 -0800, Martin Aspeli wrote:
I do wonder what would happen if you had both
lib/python/Products/CMFCore
and Products/CMFCore, though. Would there be an explicit preference
or would
Zope fail to start up with a conflict? I think I'd prefer the latter, in
fact, so that people don't end up modifying/upgrading the wrong code by
accident!
Well, we could probably add conflict-detection to the entry point
handlers for Zope (ie so any two packages that try to register as the
same project would cause an error). But regarding Products/CMFCore and
lib/python/Products/CMFCore conflicting... that would be up to the
standard pythonpath mechanism of the python interpreter (whichever is
first on the path wins).
Zope 2 itself manipulates Products.__path__ to add INSTANCE/Products (or
any other directory specified in zope.conf) as a directory which can
contain further products (the original Products package lives at
ZOPE/lib/python/Products). pkg_resources uses the same mechanism for
namespace packages (that's what the
pkg_resources.declare_namespace('Products') call is all about); it
appends to __path__.
Therefore, Zope will treat /path/to/the/CMFCore-egg/Products as another
directory that contains a product (in this case just one, CMFCore). Thus
the standard product override rules apply when the same product is
installed in multiple directories.
I updated the proposal text with this information.
I imagine it would be pretty trivial to write something that would
generate a setup.py from an svn:externals property so you could create a
"Products" distribution in one fell swoop, especially since the entry
point section of the setup.py can handle config parser output.
-w
--
-- d. whit morriss --
- senior engineer, opencore -
- http://www.openplans.org -
- m: 415-710-8975 -
"If you don't know where you are,
you don't know anything at all"
Dr. Edgar Spencer, Ph.D., 1995
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: Ian Bicking wrote: Martin Aspeli wrote: whit wrote: actually, in my current workplace, workingenv is the standard way to set up one's dev environment. but in the context of the previous statement, familar is perhaps a better word. I'm still not clear how widely used workingenv is? Is it "officially endorsed" anywhere else? It steps more lightly than buildout does. What does that mean? It implements less and relies more on what other tools already do. It's also mostly equivalent in mechanism to virtual-python, which is used quite widely. Both use setuptools conventions more closely than buildout does. It would be nice if I could say "then you get access to all the setuptools-related management tools", except there are almost none :( But if they *did* exist you'd get access to them ;) I suggest that workingenv and buildout are both such tools. Build stuff seems surprisingly contentious. The debate around setuptools itself has always been far more difficult than it should be; there's a lot of stop energy. So the Python community as a whole is moving very slowly on this stuff. I suggest that, other than contention, this is somewhat healthy. People with different goals will often need different tools and make different tradeoffs. Sure, but I'd also like if there was a clear story for people doing this sort of stuff. I hate the difficulty describing how to do this general stuff, especially when describing it to people who don't even know what their goals are and just need *one* good solution rather than a choice of solutions. Which is to say, I'd rather we try to figure out... something, rather than just chock it up to different goals and go our separate ways. I haven't yet figured out what that something is, and probably that's the hard part. Maybe part of it is a clear and simple scaling from something fairly ad hoc (like a workingenv that you've manually installed stuff into) into something more formalized (like a buildout). Casual and beginning users work best with something they can directly touch and inspect. In a more formalized system it's better to have a central place that described intention and the full system -- the casual user probably hasn't figured out their intention or the full system until well after they've completed their task. Of course, some of buildout's scope is outside of workingenv's -- like building non-Python libraries, and setting up specific application instances. I think it's fine if non-Python libraries require a more advanced setup, but application instances are something I'd like to have a better story for. (Indirectly I'm still trying to figure out the best way to create application instances for PasteDeploy apps too -- I have some stuff in there, but I'm not terribly happy with it, and I haven't figured out what instance creation should be attached to.) -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
On Jan 29, 2007, at 2:25 PM, Ian Bicking wrote: Jim Fulton wrote: Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment Yes, depending on how complex Pylon's setup was. For example, if Pylons exposed it's scripts using setuptools entrypoints, you you could install them with buildout. Pylon's doesn't have its own scripts. Generally an application will require Pylons, Pylons requires PasteScript, and PasteScript provides a paster script. A quirk in buildout was that asking for the paster script to go in bin/ didn't seem to work unless I specifically was installing PasteScript. Anyway, they all use entry points, so that part is fine. Right. The eggs recipe only installs scripts for the named eggs, not for dependencies. I've considered encouraging each WSGI/PasteDeploy application to ship its own script that hides the existence of paster (or any specific framework). But that doesn't seem quite right either, since a WSGI application can be (and in production usually is) used as a library in addition to a stand-alone application. At that point it becomes similar to a Zope product, or Plone content type, or... well, I guess there's limits to the similarity as I try to find analogs. I guess that's the style differences between Zope and... all the other frameworks. I'm not sure what name really describes the difference in pattern. I think I do. :) Traditionally, Zope was a pluggable *application*, rather than a library. Typically eggs are meant to be used as libraries. We're moving in the direction of having most of Zope available as eggs that can be used as libraries. Zope 3 is already being widely used that way. In the future there will still be pluggable applications (e.g. Plone) built on libraries provided by the Zope, Plone, and other projects. (in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them). Zope startup scripts are sort of interesting because they are instance specific -- combining instance-specific configuration with software definition. This is something that setuptools entry points don't handle very will by themselves. That's why buildout has custom script generation facilities that alllow definition of extra initialization code and entry point arguments. I wouldn't be surprised if Pylons had similar needs. You just have to give the config file as an argument when you start the server, there's nothing setup that binds a script with a config file. Right. And for convenience, you often want that. I played with a #! setup where the config file became a script, but I found it rather hard to work with (there's a lot of problems with how #! works across platforms). yup So for now it's just always explicit, and if you want something else you have to set up your own shell scripts. I think that in practice, you'll want to automate this. At least we did. In many cases, you can avoid generating shell scripts and generate Python scripts instead. I like this much better and the techniques that Phillip Eby worked out can be used to make this work nicely on both Unix-based systems and Windows. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
On Jan 29, 2007, at 2:03 PM, Ian Bicking wrote: Jim Fulton wrote: Ian Bicking wrote: What I would *like* the distinction between workingenv and buildout to be is that workingenv is interactive (i.e., install with easy_install) and buildout is declarative (i.e., specify your environment with buildout.cfg). Well said. I was looking for a way to bring this up in the discussion. This is a key difference. When you use workingenv, how to you reproduce an existing workingenv? Is there something you can check in and later check out to get the same workingenv? Or is this just not in scope? Maybe it should be out of scope, but I have built something into workingenv for it. You specify a file or URL with --requirements, a text file of things to install. You can also indicate --find-links and --editable in the file, as well as refer to another requirements file. The files look like: Pylons==0.8.2 # Adds several eggs and links that aren't on PyPI: -f http://pylonshq.com/download/0.8.2/ ZPTKit SQLObject==0.7.2 Then it will install all these packages. You might then put this at http://svn.myrepo.org/svn/projects/pylons_project.txt and then create another like: -r http://svn.myrepo.org/svn/projects/pylons_project.txt -f http://svn.myrepo.org/svn/projects/project_index.html -e MyProject This would set up the basic Pylons environment, but also add an editable version of MyProject into src/, including activating the file. (Notably MyProject *should* require Pylons and the other packages, seemingly rendering the pylons_project.txt useless, but probably *shouldn't* require exactly Pylons==0.8.2 because then you'd have to break that requirement to even test if such an exacting requirement was actually true.) So this is a bit like the buildout configuration file. Jim -- Jim Fulton mailto:[EMAIL PROTECTED]Python Powered! CTO (540) 361-1714 http://www.python.org Zope Corporationhttp://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment Yes, depending on how complex Pylon's setup was. For example, if Pylons exposed it's scripts using setuptools entrypoints, you you could install them with buildout. Pylon's doesn't have its own scripts. Generally an application will require Pylons, Pylons requires PasteScript, and PasteScript provides a paster script. A quirk in buildout was that asking for the paster script to go in bin/ didn't seem to work unless I specifically was installing PasteScript. Anyway, they all use entry points, so that part is fine. I've considered encouraging each WSGI/PasteDeploy application to ship its own script that hides the existence of paster (or any specific framework). But that doesn't seem quite right either, since a WSGI application can be (and in production usually is) used as a library in addition to a stand-alone application. At that point it becomes similar to a Zope product, or Plone content type, or... well, I guess there's limits to the similarity as I try to find analogs. I guess that's the style differences between Zope and... all the other frameworks. I'm not sure what name really describes the difference in pattern. (in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them). Zope startup scripts are sort of interesting because they are instance specific -- combining instance-specific configuration with software definition. This is something that setuptools entry points don't handle very will by themselves. That's why buildout has custom script generation facilities that alllow definition of extra initialization code and entry point arguments. I wouldn't be surprised if Pylons had similar needs. You just have to give the config file as an argument when you start the server, there's nothing setup that binds a script with a config file. I played with a #! setup where the config file became a script, but I found it rather hard to work with (there's a lot of problems with how #! works across platforms). So for now it's just always explicit, and if you want something else you have to set up your own shell scripts. -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: Ian Bicking wrote: What I would *like* the distinction between workingenv and buildout to be is that workingenv is interactive (i.e., install with easy_install) and buildout is declarative (i.e., specify your environment with buildout.cfg). Well said. I was looking for a way to bring this up in the discussion. This is a key difference. When you use workingenv, how to you reproduce an existing workingenv? Is there something you can check in and later check out to get the same workingenv? Or is this just not in scope? Maybe it should be out of scope, but I have built something into workingenv for it. You specify a file or URL with --requirements, a text file of things to install. You can also indicate --find-links and --editable in the file, as well as refer to another requirements file. The files look like: Pylons==0.8.2 # Adds several eggs and links that aren't on PyPI: -f http://pylonshq.com/download/0.8.2/ ZPTKit SQLObject==0.7.2 Then it will install all these packages. You might then put this at http://svn.myrepo.org/svn/projects/pylons_project.txt and then create another like: -r http://svn.myrepo.org/svn/projects/pylons_project.txt -f http://svn.myrepo.org/svn/projects/project_index.html -e MyProject This would set up the basic Pylons environment, but also add an editable version of MyProject into src/, including activating the file. (Notably MyProject *should* require Pylons and the other packages, seemingly rendering the pylons_project.txt useless, but probably *shouldn't* require exactly Pylons==0.8.2 because then you'd have to break that requirement to even test if such an exacting requirement was actually true.) workingenv also saves some of these settings (like the requirements file) so you can rerun it on a directory. If the requirements file had changed you'd get the updates, or if workingenv had changed it'd rewrite its files. Ideally you'll switch things around, rerun your tests, do some development, and once you've reached a stable state push those changes back into the requirements file. Another motivation for this was to make a simple command for people to try out a new framework, like: wget http://svn.colorstudy.com/home/ianb/workingenv/workingenv.py # You might also have several files attached to specific tutorials or # use cases: python workingenv.py \ -r http://some-framework.org/requirements.txt MyProject cd MyProject source bin/activate cd src paster create -t some_framework MyCode cd MyCode paster serve development.ini And they'd have an editable hello world at that point pretty reliably, without any system dependencies that might make the installation finicky. I had this bootstrapping idea in mind when I added --requirements, I wasn't really thinking much about deployment. It doesn't work well for things like database drivers (which are often needed for tutorials), which are often hard to install with easy_install; this is probably why I should make --site-packages default to on, and then just tell developers to use --no-site-packages to tests their requirements files. But anyway, with the goal of bootstrapping new developers without having to explain the tedious details of Python's packaging conventions, I think workingenv does reasonably well. -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Ian Bicking wrote: What I would *like* the distinction between workingenv and buildout to be is that workingenv is interactive (i.e., install with easy_install) and buildout is declarative (i.e., specify your environment with buildout.cfg). Well said. I was looking for a way to bring this up in the discussion. This is a key difference. When you use workingenv, how to you reproduce an existing workingenv? Is there something you can check in and later check out to get the same workingenv? Or is this just not in scope? They aren't *far* apart, but there's some distinctly different mechanics. And different goals. > workingenv's techniques (which are mostly just setuptools/easy_install's) aren't great, and I wish they were better, but buildout's don't seem right to me either. I'm very happy with the approach so far. I know lots of improvements can be made and I expect buildout to evolve over time and people learn from experience with it. Feedback is very welcome. A number of comments you've made have led to improvements and your *specific* comments are much appreciated. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Martin Aspeli wrote: >> Unless Utopia really exists I think developers all have their own >> thoughts about setting up their development environment. > > Maybe. Except if we (the plone core developers) use ploneout then we are > all using the same environment, and we duplicate less work. Same reasoning I had with instancemanager, so I agree. I explicitly made it reasonable easy to set some defaults to have it fit everyone's favourite way of ordering his stuff. Instance in ~/instances/philips by default, but Jean-Paul can set it to a default of ~/Sites/philips, for instance. But even with that level of adaptability-to-individual-taste, not everyone uses it for our zest projects. Rocky not, iirc, for instance. Neither nouri. So: "it duplicates less work" and "we all would have the same environment" are great reasons, but they don't always work :-) >> Again most developers are pretty demanding about setting up a >> development environment. Here at Zest most of us are comfortable with >> instancemanager. > > That's possibly something to do with the fact that you wrote it :) > > I'm not saying it isn't great, and there's no reason why people > shouldn't use it if it works for them. But zc.buildout predates it, > seems to be adopted more and more in the Zope 3 world (including people > like grok), seems more flexible to me... and I like it. :) Well, predates it... Theoretically, yes. But in practice it was some internal zope corp tool. I only first noticed it when Jim gave a talk about the new improved egg-using version at 2006's Europython. Later that same day I gave a lightning talk about instancemanager. So at least to my viewpoint, it has the same age. http://vanrees.org/weblog/archive/2006/07/04/jim-fulton-zc-buildout http://vanrees.org/weblog/archive/2006/07/04/europython-lightning-talks > Ultimately, though, it doesn't really matter what people use. However, > at the upcoming Amsterdam sprint, I intend to ask people to do this: > > $ svn co https://svn.plone.org/svn/plone/ploneout/trunk plone3 > $ cd plone3 > $ python bootstrap.py > $ ./bin/buildout > > and they will have a Plone 3 instance to develop from. I won't be > helping anyone set up Plone in another way, simply because from previous > sprints I know that we waste an awful lot of time on something so simple > as getting people to symlinks eggs around (yipes). During Plone 3 > development, dealing with two bundles (which meant each review bundle > needed two bundles as well) that explicitly check out various eggs was a > pain in the ass as well. Something that automates this is very useful. I started instancemanager for the same reason: to do away with hour-long bughunts just because someone had an old product lying around. >From what I understand, Jean-Paul has an instancemanager configuration that builds a nice plone 3.0 instance from the normal, unmodified bundle with just a few lines of instancemanager configuration. Just keep the bundle sane and everything should be OK. Either buildout/ploneout or instancemanager can help. The goal is the same. Reinout -- Reinout van Rees r.van.rees @ zestsoftware.nl http://vanrees.org/weblog/ http://zestsoftware.nl/ "Military engineers build missiles. Civil engineers build targets." ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 7 OK
Summary of messages to the zope-tests list. Period Sun Jan 28 12:00:00 2007 UTC to Mon Jan 29 12:00:00 2007 UTC. There were 7 messages: 7 from Zope Unit Tests. Tests passed OK --- Subject: OK : Zope-2.6 Python-2.1.3 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:05:38 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007139.html Subject: OK : Zope-2.6 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:07:09 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007140.html Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:08:39 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007141.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:10:09 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007142.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:11:39 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007143.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:13:09 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007144.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Sun Jan 28 21:14:39 EST 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-January/007145.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
