[Zope-dev] Re: [Repoze-dev] repoze.bfg
Chris McDonough wrote: I had planned to create another package named repoze.lemonade which: ... - Did indexing of content. What were you thinking of for indexing? Just catalog stuff? More general? There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened. That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.) -- Ian Bicking : [EMAIL PROTECTED] : http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [Repoze-dev] repoze.bfg
Chris McDonough wrote: There's been a tension in the opencore stuff with the catalog, mostly that it's easy to setup and use for things, but it doesn't really work for things outside of the ZODB. Or, I guess theoretically you could catalog things not in the ZODB, but it's never happened. IMO, mostly it's a matter of deciding what index and catalog means for searching. If you only need full-text search, some indexing systems are totally inappropriate. Likewise, if you only need field indexes that match just one particular value, it's sometimes vastly simpler just to come up with your own system based on, e.g. a relational table, than it is to try to use somebody else's indexer or catalog. They are similar in that they both need to get information about updates, and a way to reindex. Full text search can be a little lazier, as being 100% up-to-date isn't such a big issue. That said, there's a real need for cross-system indexing of different kinds. There's text search indexes, but other kinds of more strict indexes also make sense. It would be very handy to have an index that wasn't tied to the ZODB, a database, or anything else. (It could be implemented using the ZODB or a database, of course.) Xapian seems like a good choice too. It has its own persistence mechanism and reasonable Python bindings (although from experience maybe slightly memory-leaky). Yes, once you get the documents into it. Actually, it's really a many-to-many notification system that's needed. We have one that needs documenting and review (http://www.openplans.org/projects/cabochon/project-home) but while it handles notifications and events (as do several other systems), that doesn't cover reindexing the site. And then you also need a set of useful endpoints, but those can grow over time. -- Ian Bicking : [EMAIL PROTECTED] : http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneenv - Or how using workingenv for a common Zope2 project might look like ;-)
Daniel Nouri wrote: [...] - There is a single file that shows me what eggs and development eggs make up my package. This makes it easy to add new eggs, for example - and also easy to remove them again. With workingenv-based solutions, you can specify a file full of eggs to install when it's first set up, but from that point onwards, the environment can grow as you easy_install new things. It's not immediately clear to me how you reconcile all the eggs you've installed (not all of which may be needed when you're done trying things out, and some of which may just be dependencies you don't want to think about directly) into a list that are clearly dependencies of your application. Querying the list of installed eggs is what tools like Yolk[5] are good for. Note that for setuptools packages, the setup.py file lists all dependencies. If you develop for e.g. Listen and Plone, you only need to keep track of Listen and Plone. One of the things that I think is pretty easy with workingenv, and a bit confusing with buildout, is moving one package into development. In workingenv you get the package you want (however you do that -- check out a branch, make your own local repository, unpack a tarball, etc), and you run (after activating the environment) python setup.py develop. Or if the package isn't using setuptools, python -c 'import setuptools; execfile(setup.py)' develop. Note that this is actually one of the few places you actually have to activate the environment. And heck, if I just compiled a little something into bin/python then even that wouldn't be necessary. (Maybe even a hard link would be enough, I'm not sure.) In buildout it's a bit more complicated. You can move an egg into develop-eggs, but that relies on buildout finding the right package. That's not really that easy, especially because setuptools only really understands packages being newer or older, it doesn't understand things like branches. It's hard even when you don't have this problem. - With workingenv, when I run easy_install SomePackage I need to worry about whether I'm actually in the global environment or the workingenv of the instance. That is, workingenv requires activation (putting your shell into a special state where the python environment is the one in your workingenv, until you deactivate it). By contrast, other people may not like the add to buildout.cfg, re-run buildout dance that buildout uses instead. If you use the easy_install script in the workingenv bin/, then you don't have to activate the environment. Very similar to buildout, workingenv scripts contain their path/environment. - It works in Windows. :) I have no idea how hard it's to make ploneenv work on Windows, but I hope it's not too bad. The scripts Hanno wrote do give us a near-proper zopectl for Windows as well, which is nice. I imagine these could be adapted to be used with plain Zope instances, though. I assume workingenv gives us setuptools script support locally as well. Support for Windows should be fairly trivial. I would appreciate it if someone (Hanno?) gave it a try. Basically, all we need is the correct way to patch bin/zopectl so that it runs the bin/activate.bat script before startup. We shouldn't seriously consider runs on Windows as an argument for ploneout, just because ploneenv hasn't seen a Windows developer yet. :-) Workingenv does, as far as I know, work with Windows. At least I've received several patches (I've never used it myself). -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: Ian Bicking wrote: What I would *like* the distinction between workingenv and buildout to be is that workingenv is interactive (i.e., install with easy_install) and buildout is declarative (i.e., specify your environment with buildout.cfg). Well said. I was looking for a way to bring this up in the discussion. This is a key difference. When you use workingenv, how to you reproduce an existing workingenv? Is there something you can check in and later check out to get the same workingenv? Or is this just not in scope? Maybe it should be out of scope, but I have built something into workingenv for it. You specify a file or URL with --requirements, a text file of things to install. You can also indicate --find-links and --editable in the file, as well as refer to another requirements file. The files look like: Pylons==0.8.2 # Adds several eggs and links that aren't on PyPI: -f http://pylonshq.com/download/0.8.2/ ZPTKit SQLObject==0.7.2 Then it will install all these packages. You might then put this at http://svn.myrepo.org/svn/projects/pylons_project.txt and then create another like: -r http://svn.myrepo.org/svn/projects/pylons_project.txt -f http://svn.myrepo.org/svn/projects/project_index.html -e MyProject This would set up the basic Pylons environment, but also add an editable version of MyProject into src/, including activating the file. (Notably MyProject *should* require Pylons and the other packages, seemingly rendering the pylons_project.txt useless, but probably *shouldn't* require exactly Pylons==0.8.2 because then you'd have to break that requirement to even test if such an exacting requirement was actually true.) workingenv also saves some of these settings (like the requirements file) so you can rerun it on a directory. If the requirements file had changed you'd get the updates, or if workingenv had changed it'd rewrite its files. Ideally you'll switch things around, rerun your tests, do some development, and once you've reached a stable state push those changes back into the requirements file. Another motivation for this was to make a simple command for people to try out a new framework, like: wget http://svn.colorstudy.com/home/ianb/workingenv/workingenv.py # You might also have several files attached to specific tutorials or # use cases: python workingenv.py \ -r http://some-framework.org/requirements.txt MyProject cd MyProject source bin/activate cd src paster create -t some_framework MyCode cd MyCode paster serve development.ini And they'd have an editable hello world at that point pretty reliably, without any system dependencies that might make the installation finicky. I had this bootstrapping idea in mind when I added --requirements, I wasn't really thinking much about deployment. It doesn't work well for things like database drivers (which are often needed for tutorials), which are often hard to install with easy_install; this is probably why I should make --site-packages default to on, and then just tell developers to use --no-site-packages to tests their requirements files. But anyway, with the goal of bootstrapping new developers without having to explain the tedious details of Python's packaging conventions, I think workingenv does reasonably well. -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: Similarly, say I had two applications, one in Zope and one in Pylons, part of the same deployment (possibly interwoven using wsgi). If I installed elementtree as an egg in buildout.cfg, I'd expect it to be available to both. If that means patching some of pylon's confg files or startup scripts to explicitly reference eggs that buildout is managing, it would mean that I'd need a very specific recipe for Pylons development that would need to know a lot of detail about how that sets up its environment Yes, depending on how complex Pylon's setup was. For example, if Pylons exposed it's scripts using setuptools entrypoints, you you could install them with buildout. Pylon's doesn't have its own scripts. Generally an application will require Pylons, Pylons requires PasteScript, and PasteScript provides a paster script. A quirk in buildout was that asking for the paster script to go in bin/ didn't seem to work unless I specifically was installing PasteScript. Anyway, they all use entry points, so that part is fine. I've considered encouraging each WSGI/PasteDeploy application to ship its own script that hides the existence of paster (or any specific framework). But that doesn't seem quite right either, since a WSGI application can be (and in production usually is) used as a library in addition to a stand-alone application. At that point it becomes similar to a Zope product, or Plone content type, or... well, I guess there's limits to the similarity as I try to find analogs. I guess that's the style differences between Zope and... all the other frameworks. I'm not sure what name really describes the difference in pattern. (in the same way, the z2c.recipe.zope2instance recipe knows about how the zopectl and runzope scripts set their PYTHONPATH and patches them). Zope startup scripts are sort of interesting because they are instance specific -- combining instance-specific configuration with software definition. This is something that setuptools entry points don't handle very will by themselves. That's why buildout has custom script generation facilities that alllow definition of extra initialization code and entry point arguments. I wouldn't be surprised if Pylons had similar needs. You just have to give the config file as an argument when you start the server, there's nothing setup that binds a script with a config file. I played with a #! setup where the config file became a script, but I found it rather hard to work with (there's a lot of problems with how #! works across platforms). So for now it's just always explicit, and if you want something else you have to set up your own shell scripts. -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: Ian Bicking wrote: Martin Aspeli wrote: whit wrote: actually, in my current workplace, workingenv is the standard way to set up one's dev environment. but in the context of the previous statement, familar is perhaps a better word. I'm still not clear how widely used workingenv is? Is it officially endorsed anywhere else? It steps more lightly than buildout does. What does that mean? It implements less and relies more on what other tools already do. It's also mostly equivalent in mechanism to virtual-python, which is used quite widely. Both use setuptools conventions more closely than buildout does. It would be nice if I could say then you get access to all the setuptools-related management tools, except there are almost none :( But if they *did* exist you'd get access to them ;) I suggest that workingenv and buildout are both such tools. Build stuff seems surprisingly contentious. The debate around setuptools itself has always been far more difficult than it should be; there's a lot of stop energy. So the Python community as a whole is moving very slowly on this stuff. I suggest that, other than contention, this is somewhat healthy. People with different goals will often need different tools and make different tradeoffs. Sure, but I'd also like if there was a clear story for people doing this sort of stuff. I hate the difficulty describing how to do this general stuff, especially when describing it to people who don't even know what their goals are and just need *one* good solution rather than a choice of solutions. Which is to say, I'd rather we try to figure out... something, rather than just chock it up to different goals and go our separate ways. I haven't yet figured out what that something is, and probably that's the hard part. Maybe part of it is a clear and simple scaling from something fairly ad hoc (like a workingenv that you've manually installed stuff into) into something more formalized (like a buildout). Casual and beginning users work best with something they can directly touch and inspect. In a more formalized system it's better to have a central place that described intention and the full system -- the casual user probably hasn't figured out their intention or the full system until well after they've completed their task. Of course, some of buildout's scope is outside of workingenv's -- like building non-Python libraries, and setting up specific application instances. I think it's fine if non-Python libraries require a more advanced setup, but application instances are something I'd like to have a better story for. (Indirectly I'm still trying to figure out the best way to create application instances for PasteDeploy apps too -- I have some stuff in there, but I'm not terribly happy with it, and I haven't figured out what instance creation should be attached to.) -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Martin Aspeli wrote:
Eggs exist to pkg_resources (the runtime portion of setuptools) simply
by being available on a path. E.g., if you have lib/python/ on the
path, and lib/python/Foo-1.0.egg/ exists, then if you do
pkg_resources.require('Foo') that will add lib/python/Foo-1.0.egg/ to
the path, and you can import from it. When you can actually import from
the egg it is active. You can also see its entry points at that time,
but not if it is just available.
If lib/python/Foo-1.0.egg/ is on the path to start with you can import
from it directly.
This is what zc.reipe.egg does I believe.
It activates (i.e., adds eggs to the path) in the scripts. I think
setuptools' egg activation will be superfluous at that point, because a
complete and consistent set of eggs is activated up-front. I'm not even
sure if you can activate any extra eggs (that might be available on
eggs/ or something) at that point, because the containing directory
isn't on the path. But since buildout controls those directories that
would be fine, I guess, assuming you don't put anything in those
locations on your own.
In that way I suppose lib/pythonX.Y/ would be a misnomer of sorts, as
eggs/ is just a storage area for eggs that are managed elsewhere (in
buildout.cfg).
OTOH, easy_install and setuptools generally has no problem dealing with
eggs that *aren't* on the path, if you use a .egg-link or .pth file to
point to the egg (preferably both). So while Jim hopes to have a
centrally located repository of eggs (where buildouts only draw on a
subset), this is already possible using setuptools' conventions. Then
lib/python could become
(BTW, existing Zope 2 stuff often seems to prefer lib/python/ over
lib/pythonX.Y/, for reasons I don't really understand -- it basically
prefers the distutils --home option over the --prefix option. So using
workingenv with Zope I use workingenv.py --home, which sets up the
environment slightly differently)
Secondly, have you seen this proposal? I'm sure you'd have something to
add.
http://wiki.zope.org/zope2/EggifyingZopesExtensionMechanismQuotProductsQuot
zc.buildout also uses entry points for its recipes, and several people
have talked about entry points instead of ZCML slugs, so I think things
are moving in this direction.
I left some comments over here a while back:
http://wiki.zope.org/zope3/ZopeConfigurationEggSupport
But that's more about ZCMLy things. Philipp's proposal looks fine to
me, though it doesn't seem to address how products would be activated or
not (it seems to mention that eggs wouldn't be installed as products by
default, but doesn't propose a mechanism to select the products).
Because of the difficulty scanning for entry points if the egg isn't
activated, I'd advocate sticking to something like enumerating (or
blacklisting) distribution names. Then if you have:
[zope2.products]
CMFDefault = Products.CMFDefault
the entry point name CMFDefault wouldn't be significant. Or you could
select not just distributions, but distributions+entry_point name. In
Paste I use egg:Package#entry_point_name (where entry_point_name
defaults to main). I believe buildout uses Package:entry_point_name,
where entry_point_name defaults to default. If we could all agree on
a convention, that'd be nice.
I strongly prefer main, because that's really what I think the unnamed
entry point is, not just default -- default just refers to the
parsing/locating strategy, not the intent. I also would like something
that is unambiguously different from a Python object specifier, so that
you could take either an explicit object reference or an entry point in
the same place and be able to clearly tell the difference. But if we
can agree on something and document it, I'm willing to update Paste
Deploy to support the new/standard syntax.
Ian
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Martin Aspeli wrote: whit wrote: actually, in my current workplace, workingenv is the standard way to set up one's dev environment. but in the context of the previous statement, familar is perhaps a better word. I'm still not clear how widely used workingenv is? Is it officially endorsed anywhere else? It steps more lightly than buildout does. It's also mostly equivalent in mechanism to virtual-python, which is used quite widely. Both use setuptools conventions more closely than buildout does. It would be nice if I could say then you get access to all the setuptools-related management tools, except there are almost none :( But if they *did* exist you'd get access to them ;) Build stuff seems surprisingly contentious. The debate around setuptools itself has always been far more difficult than it should be; there's a lot of stop energy. So the Python community as a whole is moving very slowly on this stuff. Ian ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Martin Aspeli wrote: I don't have a usecase for executing the scripts with any python interpeter other than the one which ran setuptools to generate them, and therefore don't care for the hard-wired path manipulation I would agree that having to mangle multiple scripts is annoying. On the other hand, I find the activate/deactivate dance annoying. I rather think that the scripts that buildout has created should not require any special activation or introduction to a particular environment. workingenv doesn't require that you activate before running scripts, each script contains in effect what active/deactive does. Basically it munges script creation to do: #!/usr/bin/python -S import sys, os sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'lib/python%s.%s' % (sys.version_info[0], sys.version_info[1])) import site virtual-python -- which might be the better/simpler solution, if you just ignore the tree of symlinks -- selects the environment through the interpreter itself (i.e., #!/path/to/virtual-python/bin/python). Ian ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Martin Aspeli wrote:
I don't have a usecase for executing the scripts with any python
interpeter other than the one which ran setuptools to generate them, and
therefore don't care for the hard-wired path manipulation
I would agree that having to mangle multiple scripts is annoying. On
the other hand, I find the activate/deactivate dance annoying. I
rather think that the scripts that buildout has created should not
require any special activation or introduction to a particular
environment.
Perhaps a solution is to let buildout put things in standard places
(as I'm typing this, we now have all the ploneout .egg and .egg-link
files/directories in ${buildout}/lib/python) and then have an optional
workingenv at the root of the buildout that could be activated when
you want that, but let the buildout-managed scripts (e.g. zope
start/stop and test runner) not require this?
I have a feeling there's more to this than agreeing on directory
layout though. But I hope it oughtn't be *that much more*.
I did some experimentation with this. Basically, I did:
- make egg-directory and develop-egg-directory in buildout.cfg be
lib/python2.4 (which is what workingenv uses)
- run the buildout again
- create standard workingenv (python workingenv.py .) in the root
buildout directory
The results are fairly encouraging:
- buildout-installed eggs don't work in the workingenv jail...
- ...but they do if I manually add them to lib/python2.4/easy_install.pth
- easy_installed packages (i.e. installed with workingenv's
easy_install) are available to zope
This was only a brief test, but seems to me that if buildout was
managing the easy_install.pth file we wouldn't be a million miles off
something where the source bin/activate dance was optional and would
work as a generic solution, but would not be required for zope's startup
and other buildout-managed scripts.
I would assume that buildout is specifically disabling easy_install's
updating of easy-install.pth -- buildout is still installing packages
with easy_install, but with some monkeypatches of the command.
workingenv also monkeypatches easy_install, but only to change script
creation (as noted in another email) and to keep it from rejecting
workingenv's site.py.
Anyway, if easy_install was allowed to do its thing then they'd be more
compatible.
However, I'm not sure whether and how buildout should manage
easy_install.pth in a non-hacky way. A hacky way would be to have a
buildout recipie that, if included, would just patch that file up based
on buildout['buildout']['eggs'] (i.e. the list of eggs listed in
buildout.cfg under the main [buildout] section).
I don't really understand the purpose of easy_install.pth and why it
matters here, though, so suggestions welcome. :)
Eggs exist to pkg_resources (the runtime portion of setuptools) simply
by being available on a path. E.g., if you have lib/python/ on the
path, and lib/python/Foo-1.0.egg/ exists, then if you do
pkg_resources.require('Foo') that will add lib/python/Foo-1.0.egg/ to
the path, and you can import from it. When you can actually import from
the egg it is active. You can also see its entry points at that time,
but not if it is just available.
If lib/python/Foo-1.0.egg/ is on the path to start with you can import
from it directly. The eggs listed in easy-install.pth are put on the
path, and hence are activated by default. You can still require a
different egg version (e.g., Foo==1.1), and I believe setuptools will
(if it can) switch the path to remove the default activated egg and put
on the new egg.
If you install a package with easy_install -m (multiversion) then none
of the versions go on the path, and eggs will only be added when they
are specifically required. Any egg listed in install_requires in
setup.py will get added as soon as you require the original egg, and
when you run a script from an egg it'll activate that egg as well, so if
you are thorough with your requirements then none of the eggs need to be
activated.
One problem if you are using entry points is that if your egg isn't
activated and you are searching entry points by name (not by egg) then
your egg has to be activated to work (Buffet and paster create templates
are both searched by name, for example). I have expected this to cause
more problems than it has, but since people have stopped leaning too
heavily on setuptools versioning (thanks to virtual-python, workingenv,
and buildout) I think it hasn't come up much. Anyway, if Zope was to
start using entry points to replace some of its ZCML (which I personally
think would be a good idea) this is something to be aware of.
Ian
___
Zope-Dev maillist - Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
[Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
whit wrote: Not everybody likes the activate dance. With buildout, you don't need it. The recipes make sure that the scripts that get installed into the buildout's 'bin' directory have the right PYTHONPATH set and have access to the eggs you requested for the buildout. is there really a difference between zopectl and source bin/activate? I guess the main difference here is where PYTHONPATH gets set and how people work with it. for example, if you just start python and want to play with code sounds like with zc.buildout you are out of luck for things installed in the buildout. In our situation, we might have multiple versions of software running on different processes started in different workingenv (often not even using zope).being able to contextualize the python path is a useful development tool; this is understandable for buildout to avoid(as a deployment tool), but if we are considering using buildout as a prescribed way for people to manage their development environments, it seems lacking. I think this is the basic difference -- workingenv is development-centric, while buildout is deployment-centric. This does not necessarily mean the best tool for the job, because focusing on development and ignore deployment isn't a good job, nor the other way around. At TOPP we know how to address both stories with workingenv. We haven't figured it out with buildout, despite trying. And we definitely aren't satisfied with just a deployment tool that doesn't address our development needs. But there's a lot of use cases for Plone specifically where carefully developing just a deployment solution makes sense. That doesn't make sense for us, because we're not in the kind of consulting business where the relative scale of development and deployment works like that. But eh; the presence of a buildout for Plone doesn't hurt our position any. Anything that gets rid of another ad hoc crappy deployment is good by me. And if other people are working on it, all the better! If *Plone* becomes incompatible with workingenv that'd be bothersome (though it was not compatible with workingenv without some work in the first place). But if a buildout is incompatible, eh... who knows, it might even make sense to create something like a freeze this workingenv as a buildout script. That one directory structure can't be both at the same time isn't a huge problem in my mind. I'm not too fond of zc.buildout's directory scheme eiher. In particular, I wish that it would use 'lib/python' instead of 'eggs' and 'src' instead of 'develop-eggs'. I don't know enough zc.buildout details to be able to say whether that can be chagned by remimplementing the egg installation recipe. I would sure hope it is. +1 this may be all that is required to make the two play well together(or have buildout respect an existing workingenv when doing it's local install of eggs). Maybe it's just a matter of zc.buildout seeing workingenv is in effect and not composing special pythonpaths. harmony is my interest here. workingenv is pretty low-level and works hard to work well with python. there is no reason that zc.buildout should have a problem with that.. it just needs to do less when appropriate. Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy. I'd rather see easy-install.pth become a better package database, or some other strategy of external requirement specifications, instead of building it into scripts. I understand the issue Jim is trying to solve here, but putting everything into the buildout.cfg and then imperatively setting up the files from there bothers me and does not make development easy IMHO. That's mostly the problem. Then workingenv would do its part by monkeypatching distutils and setuptools to install things locally, and changing site.py to not automatically pick up things globally. And buildout could do its stuff that applies to other more complicated setups than just setting up some libraries, which is about where workingenv stops. Well, what I'd *really* like is an idea of a working environment that applies more widely than Python, because other languages (including but not limited to the dynamic languages) have all these same problems, and we all have half-assed solutions. But I don't even know how to start that conversation. And don't get me started on how the distributions are going to look at all this! Ian ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists -
Re: [Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
Jim Fulton wrote: If *Plone* becomes incompatible with workingenv that'd be bothersome I agree. But if a buildout is incompatible, eh... who knows, I would hope that buildout would not have to be compatible with workingenv, whatever that means, in order for Plone to be compatible. Then again, I'm not sure what compatibility means in this context. It would be a concern if, for instance, Plone started depending on buildout recipes for installation, without plain distutils recipes. Of course right now there are no distutils recipes for old-style Products. So actually it's an active issue -- if buildout enables Plone to keep from moving to distutils/setuptools/egg-style package deployment (because buildout is flexible enough to support the old patterns) then that would make it harder for workingenv. But I don't think anyone is proposing that buildout means that Plone (and Zope 2/Five) shouldn't continue its move towards traditional Python packaging. So basically I would just hope that Plone doesn't lean to heavily on buildout. it might even make sense to create something like a freeze this workingenv as a buildout script. That one directory structure can't be both at the same time isn't a huge problem in my mind. Deployment involves far more than getting the software installed. Yes; in a workingenv model you start with an environment, then you actually make your deployment using tools of your choice. workingenv doesn't deploy for you. Admittedly your choice isn't always good, because maybe you didn't want to make a choice ;) Path names aren't really the problem. We got a little guerrilla war going on over the setuptools' script-generating monkey patches. We'd both probably prefer a proper way to change how setuptools generates scripts, but it's not clear that we could really be compatible -- enumeration vs. changing the path is a totally different strategy. Um, we're both changing the path -- just in different ways. Maybe enumeration vs adding a directory of eggs is a better description. Setuptools controls the directory of eggs (via easy-install.pth), while buildout controls the scripts and doesn't give setuptools that control. I'd rather see easy-install.pth become a better package database, or some other strategy of external requirement specifications, instead of building it into scripts. We certainly disagree there. OTOH, I wouldn't be opposed to having a recipe for generating scripts that used the .pth files created by workingenv. I'm not sure what the problem would be? I appreciate the desire for a set of requirements that is different from the requirements of the package, and workingenv has some support for that (with the --requirements option), but it's a different style from buildout. easy-install.pth would almost be okay, except for the fact that it is constantly pointing to things that don't exist, has setuptools' hacks in it (to work around the atrocious Python standard site.py), and doesn't describe intent, it's just an enumeration of what is available and activated (i.e., available without specifically requiring something). Besides all that, it's still a workable foundation IMHO. -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: ploneout - Or how using zc.buildout for a common Zope2 project might look like
/activate lets you into that jail, and lets you work there. There is no way into a buildout. I'm not familiar with bin/activate, but it sounds like an interpreter script created with buildout. It's created by workingenv, and you have to source it because basically its only function is to add the workingenv/lib/pythonX.Y to $PYTHONPATH. Adding that path to $PYTHONPATH is the only thing that really activates a workingenv. Frankly this weirds me out, and is a big part of my past frustration with it. Maybe that's because I'm in the relatively uncommon situation that I actually know what's going on under the hood of Python imports and packaging, and so it bothers me that I can't debug things directly. Anyway, neither requires activation when using scripts generated in the environment. And bin/activate is really just something that sets PYTHONPATH and then does other non-essential things like changing the prompt and $PATH -- I should probably document that more clearly. sounds a lot like an buildout interpreter script. Once you've changed $PYTHONPATH any Python script will notice the change. This can actually be a bit awkward if you have fully isolated the working environment, as it means a script may not see the global Python paths. But if you don't isolate the environment, the script can see the workingenv path in addition to its own. Neither can be entirely compatible with a system-wide Python installation, because Python's standard site.py f**ks up the environment really early in the process, and avoiding that isn't all that easy. This reminds me of a place where buildout is looser than workenv. buildout doesn;t try to disable anything in the system python. It just augments it. I always use a clean python, so avoiding customizations in the Python I use isn't a problem. If I wanted to take advantage of something in a system Python, as I occasionally do, I can do that with buildout. I find the isolation useful when testing things for release; I can be sure that I haven't been using any packages that I don't explicitly include in the egg requirements or instructions. But it can be annoying in other cases, like when there's a library that doesn't install cleanly (of which there's still quite a few). Anyway, if you do want to include the global packages, --site-packages will change your workingenv to do so. It could be argued that workingenv's default should be to include site-packages. Another option would be to have a tool that allows you to easily include something from the system Python (probably just a tool to manage a custom .pth file, which works even when setuptools' fairly heroic attempts to fix broken setup.py's doesn't work). -- Ian Bicking | [EMAIL PROTECTED] | http://blog.ianbicking.org ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZPT] Re: RFC: TALES adapters and TAL/Tales variable namespaces
Jim Fulton wrote: One disadvantage I see with the cast notation is that it's a bit jarring in: a/b/(adapter)c/d as the adapter is applied to a/b/c. The order just doesn't seem quite right. That is indeed jarring, especially since c isn't an object, it's just a name, and a/b/c is the object in question. This would look somewhat better with the current : mechanism, like a/b/c:adapter/d My concern with this use of : was that it looked like it should be parsed like (a/b/c):(adapter/d), as opposed to ((a/b/c):adapter)/d -- maybe using something other than : wouldn't imply this grouping. Or maybe if I got used to it the grouping would seem more natural. I guess my intuition is that / binds more closely than : (even if there isn't any real precedence at all in TAL expressions). -- Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZPT] RFC: TALES adapters and TAL/Tales variable namespaces
Jim Fulton wrote:
I've posted two proposals:
http://dev.zope.org/Zope3/TALESPathExpressionAdapters
Not much opinion. Though I fear that (adapter)object could lead to
this syntax in Python itself, which would be horrid ;) I agree that
adapter(object) is a bad direction. object*adapter looks fine to me,
and it seems reasonable that only a specific set of adapters would be
available in TAL expressions (i.e., adapters which provide ITALESAdapter).
Proposes a mechanism for easily using adapters in TALES expressions.
http://dev.zope.org/Zope3/ZPTVariableNamespaces
proposes a mechanism for qualifying names defined in TAL and used in
TALES expressions.
I'm suspicious of namespaces, as they seem redundant. Namespaces in
Python, after all, are expressed like any other attribute traversal
(e.g., os.path.exists). The analog in TAL would be adapter/foo. This
is how TAL works right now in practice, with a small number of
namespaces like request, container, etc.
I see a few problems with the current situation:
1. There's no clear way to indicate that you want to use a name as a
namespace, as opposed to a normal name. So there may be a conflict
between the adapter you want to use as a namespace, and a template
that someone writes that happens to use the variable adapter in an
unrelated way. This is fine now, because there is a fairly fixed number
of namespaces (six or seven, I think), and you just don't use those
names -- as namespaces are added (especially on a per-metal-template
basis) this conflict is more likely, and you may not know what names
will cause conflicts in the future.
But I'm not sure how bad the name conflict really is. In my experience
it's not too bad in Python code, and when it's a problem it's fairly
easily resolved. Or maybe another one or two namespaces can be added
which would sufficient, and we don't need to extend the number of
namespaces indefinitely. Like an adapter namespace and a metal
namespace (maybe you'd use things like
metal/name_of_template.pt/variable_name). To some degree this could
even be convention, instead of building it in explicitly.
2. Another issue might be the difficulty of creating a namespace for use
with templates -- with the proposal all namespaces start out empty and
ready to accept new values, but if you use normal variables they start
out as undefined, and you'd have to assign them to {} or something.
(A little thought: if you had def namespace(): return {}, then
tal:define=adapter namespace would work and reads fairly well)
3. Explicit namespaces support deeper, structured assignment (but only a
*little* deeper). Does TAL currently allow tal:define=var/attr
something? I've never tried it. It should. Maybe the specific
semantics of this assignment could be refined to resolve (2) -- e.g., if
you get a LookupError during the second-to-last segment of the
traversal, try to assign it to {}.
Anyway, whenever I look at a language with explicit namespaces (e.g.,
Ruby), it seems really pointless. I think they should be avoided, and
that normal TAL path expressions can be refined instead.
It's also annoying that we'd have namespace['adapter'] in Python
expressions. Namespaces might be a way to introduce a more accessible
set of typical functions, like DTML's nl2br and other formatting
functions -- these are currently too hard to get to. But these have to
be used with Python syntax (at least currently), and doing
namespace['formatters']['nl2br'](options['message']) is just bad. I
don't much care for tal:define=nl2br formatters:nl2br either, as it
feels like boilerplate. I suppose
path('formatters:nl2br')(path('options/message')) is maybe a little
better, but only a very little.
--
Ian Bicking / [EMAIL PROTECTED] / http://blog.ianbicking.org
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
