Re: [Zope3-Users] buildout and egg depedencies
On Dec 11, 2007 8:58 AM, Jim Fulton <[EMAIL PROTECTED]> wrote: > I think you are misunderstanding the context of Benji's answer. > > In this case, someone is running into trouble because they are trying > to mix egg-based package management with subversion-based package > management. The Zope 2 tree uses subversion. It obtains many Zope 3 > packages via subversion externals. The original poster is trying to > use some other Zope 3 packages via eggs and is packages via egg > dependencies that he already has via subversion. This is at best > annoying and at worst error provoking. Ah, sorry. I didn't realize this was the situation. But it does sound similar to the problem that affects us. We're using a mix of CVS and Subversion based package management. I want to get away from this, as it's turning into a configuration-management nightmare. And even though our internally made tools help automate checking out known-good-revisions or branches, deployments can still be much more painful than they should be. So, I want to change! It's just been hard. And a lot of that is due to time constraints on my part. I never have enough time to get to that break-through 'ah-ha!' moment. And I'm still struggling with Eggs and .egg-info directories and all of the path-monkeying involved and how it all works. I understand why they have to be there, but it goes against my intuition of just putting modules and packages into a single directory that's on PYTHONPATH and needs no other magic. And in high pressure situations, we keep reverting back to checkouts. > > ... > > Whatever. Buildout looks like it tries to address many of those > > issues, but again I find myself fighting against my natural instincts. > > Where's some end user documentation? > > http://pypi.python.org/pypi/zc.buildout > http://us.pycon.org/common/talkdata/PyCon2007/116/buildout.pdf I forgot about that second one. I think I've scanned through it before. I'll keep it close by as I try to sneak a buildout based app through the system later this week. I hope. > > The doc-test is difficult to read > > and speaks in generics, not about day-to-day problems. > > Everyone's a critic. I greatly, greatly appreciate the fact that there is documentation and that it is a test and therefor up-to-date. But it's just not good introductory nor quick reference documentation. It's an extremely intimidating document, with the good information (command line options, default buildout config options with a brief summary of each) being buried way down in the document - after a lot of things about writing Recipes (most of which seem in place as a way of providing means to show what buildout is doing within the confines of a doctest). Every time I use buildout, I scroll up and down that document trying to find what I need. And then there are the inline examples with the '%s' placeholders: awesome for testing and not bad for reading. But not great for reading, either. It adds extra line noise that I have to filter out. Under the constraints of what can be done within a doc test, I think you guys have performed miracles. And it's much much better than no documentation at all. But it's a lot of information, all presented within a single page that seems to be the de-facto homepage for the tool and the sole source of documentation (and the recipe links are busted). If the tool is maturing, it may be time to give buildout a bone-simple site, maybe all generated from Restructured Text, which could house the big-gigantic-doctest-readme and could start to house a manual written along the same lines (perhaps extracted from the big doctest readme) along with tutorials. I can understand waiting on doing that until the tool matures, but it seems to be doing alright (2-3 months between recent beta releases). > > I've gotten Buildout to work on some small components. It's great - > > check out the source, run buildout, wait, wait, wait, and then have a > > nice little self contained testing and development environment. But > > I've never been able to get a full Zope 3 "Application" up and running > > in that environment. For what it's worth, I've gotten full third party apps (z3c.formdemo) running, and I think I got a zopeproject based thing running; but I haven't been able to get our existing applications to that point. None of them are in distutils friendly format, let alone buildout friendly. That's the part I struggle with. Part of the struggle is due to us still being on CVS, but we'll be moving to Git soon, and I'll finally be able to restructure our libraries without the hacks I currently have in place. I think the information that would be most beneficial is good practices when it comes to declaring dependencies. In setup.py? In buildout.cfg? Repeated in both? Our big old libraries (which I also hope to break into smaller components as we change source code management systems) use a LOT of Zope 3. Do I grep the source for 'from zope ...' and 'import zope.*' and list all unique
Re: [Zope3-Users] buildout and egg depedencies
On Dec 11, 2007, at 1:37 PM, Christophe Combelles wrote: Jim Fulton a écrit : (...) issues, but again I find myself fighting against my natural instincts. Where's some end user documentation? http://pypi.python.org/pypi/zc.buildout http://us.pycon.org/common/talkdata/PyCon2007/116/buildout.pdf I've started learning buildout 3 weeks ago and I've found this document (the latter) extremely useful, because it is not a low- level reference but rather a 3-feets view with opinions, some examples, some ideas, and it explains the 'why' and the 'when'. It has helped me understanding buildout quickly, to the point of using it for production on small projects. And I'm now saving lots of time installing preview and final versions. Temporarily switching final eggs to develop-eggs to quickly fix some bug is another interesting idea suggested by this document and it will eventually help improving zope because it makes it so easy to test and commit even a minor issue in a particular environment, without interfering with anything else. Btw, I've started a translation of this tutorial, and I was suddenly wondering what was its license? (I didn't find a mention) I'm glad you found it useful. It is ZPL. Translate away. BTW, as you find bugs/typos, feel free to make corrections. The source is in the doc directory of the buildout project. Jim -- Jim Fulton Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
Jim Fulton a écrit : (...) issues, but again I find myself fighting against my natural instincts. Where's some end user documentation? http://pypi.python.org/pypi/zc.buildout http://us.pycon.org/common/talkdata/PyCon2007/116/buildout.pdf I've started learning buildout 3 weeks ago and I've found this document (the latter) extremely useful, because it is not a low-level reference but rather a 3-feets view with opinions, some examples, some ideas, and it explains the 'why' and the 'when'. It has helped me understanding buildout quickly, to the point of using it for production on small projects. And I'm now saving lots of time installing preview and final versions. Temporarily switching final eggs to develop-eggs to quickly fix some bug is another interesting idea suggested by this document and it will eventually help improving zope because it makes it so easy to test and commit even a minor issue in a particular environment, without interfering with anything else. Btw, I've started a translation of this tutorial, and I was suddenly wondering what was its license? (I didn't find a mention) Christophe The doc-test is difficult to read and speaks in generics, not about day-to-day problems. Everyone's a critic. ... That I'm still frustrated by these tools all this time later is disappointing. Yes. I'm disappointed that you're frustrated. Thanks for sharing. I find this post inspires me to want to take the day off to write better tutorials. ... I've gotten Buildout to work on some small components. It's great - check out the source, run buildout, wait, wait, wait, and then have a nice little self contained testing and development environment. But I've never been able to get a full Zope 3 "Application" up and running in that environment. Lots of people have. I expect that there are a number of examples. One example is the zc.z3monitor buildout, svn://svn.zope.org/repos/main/zc.z3monitor/trunk. This buildouts out a working ZEO server and a minimal Zope 3 application that uses it. Others might be able to suggest better examples, Jim -- Jim Fulton Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
Jeff Shell a écrit : > On Dec 6, 2007 8:08 AM, Benji York <[EMAIL PROTECTED]> wrote: >> Jim Fulton wrote: >>> None of the above. What is the harm of the dependencies? >> One of the options included in "none of the above" is to not use >> buildout at all. Forget that the project in question uses buildout (or >> setuptools) and integrate it into your project however you would have >> before those tools existed (svn:externals, make a checkout, whatever). > > That's such a disappointing answer. It's one that I've gotten a couple > of times when I've said "hey, I'm trying to move to a release based > system using distutils and setuptools and I'm floundering." And I have > made our own system. And it's I don't know. I'd like to spend more > time solving customer problems than figuring out how to install our > solutions to customer problems. We're in desperate need of reliable, > repeatable distributions. DESPERATE. > > How did it come to be that the Python tools are so bad at this? > Setuptools is horrible when it comes to doing local > (instance-home-ish) installations, requiring virtualenv or whatever. > And I've had little success getting those to work. Maybe they just > break my way of thinking about how Python does and should work. > Whatever. Buildout looks like it tries to address many of those > issues, but again I find myself fighting against my natural instincts. > Where's some end user documentation? The doc-test is difficult to read > and speaks in generics, not about day-to-day problems. The Recipes are > even worse, leaving one to clamor through the web to get back to the > cheeseshop page and then face the same difficult to read doc-test kind > of 'help'. Which I wouldn't mind reading, if I could easily read that > help locally, like a man page or using Python's 'help()' system. > ``buildout help zc.recipe.egg``, ``buildout help zc.recipe.cmmi``, > whatever. > > That I'm still frustrated by these tools all this time later is > disappointing. And yes, it's easier to write your own. That's the > Python way. Don't understand [zope, pylons, cherrypy, quixote, > skunkweb]? Write your own web framework! Python does make it easy to > do that because it's such a fantastic language. But I think that > attitude, in turn, gives us worse tools, because everyone scratches > their own itch and moves on, leaving everything incomplete. It seems > almost like it's easier to write your own tool than to read whatever > cryptic documentation exists for another. > > I've gotten Buildout to work on some small components. It's great - > check out the source, run buildout, wait, wait, wait, and then have a > nice little self contained testing and development environment. But > I've never been able to get a full Zope 3 "Application" up and running > in that environment. > > There's just no time and the tools are just too hard to learn under > the circumstances my little company is operating in right now. > Buildout *seems* like it could fix some big problems that have been > hitting us hard in recent weeks. But I still can't wrap my head around > how. > For this specific problem, I came down to write a little buildout recipe (thanks to Jim lights), more info at http://www.jfroche.be/blogging/archive/2007/12/10/avoid-fetching-zope-3-libraries-in-buildout and http://pypi.python.org/pypi/affinitic.recipe.fakezope2eggs/0.1 Regards, Jeff ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
On Dec 11, 2007, at 10:21 AM, Jeff Shell wrote: On Dec 6, 2007 8:08 AM, Benji York <[EMAIL PROTECTED]> wrote: Jim Fulton wrote: None of the above. What is the harm of the dependencies? One of the options included in "none of the above" is to not use buildout at all. Forget that the project in question uses buildout (or setuptools) and integrate it into your project however you would have before those tools existed (svn:externals, make a checkout, whatever). That's such a disappointing answer. It's one that I've gotten a couple of times when I've said "hey, I'm trying to move to a release based system using distutils and setuptools and I'm floundering." And I have made our own system. And it's I don't know. I'd like to spend more time solving customer problems than figuring out how to install our solutions to customer problems. We're in desperate need of reliable, repeatable distributions. DESPERATE. I think you are misunderstanding the context of Benji's answer. In this case, someone is running into trouble because they are trying to mix egg-based package management with subversion-based package management. The Zope 2 tree uses subversion. It obtains many Zope 3 packages via subversion externals. The original poster is trying to use some other Zope 3 packages via eggs and is packages via egg dependencies that he already has via subversion. This is at best annoying and at worst error provoking. Using eggs with the current Zope 2 tree is an abuse of the setuptools- based packaging system. There are two ways to fix this. The one I suggested was to add egg-info directories to the Zope 2 tree to make it compatible with eggs. Benji's suggestion was to use svn to get additional zope 3 packages, rather than eggs. Both are reasonable solutions in that they avoid the clash between the separate packaging systems. How did it come to be that the Python tools are so bad at this? I dunno. I think we're making a lot of progress. This particular example doesn't make me question the Python tools at all. Setuptools is horrible when it comes to doing local (instance-home-ish) installations, requiring virtualenv or whatever. And I've had little success getting those to work. Maybe they just break my way of thinking about how Python does and should work. Whatever. Buildout looks like it tries to address many of those issues, but again I find myself fighting against my natural instincts. Where's some end user documentation? http://pypi.python.org/pypi/zc.buildout http://us.pycon.org/common/talkdata/PyCon2007/116/buildout.pdf The doc-test is difficult to read and speaks in generics, not about day-to-day problems. Everyone's a critic. ... That I'm still frustrated by these tools all this time later is disappointing. Yes. I'm disappointed that you're frustrated. Thanks for sharing. I find this post inspires me to want to take the day off to write better tutorials. ... I've gotten Buildout to work on some small components. It's great - check out the source, run buildout, wait, wait, wait, and then have a nice little self contained testing and development environment. But I've never been able to get a full Zope 3 "Application" up and running in that environment. Lots of people have. I expect that there are a number of examples. One example is the zc.z3monitor buildout, svn://svn.zope.org/repos/ main/zc.z3monitor/trunk. This buildouts out a working ZEO server and a minimal Zope 3 application that uses it. Others might be able to suggest better examples, Jim -- Jim Fulton Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
On Dec 6, 2007 8:08 AM, Benji York <[EMAIL PROTECTED]> wrote: > Jim Fulton wrote: > > None of the above. What is the harm of the dependencies? > > One of the options included in "none of the above" is to not use > buildout at all. Forget that the project in question uses buildout (or > setuptools) and integrate it into your project however you would have > before those tools existed (svn:externals, make a checkout, whatever). That's such a disappointing answer. It's one that I've gotten a couple of times when I've said "hey, I'm trying to move to a release based system using distutils and setuptools and I'm floundering." And I have made our own system. And it's I don't know. I'd like to spend more time solving customer problems than figuring out how to install our solutions to customer problems. We're in desperate need of reliable, repeatable distributions. DESPERATE. How did it come to be that the Python tools are so bad at this? Setuptools is horrible when it comes to doing local (instance-home-ish) installations, requiring virtualenv or whatever. And I've had little success getting those to work. Maybe they just break my way of thinking about how Python does and should work. Whatever. Buildout looks like it tries to address many of those issues, but again I find myself fighting against my natural instincts. Where's some end user documentation? The doc-test is difficult to read and speaks in generics, not about day-to-day problems. The Recipes are even worse, leaving one to clamor through the web to get back to the cheeseshop page and then face the same difficult to read doc-test kind of 'help'. Which I wouldn't mind reading, if I could easily read that help locally, like a man page or using Python's 'help()' system. ``buildout help zc.recipe.egg``, ``buildout help zc.recipe.cmmi``, whatever. That I'm still frustrated by these tools all this time later is disappointing. And yes, it's easier to write your own. That's the Python way. Don't understand [zope, pylons, cherrypy, quixote, skunkweb]? Write your own web framework! Python does make it easy to do that because it's such a fantastic language. But I think that attitude, in turn, gives us worse tools, because everyone scratches their own itch and moves on, leaving everything incomplete. It seems almost like it's easier to write your own tool than to read whatever cryptic documentation exists for another. I've gotten Buildout to work on some small components. It's great - check out the source, run buildout, wait, wait, wait, and then have a nice little self contained testing and development environment. But I've never been able to get a full Zope 3 "Application" up and running in that environment. There's just no time and the tools are just too hard to learn under the circumstances my little company is operating in right now. Buildout *seems* like it could fix some big problems that have been hitting us hard in recent weeks. But I still can't wrap my head around how. -- Jeff Shell ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
Jim Fulton wrote: None of the above. What is the harm of the dependencies? One of the options included in "none of the above" is to not use buildout at all. Forget that the project in question uses buildout (or setuptools) and integrate it into your project however you would have before those tools existed (svn:externals, make a checkout, whatever). -- Benji York Senior Software Engineer Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
On Dec 6, 2007, at 9:06 AM, Jean-François Roche wrote: Jim Fulton a écrit : On Dec 6, 2007, at 8:51 AM, Jean-François Roche wrote: Hello, I am working with zope 2 and an happy user of buildout. But ... as soon as I am trying to use package such as z3c.sqlalchemy or z3c.menu I get "polluted" by zope 3 depedencies which of course brake my buildout environment. So question: Does someone know a way to tell to zc.buildout that he shouldnt install the depedencies of an egg which is defined in the "egg =" of the buildout section inside the buildout.cfg ? There is no way to do that. Doing so doesn't sound like a good idea to me. Ok :( , what should i do ? * Recompile my own eggs without depedencies ? * run my instance with a Makefile with a script which removes unwanted depedencies ? * using/create another recipe to remove selected depedencies ? * try to convince you that it is a good idea ? None of the above. What is the harm of the dependencies? If it is because Zope 2 includes many of the same packages outside of eggs, then this is a Zope 2 problem, not a buildout problem. This particular issue has been discussed a number of times. The solution, short of eggifying Zope 2, is to add egg-info directories to Zope 2 so that setuptools can see that the dependencies are already satisfied. If it's because extra dependencies annoy you, I'm unmotivated. Jim -- Jim Fulton Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
Jim Fulton a écrit : > > On Dec 6, 2007, at 8:51 AM, Jean-François Roche wrote: > >> Hello, >> >> I am working with zope 2 and an happy user of buildout. >> >> But ... as soon as I am trying to use package such as z3c.sqlalchemy >> or z3c.menu I get "polluted" by zope 3 depedencies which of course brake >> my buildout environment. >> >> So question: Does someone know a way to tell to zc.buildout that he >> shouldnt install the depedencies of an egg which is defined in the "egg >> =" of the buildout section inside the buildout.cfg ? > > > There is no way to do that. Doing so doesn't sound like a good idea to me. Ok :( , what should i do ? * Recompile my own eggs without depedencies ? * run my instance with a Makefile with a script which removes unwanted depedencies ? * using/create another recipe to remove selected depedencies ? * try to convince you that it is a good idea ? Thanks for your reply, Jeff ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] buildout and egg depedencies
On Dec 6, 2007, at 8:51 AM, Jean-François Roche wrote: Hello, I am working with zope 2 and an happy user of buildout. But ... as soon as I am trying to use package such as z3c.sqlalchemy or z3c.menu I get "polluted" by zope 3 depedencies which of course brake my buildout environment. So question: Does someone know a way to tell to zc.buildout that he shouldnt install the depedencies of an egg which is defined in the "egg =" of the buildout section inside the buildout.cfg ? There is no way to do that. Doing so doesn't sound like a good idea to me. Jim -- Jim Fulton Zope Corporation ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
