Re: [Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
On Wednesday 07 November 2007, Jeff Shell wrote: So now I see all that's going on with Zope 3 and I still don't understand a thing about how to move forward. We have about twelve active customers deployed on Zope 3, on top of one or two large frameworks that we've built on top if Z3. We still use the basic 'instance home' layout. We don't use eggs - we just check things out directly into the instance home. Even stuff from the Zope community, because it's easier to do that than to even get workingenv or any of these other things running. I'm still lost about what to do. Grok is of no use to us at the moment. Many of these apps have been up and running for longer than Grok's been alive. Is there any kind of migration documentation out there? Or have these new ways of doing things been in use for so long that most people have just rolled along with them? I just don't have the time. I scarcely have time to keep up with the lists (which is obvious since I haven't read a thing for nearly three months). Hi Jeff, last time I promised I would be more responsive to your mails, so here you go. :-) First of all, let me say that I have no system in production yet, which uses eggs. I think only Lovely Systems and Zope Corp. do; and maybe some smaller applications. So I would not worry about not being up-to-date. In fact, the last stable release is still 3.3.1, which tells the story of the big Zope 3 tree. That said, have you seen my Zope 3.4.0b2 announcement _[1]? It was the first public announcement making people aware that eggs are really happening. I purposefully did not say that Zope 3.4 will be the last tarball release, because (a) I think we cannot abandon people that quickly and (b) it is now really easy to create a new tarball release thanks to a couple of scripts I wrote. In the Zope 3.4.0b2 announcement I also sketched out two ways projects can be built using eggs, buildout and recipes. That said, writing a comprehensive migration document is still one of the tasks that need to be done. I was really hoping that the community would step up in writing one. I think that Darryl's response to your mail is a great start actually. It could easily be used as a starting point. .. [1] http://mail.zope.org/pipermail/zope3-users/2007-November/007106.html Regards, Stephan -- Stephan Richter CBU Physics Chemistry (B.S.) / Tufts Physics (Ph.D. student) Web2k - Web Software Design, Development and Training ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
Oops. Forgot to send this to the list as well as Stephan. Goddamn Gmail. On Nov 8, 2007 8:28 AM, Stephan Richter [EMAIL PROTECTED] wrote: Hi Jeff, last time I promised I would be more responsive to your mails, so here you go. :-) Thanks! First of all, let me say that I have no system in production yet, which uses eggs. I think only Lovely Systems and Zope Corp. do; and maybe some smaller applications. So I would not worry about not being up-to-date. In fact, the last stable release is still 3.3.1, which tells the story of the big Zope 3 tree. That said, have you seen my Zope 3.4.0b2 announcement _[1]? It was the first public announcement making people aware that eggs are really happening. I purposefully did not say that Zope 3.4 will be the last tarball release, because (a) I think we cannot abandon people that quickly and (b) it is now really easy to create a new tarball release thanks to a couple of scripts I wrote. This is where I start to get confused. I understand, and am glad for, the continuation of tarball releases for this round. My main question is: how does one mix and match? We have a lot of external software requirements and I'd rather get some of those as eggs than keep doing individual subversion checkouts. My big concern, which is something that affects all packaging systems, is how to tell packaged stuff that hand-build stuff meets the requirements. If I want to use an egg in a particular environment that has dependencies on: - zope.component 3.4.0 (or anything built into the zope 3.4 tarball) - sqlalchemy 0.4.0 - simplejson Would it know that zope.component was already offered via the whole Zope 3.4 tarball? It seems like you do things the classic way, or the egg/buildout way, and nothing in between. This makes things like z3c.form intimidating to me because I don't know if/how it will work in my vanilla zope environment (and with such a long time span between 3.3.1 and the eventual 3.4, I doubt it or its dependencies could work in 3.3.1). I may be wrong about all of that, but that's how I currently perceive things. Things just look so different between the two. -- Jeff Shell ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
Am Freitag, 9. November 2007 01:31 schrieb Jeff Shell: Oops. Forgot to send this to the list as well as Stephan. Goddamn Gmail. On Nov 8, 2007 8:28 AM, Stephan Richter [EMAIL PROTECTED] wrote: Hi Jeff, - zope.component 3.4.0 (or anything built into the zope 3.4 tarball) - sqlalchemy 0.4.0 - simplejson Btw., how do you integrate Zope3 with SQLAlchemy 0.4? As far as I know zalchemy is not SQLAlchemy-0.4 ready yet? Best Regards, Hermann -- [EMAIL PROTECTED] GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
On Wed, 2007-11-07 at 21:22 -0700, Jeff Shell wrote: On Nov 7, 2007 7:53 PM, Darryl Cousins [EMAIL PROTECTED] wrote: Hi Jeff, I hope that this might at least give you the optimism you appear to be lacking. Thank you. That does give me a much more clear starting point. I was having a hard time seeing all of the recipes come together. Cheers Jeff. A note about pinning down the versions, I'm following this: http://mail.zope.org/pipermail/zope3-users/2007-November/007106.html now rather than using the grok versions. I didn't have any trouble at all resolving deprecation and import errors when upgrading from Zope3.3.0 tarball to the 3.4 eggs as described in the above mail. There were hardly any compared with the upgrade from 3.2.0 to 3.3.0. Application is up and running on eggs. Brrroook. Regards, Darryl As for optimism... We'll see. :) But this was the step I could never make work (getting a buildout-managed instance up and running) before running out of time. Thanks again. ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
On Nov 7, 2007 7:53 PM, Darryl Cousins [EMAIL PROTECTED] wrote: Hi Jeff, I hope that this might at least give you the optimism you appear to be lacking. Thank you. That does give me a much more clear starting point. I was having a hard time seeing all of the recipes come together. As for optimism... We'll see. :) But this was the step I could never make work (getting a buildout-managed instance up and running) before running out of time. Thanks again. -- Jeff Shell ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
Hi Jeff,
I hope that this might at least give you the optimism you appear to be
lacking.
I've been migrating a complex project which was running on Zope-3.2
tarball install.
My first step was to get it all into a buildout. buildout.cfg looked
something like this [1]. I've missed out lots here (postgresql, lxml,
sqlos, sqlobject, psycopg2, pyro, zif.jsonserver amongst others). Note
that I have included a local install of python in parts to keep things
clean and tidy.
Second step was to upgrade to Zope-3.3 - this just meant changing the
url in the zope3 section. An hour or two of fixing deprecation warnings
(really only deprecation stuff, nothing major).
Step Three, where I'm at now, is to change the buildout to use Zope-3.4
eggs, and Paste. Zopeproject buildout.cfg and other files could provide
you a starting point. I'm using
http://grok.zope.org/releaseinfo/grok-0.10.2.cfg to pin down versions.
See http://grok.zope.org/releaseinfo/readme.html for info on using this
file.
I haven't got the server running yet, but then I've only just finished
getting buildout.cfg right, and all parts built.
In all there has been several hours work so far, but that includes
sorting out the additional dependencies (postgresql etc). Before the
buildout the whole application was a nightmare to install. Now its
looking sooo much better.
I would skip step two if I did this again, although I imagine there will
be a lot more import and deprecation errors to correct.
Hope that helps.
Regards,
Darryl
[1] buildout.cfg
[buildout]
index = http://download.zope.org/ppix
develop = .
parts = zope3
server
instance
database
test
projectpackage # the application code
find-links = http://download.zope.org/distribution/
download-cache = cache
eggs-directory = eggs # keeping this install fully separate from others
develop-eggs-directory = develop-eggs
[zope3]
recipe = zc.recipe.cmmi
url = http://www.zope.org/Products/Zope3/3.2.0/Zope-3.2.0.tgz
extra_options = --with-python=${python:location}/bin/python
--force
[server]
recipe = zc.recipe.egg
eggs = ${projectpackage:eggs}
[instance]
recipe = zc.zope3recipes:instance
application = projectpackage
zope.conf = ${database:zconfig}
[database]
recipe = zc.recipe.filestorage
[test]
recipe = zc.recipe.testrunner
eggs = ${projectpackage:eggs}
extra-paths = ${zope3:location}/lib/python
working-directory = ${buildout:directory}/test
defaults = ['--tests-pattern', '^f?tests$',
'-v'
]
[projectpackage]
recipe = zc.zope3recipes:app
eggs = ${interpreter:eggs}
site.zcml = include package=projectpacakge / # all zcml defined here
[python]
recipe = zc.recipe.cmmi
url = http://www.python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2
executable = ${buildout:parts-directory}/python/bin/python
extra_options = --with-threads
--with-readline
--enable-unicode=ucs2
[interpreter]
recipe = zc.recipe.egg
eggs = setuptools
# list-of-eggs: lxml, sqlos etc, etc
interpreter = python
extra-paths = ${buildout:parts-directory}/zope3/lib/python
On Wed, 2007-11-07 at 17:24 -0700, Jeff Shell wrote:
The last time I was active on this list was back in mid-august. I was
fighting an uphill battle to try to figure out how to migrate our
software to 'Eggs'. I ultimately had to abandon the process as there
was just too much real work to be done, and I was getting conflicting
answers about how to move forward. Actually, I couldn't get much help
from the community at all. I apologize for the crankiness of this
email. The last time I tried to get help on this topic was an
excercise in sheer frustration. Since then, the gulf I've worried
about seems to have gotten even bigger, and I'm feeling very stressed
right now.
So now I see all that's going on with Zope 3 and I still don't
understand a thing about how to move forward. We have about twelve
active customers deployed on Zope 3, on top of one or two large
frameworks that we've built on top if Z3. We still use the basic
'instance home' layout. We don't use eggs - we just check things out
directly into the instance home. Even stuff from the Zope community,
because it's easier to do that than to even get workingenv or any of
these other things running.
I'm still lost about what to do. Grok is of no use to us at the
moment. Many of these apps have been up and running for longer than
Grok's been alive. Is there any kind of migration documentation out
there? Or have these new ways of doing things been in use for so long
that most people have just rolled along with them? I just don't have
the time. I scarcely have time to keep up with the lists (which is
obvious since I haven't read a thing for nearly three months).
I'm going to resume my conversation here, responding to myself.
August 16, 2007, Jeff Shell wrote:
Jeff Shell wrote:
But now I'd like to be able to install that into classic zope 3.3
instance homes, trying to be as
[Zope3-Users] Eggs and classic instance homes, again... (was Re: Eggs, workingenv.py, and 'classic' instance homes)
The last time I was active on this list was back in mid-august. I was fighting an uphill battle to try to figure out how to migrate our software to 'Eggs'. I ultimately had to abandon the process as there was just too much real work to be done, and I was getting conflicting answers about how to move forward. Actually, I couldn't get much help from the community at all. I apologize for the crankiness of this email. The last time I tried to get help on this topic was an excercise in sheer frustration. Since then, the gulf I've worried about seems to have gotten even bigger, and I'm feeling very stressed right now. So now I see all that's going on with Zope 3 and I still don't understand a thing about how to move forward. We have about twelve active customers deployed on Zope 3, on top of one or two large frameworks that we've built on top if Z3. We still use the basic 'instance home' layout. We don't use eggs - we just check things out directly into the instance home. Even stuff from the Zope community, because it's easier to do that than to even get workingenv or any of these other things running. I'm still lost about what to do. Grok is of no use to us at the moment. Many of these apps have been up and running for longer than Grok's been alive. Is there any kind of migration documentation out there? Or have these new ways of doing things been in use for so long that most people have just rolled along with them? I just don't have the time. I scarcely have time to keep up with the lists (which is obvious since I haven't read a thing for nearly three months). I'm going to resume my conversation here, responding to myself. August 16, 2007, Jeff Shell wrote: Jeff Shell wrote: But now I'd like to be able to install that into classic zope 3.3 instance homes, trying to be as unobtrusive as possible. How have you installed other packages before? I suspect by just dumping their source in INSTANCE/lib/python. Any reason not to continue to do that? Because I feel like we're falling further and further behind, or at least trapped in our own little world. Yep, this is becoming even more true. We're now even further behind. But yes, this is how we did it. I ended up writing most of rocketbuild/rockout (our Rake-ish tool) because of this. There are a lot of external tools out there that we still don't use because we don't have the time to deal with the headache (made minor by `rockout`, but still present) of dumping sources into lib/python. For internal stuff, it's not so bad. I want to get away from this (using source control as distribution mechanism). Eggs and buildout are supposed to help here, but ultimately it was easier to roll our own tool as we just could not figure out how to apply them to our configurations. We're about to do a massive server upgrade, which would be a good time to sneak in new deployment practices. The longer we avoid whatever is going on with setuptools and buildout, the harder it will be to migrate. It's hard enough already. Yep. Feels even harder now, but probably because they cause me so much stress just by looking at them. That seems backward. If you want to install an package into a classic environment, it doesn't seem that surprising you'll also have to do it the classic way... It doesn't seem entirely backward. I mean, it does. But at the same time... I just don't know how it's all supposed to work. But if we keep doing things in the 'classic' way without trying to step forward, then I fear we'll stay this way forever because it's already scary enough peeking over the hill at how buildout and all of that stuff works and how different it is from how we've done things. I don't want the gulf to get bigger, because I think that with our plans for growth and for a more distributed deployment server setup, 'buildout' will help tremendously. I'm just trying for baby steps right now, because it feels like we've still got a long way to go to get to anything better. I saw mention of 'workingenv' in your book, so I assumed it was or is possible to easily have instance homes with locally managed eggs. Shouldn't it be? Still have a long way to go. Still have no answers for what to do or how to start migrating. Some of our problem is caused by the stupidity of CVS, which will go away when we migrate to Git. Is there going to be a Zope 3.4, for real this time? Will it offer the clues I'm looking for? 'zopeproject' says 'You can start a new Zope-based web application from scratch with just two commands'. But what about those of us who have started many many Zope based web projects? Thanks, Jeff Shell ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
