2010/10/6 Jim Fulton <[email protected]>: > 2010/10/6 Łukasz Nowak <[email protected]>: >> Hello, >> >> I am playing with zc.buildout for some time. I experienced issues with >> system provided pythons (even virutalenved), so I switched to build my >> own python. Of course as I am lazy, I am using buildout to configure >> and compile various python versions on my machine. >> >> Quite often I need to provide "self contained" buildout profile, which >> would be able to run using python compiled by itself. >> >> So my example profile looks like this: >> >> [buildout] >> parts = >> rebootstrap2.4 >> realrun >> >> find-links = http://www.nexedi.org/static/packages/source/ >> >> [python2.4] >> prefix = ${buildout:parts-directory}/${:_buildout_section_name_} >> executable = ${:prefix}/bin/python2.4 >> >> # this is just wrapper for hexagonit.recipe.cmmi which makes it not >> returning python part >> recipe = erp5.recipe.cmmisafe >> url = >> http://python.org/ftp/python/2.4.6/Python-2.4.6.tgz >> configure-options = >> --enable-unicode=ucs4 >> --with-threads >> --with-readline >> --with-dbm >> --with-zlib >> --with-ssl >> --with-bz2 >> >> [rebootstrap2.4] >> recipe = zc.recipe.egg >> eggs = zc.buildout >> python = python2.4 >> >> [realrun] >> recipe = plone.recipe.command >> command = >> echo Running with python ${buildout:executable} >> update-command = ${:command} >> >> So first step for me is to bootstrap builodut: >> >> curl -s >> http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py >> | python -S - >> >> So now my bin/buildout is referring to "system python": >> head -n1 bin/buildout >> #!/home/luke/bin/python -S >> >> Then I need to rebootstrap it: >> >> bin/buildout install python2.4 rebootstrap2.4 >> >> python2.4* parts are installing and compiling python, rebootstrap2.4 >> part is reinstalling bin/buildout using this python. >> >> Cool, my bin/buildout is using python it compiled: >> >> head -n1 bin/buildout >> #!/home/luke/pybuildout/parts/python2.4/bin/python2.4 >> >> So I can run again, this time usually. python will be compiled, as >> buildout signature changed, but this is not an issue (ccache + distcc >> makes it really fast). Buildout is using python he provided, >> chicken-and-egg issue solved by hand: >> >> bin/buildout >> ... >> >> realrun: Running ' >> echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4' >> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4 >> Unused options for realrun: 'update-command'. >> >> Re running builodut does not affect python anymore: >> >> bin/buildout >> Updating python2.4-dbm-patch. >> Updating python2.4. >> Updating rebootstrap2.4. >> Updating realrun. >> realrun: Running >> echo Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4 >> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4 > > Sounds cool! > >> And so far I am happy, I am using correct python version. >> >> What I'd like to develop is to make those steps automatically, so that >> after running: >> >> curl -s >> http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py >> | python -S - >> bin/buildout >> >> Such steps would happen automagically: >> >> 1 normal bootstrap with system python >> 2 compile python as needed >> 3 install new bin/buildout >> 4 re-run bin/buildout >> 5 compile python as signature change >> 6 install new bin/buildout (which is the same) >> 7 re-run bin/buildout >> 8 do not compile python, as signature had not changed, continue... >> >> Theoretically steps 5,6 and 7 could be avoided, but lets say I am >> purist -- I do not trust that python compiled by buildout which is >> running python I do not trust is bad :) >> >> Easy question: Is there ability for buildout to do smart rebootstrap >> in one run, with selecting part which contains python? > > No. It might be worthwhile to build in support for this use case. > >> If no I am ready to develop such thing. > > Cool. > >> So I read a bit about >> extensions - I can hook before and after buildout is run. Extensions >> have access to buildout object, I think they can play with buildout >> externals a lot, including re-running buildout (I saw in output that >> buildout re-runs itself after upgrading himself). >> >> Are extension the way to go with such thing? > > Extensions are a valuable experiment, but they aren't > really fleshed out. > > The mechanism is very simple, which is good. To make it more > valuable would require providing more hook points in buildout itself. > > As you point out though, you could convievably use the unload > extension to cause the buildout to be rerun. > >> If needed I am even ready to play a bit more with zc.buildout >> internals code to extend if required, but according to my >> understanding how buildout works there are enough places I can "hook" >> to. > > You may be right. It sounds like a good next step. > > You could also propose additional apis that might help your > extension. > >> Any other comments? > > Only that your use case is not that uncommon.
/me raises hand, that's _exactly_ what I'm futzing with at the moment, as I try to automate building a Bacula server from tarballs, using a buildout. Currently I have a companion shell script which builds the private Python. Feels dirty. I'm doing it on a vm, reverting between attempts. I would be tickled to test any work along these lines. Thanks, Kent Perhaps there > should be a built in option to have buildout build it's own Python. > > One thought is that building Python is expensive enough that you'd like > to share the work accross buildouts. *Maybe* it would be interesting > to point buildout at a separate Python buildout. It would look at the > Python buildout and build *it* if necessary and then use the resulting > Python. > > Jim > > -- > Jim Fulton > _______________________________________________ > Distutils-SIG maillist - [email protected] > http://mail.python.org/mailman/listinfo/distutils-sig > _______________________________________________ Distutils-SIG maillist - [email protected] http://mail.python.org/mailman/listinfo/distutils-sig
