Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Benji York

Darryl Cousins wrote:

I'm trying to created an isolated environment using buildout. I had
understood that defining a custom build python would ensure that all
eggs are installed and compiled with the custom python.


An alternative to building Python with buildout is to build a single 
clean Python and use it for all their buildouts.


Some advantages of this approach are that your buildouts take less time 
to build from scratch (no building Python every time), and are a bit 
smaller (by about 115MB on my system).  I keep lots of buildouts around 
(a few dozen), so those advantages add up.


Of course it also lets you stay away from the generally dirty system 
Python (just as the Python-per-buildout strategy does as well).

--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread David Pratt
Hi Benji. This is exactly what I have been doing up till now and has 
been working well for quick work on a local development machine. My 
current thinking though is to take control of as much of the software as 
possible so that development == deployment on my local machine to 
mitigate the risk of breaking things even if it means more disk. I am 
doing this in conjunction with stripping the deployment server to it's 
barest bones and bringing as much of the software into the buildout as 
possible.


I really would like to see a two stage buildout that does the python 
construction with a python.cfg and then the main buildout with 
buildout.cfg file as part of the standard fare. I'm trying a few things 
today to see if a simple event class and callback can be used to create 
the python first and have the callback's handler run the main buildout 
as an experiment.


Regards,
David

Benji York wrote:

Darryl Cousins wrote:

I'm trying to created an isolated environment using buildout. I had
understood that defining a custom build python would ensure that all
eggs are installed and compiled with the custom python.


An alternative to building Python with buildout is to build a single 
clean Python and use it for all their buildouts.


Some advantages of this approach are that your buildouts take less time 
to build from scratch (no building Python every time), and are a bit 
smaller (by about 115MB on my system).  I keep lots of buildouts around 
(a few dozen), so those advantages add up.


Of course it also lets you stay away from the generally dirty system 
Python (just as the Python-per-buildout strategy does as well).

___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Benji York

Christian Theune wrote:

And you can mix-and match: have buildout profiles that use your personal
`good` Python for development and production profiles that build it
themselves. OTOH this is somewhat controversial regarding test what you
fly, fly what you test ... ;)


As long as you also run your tests in production, it seems fine to me.
--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Christian Theune

Am Freitag, den 21.12.2007, 08:30 -0500 schrieb Benji York:
 Darryl Cousins wrote:
  I'm trying to created an isolated environment using buildout. I had
  understood that defining a custom build python would ensure that all
  eggs are installed and compiled with the custom python.
 
 An alternative to building Python with buildout is to build a single 
 clean Python and use it for all their buildouts.
 
 Some advantages of this approach are that your buildouts take less time 
 to build from scratch (no building Python every time), and are a bit 
 smaller (by about 115MB on my system).  I keep lots of buildouts around 
 (a few dozen), so those advantages add up.
 
 Of course it also lets you stay away from the generally dirty system 
 Python (just as the Python-per-buildout strategy does as well).

And you can mix-and match: have buildout profiles that use your personal
`good` Python for development and production profiles that build it
themselves. OTOH this is somewhat controversial regarding test what you
fly, fly what you test ... ;)

Christian

-- 
gocept gmbh  co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development


signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Jim Fulton


On Dec 21, 2007, at 10:18 AM, David Pratt wrote:

Hi Benji. This is exactly what I have been doing up till now and has  
been working well for quick work on a local development machine. My  
current thinking though is to take control of as much of the  
software as possible so that development == deployment on my local  
machine to mitigate the risk of breaking things even if it means  
more disk. I am doing this in conjunction with stripping the  
deployment server to it's barest bones and bringing as much of the  
software into the buildout as possible.


I really would like to see a two stage buildout that does the python  
construction with a python.cfg and then the main buildout with  
buildout.cfg file as part of the standard fare. I'm trying a few  
things today to see if a simple event class and callback can be used  
to create the python first and have the callback's handler run the  
main buildout as an experiment.



IMO, it is much more practical to create a clean system package for  
Python.  By a clean system package, I mean one that is equivalent to  
buildoing from source and has nothing added.  This is equivalent to  
what you would build yourself but makes deployment much easier because  
you don't have to rebuild Python every time you make a new release.   
You can use this in development as well.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Zvezdan Petkovic

On Dec 21, 2007, at 9:28 AM, Christian Theune wrote:



Am Freitag, den 21.12.2007, 09:15 -0500 schrieb Benji York:

Christian Theune wrote:
And you can mix-and match: have buildout profiles that use your  
personal `good` Python for development and production profiles  
that build it themselves. OTOH this is somewhat controversial  
regarding test what you fly, fly what you test ... ;)


As long as you also run your tests in production, it seems fine to  
me.


True. OTOH you really don't want your development environment to be  
far off your production environment. Joel (Spolsky) wrote a good  
article about leaking abstractions. It can be rather nasty to find a  
failing test when wanting to put something into production. (Of  
course this is way better than going in production and not seeing  
failing tests.)



If that clean python is built as a package (e.g., an rpm) and the  
product is also built as a package (see zc.sourcerelease) which  
declares to depend on the clean python package, the development and  
production environments will be the same.  It'll fly what it tested.

:-)

--
Zvezdan Petkovic [EMAIL PROTECTED]

___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Christian Theune

Am Freitag, den 21.12.2007, 09:15 -0500 schrieb Benji York:
 Christian Theune wrote:
  And you can mix-and match: have buildout profiles that use your personal
  `good` Python for development and production profiles that build it
  themselves. OTOH this is somewhat controversial regarding test what you
  fly, fly what you test ... ;)
 
 As long as you also run your tests in production, it seems fine to me.

True. OTOH you really don't want your development environment to be far
off your production environment. Joel (Spolsky) wrote a good article
about leaking abstractions. It can be rather nasty to find a failing
test when wanting to put something into production. (Of course this is
way better than going in production and not seeing failing tests.)

Christian

-- 
gocept gmbh  co. kg - forsterstrasse 29 - 06112 halle (saale) - germany
www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 -
fax +49 345 122 9889 1 - zope and plone consulting and development


signature.asc
Description: Dies ist ein digital signierter Nachrichtenteil
___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Benji York

David Pratt wrote:
Hi Benji. This is exactly what I have been doing up till now and has 
been working well for quick work on a local development machine. My 
current thinking though is to take control of as much of the software as 
possible so that development == deployment on my local machine to 
mitigate the risk of breaking things even if it means more disk. I am 
doing this in conjunction with stripping the deployment server to it's 
barest bones and bringing as much of the software into the buildout as 
possible.


I really would like to see a two stage buildout that does the python 
construction with a python.cfg and then the main buildout with 
buildout.cfg file as part of the standard fare. I'm trying a few things 
today to see if a simple event class and callback can be used to create 
the python first and have the callback's handler run the main buildout 
as an experiment.


Where you draw the boundary line between the environment and the 
application has a big impact on how you make these types of decisions.


You want to find the optimum place to draw that line so you end up 
getting the most benefit from the least amount of work (it's kind of 
analogous to the max-flow min-cut theorem from graph theory).


Let me use an example to illustrate.  Say you decided to build Python 
with your buildout.  After all, your app uses Python, so to have good 
reproducibility you want to make sure Python is perfectly clean and 
built repeatably so you don't get any surprises.  Makes sense.


Your app is enjoying increased success and one day you need to add some 
new servers to your cluster.  You buy a few new machines, put your OS on 
them, and build your app.  You then run your tests and they fail.


Darn.

You investigate and find that your Python was built without support for 
zip compression because the zlib development libraries aren't included 
in your base OS install.  Now you have a decision to make, do you add 
zlib to your buildout, or do you add the zlib development package to 
your OS?


If Python is part of the environment you add the zlib development 
package to your OS.  Of course that means you need a good way of 
controlling what is in the OS (environment).  Conveniently, there are 
ways to do that (RPM, APT, ports, etc.).


At this point if you draw the app/environment boundary to include Python 
in your app, you should add zlib to your buildout, right?  What happens 
if you hit a bug in GCC when compiling Python?  Do you include GCC in 
your buildout to make sure you get the right version?  It gets worse the 
deeper you go. :)

--
Benji York
Senior Software Engineer
Zope Corporation
___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Zvezdan Petkovic

On Dec 21, 2007, at 10:18 AM, David Pratt wrote:
I really would like to see a two stage buildout that does the python  
construction with a python.cfg and then the main buildout with  
buildout.cfg file as part of the standard fare. I'm trying a few  
things today to see if a simple event class and callback can be used  
to create the python first and have the callback's handler run the  
main buildout as an experiment.



In other words you want to build two products with one buildout.
Python is a language your product depends on.  It does not need to be  
a part of your product as the C language and libc are not part of the  
Python.


As I mentioned in the previous post in this thread one can get the  
same development and production environments by properly packing clean  
python and their product(s).  The added advantage is that N products  
can use the same clean python instead of having it deployed N times as  
a part of their buildout.


--
Zvezdan Petkovic [EMAIL PROTECTED]

___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Jim Fulton


On Dec 20, 2007, at 8:58 PM, David Pratt wrote:

Right. Thanks for this, there are couple of solutions here and shell  
script is a reasonable solution but these methods seems counter to  
the simplicity of buildout. I had also seen a rebootstrap package in  
svn. I wonder if something similar could not be accomplished this  
way using python with a means of producing a callback. I really  
don't want anything too messy to accomplish this. Many thanks.



I'll note that much of the pain here (or at least in the distutils-sig  
thread)  arises from the fact that the system
Python and the custom Python have the same version but are built in  
incompatible ways. :(  Unfortunately, binary egg names don't capture  
enough information about the ways the eggs were built.


I'm pondering an option to provide different egg directories for  
different Python interpreters.  So, maybe something like:


  [buildout]
  parts = python24 ...

  [python24]
  recipe = zc.recipe.cmmi
  url = ...
  executable = ${buildout:parts-directory}/bin/python
  eggs-directory = ${buildout:directory}/eggs
  develop-eggs-directory = ${buildout:parts-directory}/develop-eggs

The idea is that egg-aware recipes would use the egg locations  
specified in the Python section.  I think this would provide a fairly  
clean way to build and use a custom Python.


Of course, it would be nice to have a recipe to build python that set  
the various variables.


I can see the benefit of building a custom Python as part of the  
buildout when you don't want users of a buildout to build a clean  
Python of their own.  However, if you can require your users to do  
this, for example, if you are your user, then life is much simpler  
when a clean Python is managed separately.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread Jim Fulton


On Dec 20, 2007, at 8:03 PM, David Pratt wrote:

Hey Darryl. Our minds were in a similar place today. I finally had  
decided to move away from my system python starting on my mac and  
also having a spot of trouble. I put something simple together below  
to illustrate what happened in my instance.


Here the eggs built with the python would not install. I initially  
though it was the lxml I was compiling, but it was just happened to  
be the first part requiring the custom-python. Eggs were made, and I  
could see the temp folder also, but when was finalizing the install,  
it threw the error and without much verbosity, the tempfolder it was  
installing into disappeared.


Note, I did not include the custom-python as a part in the buildout.  
I tend to like the python24:location type of notation since it is  
fairly explicit when you have a number of pieces of software you are  
building.


Regards
David

[buildout]
develop = .
parts = python24
   interpreter
index = http://download.zope.org/ppix
log-level = DEBUG

[python24]
recipe = zc.recipe:cmmi
url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz

[custom-python]
executable = ${python24:location}/bin/python

[interpreter]
recipe = zc.recipe.egg
interpreter = interpreter
eggs = Cheetah==1.0
python = custom-python


Here are the highlights:


Having *just* highlights makes it difficult to follow what's going on.




Some verification it is using the right python...

Getting distribution for 'Cheetah==1.0'.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6c7
Running easy_install:
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/parts/python24/bin/ 
python -c from setuptools.command.easy_install import main;  
main() -mUNxd /Users/davidpratt/Desktop/Buildouts/osxdeploy2/ 
eggs/tmprQq-MH -v /Users/davidpratt/Desktop/Buildouts/download- 
cache/dist/Cheetah-1.0.tar.gz
path=/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/ 
setuptools-0.6c7-py2.4.egg


Processing Cheetah-1.0.tar.gz

Last few lines of output.. some time later.

changing mode of /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/ 
tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/ 
top_level.txt to 755
changing mode of /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/ 
tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/ 
cheetah to 755
changing mode of /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/ 
tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/ 
cheetah-compile to 755


Installed /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq- 
MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg


Because this distribution was installed --multi-version, before you  
can

import modules from this package in an application, you will need to
'import pkg_resources' and then use a 'require()' call similar to  
one of

these examples, in order to select the desired version:

   pkg_resources.require(Cheetah)  # latest installed version
   pkg_resources.require(Cheetah==1.0)  # this exact version
   pkg_resources.require(Cheetah=1.0)  # this version or higher


Note also that the installation directory must be on sys.path at  
runtime for
this to work.  (e.g. by being the application's script directory, by  
being on

PYTHONPATH, or by being added to sys.path by your code.)

While:
 Installing interpreter.
 Getting distribution for 'Cheetah==1.0'.
Error: Couldn't install: Cheetah 1.0
[mac-pg:~/Desktop/Buildouts/osxdeploy2] davidpra%


That's very odd.  It feels like something is missing in the output.

I tried reproducing this. I didn't have the contents of your  
directory, so I had to comment out develop = ..  You have a typo in  
the recipe specification for the python24 part. After fixing that, I  
ran the buildout above using the system Python on Mac OS X 10.5.1.   
Worked great. The interpreter script uses the custom Python.


The interpreter runs, but when I try to import cheeta, I get a bus  
error..  Unfortunately, there is a problem with using cmmi to install  
Python on Mac OS X 10.5.  There is a bug in Python's configure that  
causes Python to be built incorrectly. The generated pyconfig.h file  
currently requires some tweaking. :(


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread David Pratt
Hi Jim. I like this idea. I then makes it trivial to identify impacts of 
using different pythons on your application also since you can just 
change a single line in your buildout. For example, below, just change 
the ${python24:location}/bin/python to whatever python you want to 
evaluate another python including pointing to your system python if you 
wish.


Regards,
David

[python24]
recipe = zc.recipe:cmmi
url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz
... more stuff

[python25]
recipe = zc.recipe:cmmi
url = http://python.org/ftp/python/2.5.1/Python-2.5.1.tgz
... more stuff

[active-python]
executable = ${python24:location}/bin/python

[interpreter]
recipe = zc.recipe.egg
interpreter = interpreter
eggs = zc.buildout
   Cheetah==1.0
python = active-python


Jim Fulton wrote:


On Dec 20, 2007, at 8:58 PM, David Pratt wrote:

Right. Thanks for this, there are couple of solutions here and shell 
script is a reasonable solution but these methods seems counter to the 
simplicity of buildout. I had also seen a rebootstrap package in svn. 
I wonder if something similar could not be accomplished this way using 
python with a means of producing a callback. I really don't want 
anything too messy to accomplish this. Many thanks.



I'll note that much of the pain here (or at least in the distutils-sig 
thread)  arises from the fact that the system
Python and the custom Python have the same version but are built in 
incompatible ways. :(  Unfortunately, binary egg names don't capture 
enough information about the ways the eggs were built.


I'm pondering an option to provide different egg directories for 
different Python interpreters.  So, maybe something like:


  [buildout]
  parts = python24 ...

  [python24]
  recipe = zc.recipe.cmmi
  url = ...
  executable = ${buildout:parts-directory}/bin/python
  eggs-directory = ${buildout:directory}/eggs
  develop-eggs-directory = ${buildout:parts-directory}/develop-eggs

The idea is that egg-aware recipes would use the egg locations specified 
in the Python section.  I think this would provide a fairly clean way to 
build and use a custom Python.


Of course, it would be nice to have a recipe to build python that set 
the various variables.


I can see the benefit of building a custom Python as part of the 
buildout when you don't want users of a buildout to build a clean Python 
of their own.  However, if you can require your users to do this, for 
example, if you are your user, then life is much simpler when a clean 
Python is managed separately.


Jim

--
Jim Fulton
Zope Corporation



___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread David Pratt

Benji York wrote:

David Pratt wrote:
Hi Benji. This is exactly what I have been doing up till now and has 
been working well for quick work on a local development machine. My 
current thinking though is to take control of as much of the software 
as possible so that development == deployment on my local machine to 
mitigate the risk of breaking things even if it means more disk. I am 
doing this in conjunction with stripping the deployment server to it's 
barest bones and bringing as much of the software into the buildout as 
possible.


I really would like to see a two stage buildout that does the python 
construction with a python.cfg and then the main buildout with 
buildout.cfg file as part of the standard fare. I'm trying a few 
things today to see if a simple event class and callback can be used 
to create the python first and have the callback's handler run the 
main buildout as an experiment.


Where you draw the boundary line between the environment and the 
application has a big impact on how you make these types of decisions.


You want to find the optimum place to draw that line so you end up 
getting the most benefit from the least amount of work (it's kind of 
analogous to the max-flow min-cut theorem from graph theory).


Hi Benji. You are absolutely right about this. Also I have to say the 
amount of software going into things was sort of freaking me out over 
the last year or so, particulary when all the eggified packages were 
just rolling out. What I am coming to learn is that the set of software 
that I am using, while relatively large is also reasonably finite. And I 
mean the essential server software starts looking small in relation to 
two or three hundred eggs after a while :-)  Secondly, a buildout.cfg is 
pretty readable - I like knowing I don't have to guess about how 
something was built, its all in black and white with setting I no longer 
need to look up or remember. In this way packaging systems like ports 
and yum make you a little lax about a distribution and its settings.


I have a buildout recipe for gcc also, how ironic :-) Crazy thing was 
that pyLucene up until recently would only work on certain platforms 
with gcc-3.4.6, so I have had to go this route to construct a compiler 
that would allow pyLucene to be built also.


I get your point though, and ports and yum are nice and easy to work 
with. The way I am attempting to mitigate issues with buildouts and 
servers is specialization. The fact that a server in as much as possible 
is reliably the same is what I want for administration.


Regards,
David



Let me use an example to illustrate.  Say you decided to build Python 
with your buildout.  After all, your app uses Python, so to have good 
reproducibility you want to make sure Python is perfectly clean and 
built repeatably so you don't get any surprises.  Makes sense.


Your app is enjoying increased success and one day you need to add some 
new servers to your cluster.  You buy a few new machines, put your OS on 
them, and build your app.  You then run your tests and they fail.


Darn.

You investigate and find that your Python was built without support for 
zip compression because the zlib development libraries aren't included 
in your base OS install.  Now you have a decision to make, do you add 
zlib to your buildout, or do you add the zlib development package to 
your OS?


If Python is part of the environment you add the zlib development 
package to your OS.  Of course that means you need a good way of 
controlling what is in the OS (environment).  Conveniently, there are 
ways to do that (RPM, APT, ports, etc.).


At this point if you draw the app/environment boundary to include Python 
in your app, you should add zlib to your buildout, right?  What happens 
if you hit a bug in GCC when compiling Python?  Do you include GCC in 
your buildout to make sure you get the right version?  It gets worse the 
deeper you go. :)



___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-21 Thread David Pratt

Jim Fulton wrote:


On Dec 20, 2007, at 8:03 PM, David Pratt wrote:

Hey Darryl. Our minds were in a similar place today. I finally had 
decided to move away from my system python starting on my mac and also 
having a spot of trouble. I put something simple together below to 
illustrate what happened in my instance.


Here the eggs built with the python would not install. I initially 
though it was the lxml I was compiling, but it was just happened to be 
the first part requiring the custom-python. Eggs were made, and I 
could see the temp folder also, but when was finalizing the install, 
it threw the error and without much verbosity, the tempfolder it was 
installing into disappeared.


Note, I did not include the custom-python as a part in the buildout. I 
tend to like the python24:location type of notation since it is fairly 
explicit when you have a number of pieces of software you are building.


Regards
David

[buildout]
develop = .
parts = python24
   interpreter
index = http://download.zope.org/ppix
log-level = DEBUG

[python24]
recipe = zc.recipe:cmmi
url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz

[custom-python]
executable = ${python24:location}/bin/python

[interpreter]
recipe = zc.recipe.egg
interpreter = interpreter
eggs = Cheetah==1.0
python = custom-python


Here are the highlights:


Having *just* highlights makes it difficult to follow what's going on.




Some verification it is using the right python...

Getting distribution for 'Cheetah==1.0'.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6c7
Running easy_install:
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/parts/python24/bin/python 
-c from setuptools.command.easy_install import main; main() 
-mUNxd 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH -v 
/Users/davidpratt/Desktop/Buildouts/download-cache/dist/Cheetah-1.0.tar.gz 

path=/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/setuptools-0.6c7-py2.4.egg 



Processing Cheetah-1.0.tar.gz

Last few lines of output.. some time later.

changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/top_level.txt 
to 755
changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah 
to 755
changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah-compile 
to 755


Installed 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg 



Because this distribution was installed --multi-version, before you can
import modules from this package in an application, you will need to
'import pkg_resources' and then use a 'require()' call similar to one of
these examples, in order to select the desired version:

   pkg_resources.require(Cheetah)  # latest installed version
   pkg_resources.require(Cheetah==1.0)  # this exact version
   pkg_resources.require(Cheetah=1.0)  # this version or higher


Note also that the installation directory must be on sys.path at 
runtime for
this to work.  (e.g. by being the application's script directory, by 
being on

PYTHONPATH, or by being added to sys.path by your code.)

While:
 Installing interpreter.
 Getting distribution for 'Cheetah==1.0'.
Error: Couldn't install: Cheetah 1.0
[mac-pg:~/Desktop/Buildouts/osxdeploy2] davidpra%


That's very odd.  It feels like something is missing in the output.


Hi Jim. Yeah, I know. Weird huh ... the folder was there, then 
disappeared and no other clue why it bailed when finalizing. I did 
buildout -vvv to give me some good verbosity for debugging also.  Thing 
is when run with system python, it does the right stuff. Anything more 
you could suggest I check out?


I am glad I decided to stick with 10.4 for a little while longer. I am 
still on a PPC platform. I have had my own trouble nasty experience with 
a 10.4.9 -- 10.4.10 upgrade a couple of months ago and ended up 
reinstalling the os and leaving it at 10.4.9 until I get a new machine. 
Did not need the os interfering with my work. Odd for apple to have this 
issue, but I was not alone at the time.


Regards,
David


I tried reproducing this. I didn't have the contents of your directory, 
so I had to comment out develop = ..  You have a typo in the recipe 
specification for the python24 part. After fixing that, I ran the 
buildout above using the system Python on Mac OS X 10.5.1.  Worked 
great. The interpreter script uses the custom Python.


The interpreter runs, but when I try to import cheeta, I get a bus 
error..  Unfortunately, there is a problem with using cmmi to install 
Python on Mac OS X 10.5.  There is a bug in Python's configure that 
causes Python to be built incorrectly. The generated pyconfig.h file 
currently requires some tweaking. :(


Jim

--
Jim Fulton
Zope Corporation




Re: [Zope3-Users] Buildout and custom python

2007-12-20 Thread Jim Fulton


On Dec 20, 2007, at 6:03 PM, Darryl Cousins wrote:

I'm trying to created an isolated environment using buildout. I had
understood that defining a custom build python would ensure that all
eggs are installed and compiled with the custom python.

But when the buildout comes to install ZODB it is being compiled  
agains
the system python headers. (-I/usr/include/python2.4). Which is not  
what

I want, and at that point the custom python has not even been built.

Any thoughts.

[buildout]
python = custom-python
parts = python
custom-python


[python]
recipe = zc.recipe.cmmi
url = http://www.python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2
extra_options = --with-threads
   --with-readline
   --enable-unicode=ucs2

[custom-python]
executable = ${buildout:parts-directory}/python/bin/python



Your buildout doesn't mention ZODB3 or anything that uses it, so it is  
hard to comment.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-20 Thread David Pratt
Hey Darryl. Our minds were in a similar place today. I finally had 
decided to move away from my system python starting on my mac and also 
having a spot of trouble. I put something simple together below to 
illustrate what happened in my instance.


Here the eggs built with the python would not install. I initially 
though it was the lxml I was compiling, but it was just happened to be 
the first part requiring the custom-python. Eggs were made, and I could 
see the temp folder also, but when was finalizing the install, it threw 
the error and without much verbosity, the tempfolder it was installing 
into disappeared.


Note, I did not include the custom-python as a part in the buildout. I 
tend to like the python24:location type of notation since it is fairly 
explicit when you have a number of pieces of software you are building.


Regards
David

[buildout]
develop = .
parts = python24
interpreter
index = http://download.zope.org/ppix
log-level = DEBUG

[python24]
recipe = zc.recipe:cmmi
url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz

[custom-python]
executable = ${python24:location}/bin/python

[interpreter]
recipe = zc.recipe.egg
interpreter = interpreter
eggs = Cheetah==1.0
python = custom-python


Here are the highlights:

Some verification it is using the right python...

Getting distribution for 'Cheetah==1.0'.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6c7
Running easy_install:
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/parts/python24/bin/python 
-c from setuptools.command.easy_install import main; main() -mUNxd 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH -v 
/Users/davidpratt/Desktop/Buildouts/download-cache/dist/Cheetah-1.0.tar.gz

path=/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/setuptools-0.6c7-py2.4.egg

Processing Cheetah-1.0.tar.gz

Last few lines of output.. some time later.

changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/top_level.txt 
to 755
changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah 
to 755
changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah-compile 
to 755


Installed 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg


Because this distribution was installed --multi-version, before you can
import modules from this package in an application, you will need to
'import pkg_resources' and then use a 'require()' call similar to one of
these examples, in order to select the desired version:

pkg_resources.require(Cheetah)  # latest installed version
pkg_resources.require(Cheetah==1.0)  # this exact version
pkg_resources.require(Cheetah=1.0)  # this version or higher


Note also that the installation directory must be on sys.path at runtime for
this to work.  (e.g. by being the application's script directory, by 
being on

PYTHONPATH, or by being added to sys.path by your code.)

While:
  Installing interpreter.
  Getting distribution for 'Cheetah==1.0'.
Error: Couldn't install: Cheetah 1.0
[mac-pg:~/Desktop/Buildouts/osxdeploy2] davidpra%

Jim Fulton wrote:


On Dec 20, 2007, at 6:03 PM, Darryl Cousins wrote:

I'm trying to created an isolated environment using buildout. I had
understood that defining a custom build python would ensure that all
eggs are installed and compiled with the custom python.

But when the buildout comes to install ZODB it is being compiled agains
the system python headers. (-I/usr/include/python2.4). Which is not what
I want, and at that point the custom python has not even been built.

Any thoughts.

[buildout]
python = custom-python
parts = python
custom-python


[python]
recipe = zc.recipe.cmmi
url = http://www.python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2
extra_options = --with-threads
   --with-readline
   --enable-unicode=ucs2

[custom-python]
executable = ${buildout:parts-directory}/python/bin/python



Your buildout doesn't mention ZODB3 or anything that uses it, so it is 
hard to comment.


Jim

--
Jim Fulton
Zope Corporation


___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


___
Zope3-users mailing list
Zope3-users@zope.org
http://mail.zope.org/mailman/listinfo/zope3-users


Re: [Zope3-Users] Buildout and custom python

2007-12-20 Thread Darryl Cousins
Hi David,

I've tried out the methods described here:
http://mail.python.org/pipermail/distutils-sig/2007-November/008489.html

and have plumped for the buildout.sh shell script which builds python
and then uses that to bootstrap and re-continue the buildout. It is
working well and I see that all eggs are then compiled with the compiled
python.

I had already begun using a Makefile to build/test/run the project so I
find it convenient to use a shell script in the mix.

Regards,
Darryl

On Thu, 2007-12-20 at 21:03 -0400, David Pratt wrote:
 Hey Darryl. Our minds were in a similar place today. I finally had 
 decided to move away from my system python starting on my mac and also 
 having a spot of trouble. I put something simple together below to 
 illustrate what happened in my instance.
 
 Here the eggs built with the python would not install. I initially 
 though it was the lxml I was compiling, but it was just happened to be 
 the first part requiring the custom-python. Eggs were made, and I could 
 see the temp folder also, but when was finalizing the install, it threw 
 the error and without much verbosity, the tempfolder it was installing 
 into disappeared.
 
 Note, I did not include the custom-python as a part in the buildout. I 
 tend to like the python24:location type of notation since it is fairly 
 explicit when you have a number of pieces of software you are building.
 
 Regards
 David
 
 [buildout]
 develop = .
 parts = python24
  interpreter
 index = http://download.zope.org/ppix
 log-level = DEBUG
 
 [python24]
 recipe = zc.recipe:cmmi
 url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz
 
 [custom-python]
 executable = ${python24:location}/bin/python
 
 [interpreter]
 recipe = zc.recipe.egg
 interpreter = interpreter
 eggs = Cheetah==1.0
 python = custom-python
 
 
 Here are the highlights:
 
 Some verification it is using the right python...
 
 Getting distribution for 'Cheetah==1.0'.
 We have the best distribution that satisfies 'setuptools'.
 Picked: setuptools = 0.6c7
 Running easy_install:
 /Users/davidpratt/Desktop/Buildouts/osxdeploy2/parts/python24/bin/python 
 -c from setuptools.command.easy_install import main; main() -mUNxd 
 /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH -v 
 /Users/davidpratt/Desktop/Buildouts/download-cache/dist/Cheetah-1.0.tar.gz
 path=/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/setuptools-0.6c7-py2.4.egg
 
 Processing Cheetah-1.0.tar.gz
 
 Last few lines of output.. some time later.
 
 changing mode of 
 /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/top_level.txt
  
 to 755
 changing mode of 
 /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah
  
 to 755
 changing mode of 
 /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah-compile
  
 to 755
 
 Installed 
 /Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg
 
 Because this distribution was installed --multi-version, before you can
 import modules from this package in an application, you will need to
 'import pkg_resources' and then use a 'require()' call similar to one of
 these examples, in order to select the desired version:
 
  pkg_resources.require(Cheetah)  # latest installed version
  pkg_resources.require(Cheetah==1.0)  # this exact version
  pkg_resources.require(Cheetah=1.0)  # this version or higher
 
 
 Note also that the installation directory must be on sys.path at runtime for
 this to work.  (e.g. by being the application's script directory, by 
 being on
 PYTHONPATH, or by being added to sys.path by your code.)
 
 While:
Installing interpreter.
Getting distribution for 'Cheetah==1.0'.
 Error: Couldn't install: Cheetah 1.0
 [mac-pg:~/Desktop/Buildouts/osxdeploy2] davidpra%
 
 Jim Fulton wrote:
  
  On Dec 20, 2007, at 6:03 PM, Darryl Cousins wrote:
  I'm trying to created an isolated environment using buildout. I had
  understood that defining a custom build python would ensure that all
  eggs are installed and compiled with the custom python.
 
  But when the buildout comes to install ZODB it is being compiled agains
  the system python headers. (-I/usr/include/python2.4). Which is not what
  I want, and at that point the custom python has not even been built.
 
  Any thoughts.
 
  [buildout]
  python = custom-python
  parts = python
  custom-python
 
 
  [python]
  recipe = zc.recipe.cmmi
  url = http://www.python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2
  extra_options = --with-threads
 --with-readline
 --enable-unicode=ucs2
 
  [custom-python]
  executable = ${buildout:parts-directory}/python/bin/python
  
  
  Your buildout doesn't mention ZODB3 or anything that uses it, so it is 
  hard to comment.
  
  Jim
  
  -- 
  Jim Fulton
  Zope 

Re: [Zope3-Users] Buildout and custom python

2007-12-20 Thread David Pratt
Right. Thanks for this, there are couple of solutions here and shell 
script is a reasonable solution but these methods seems counter to the 
simplicity of buildout. I had also seen a rebootstrap package in svn. I 
wonder if something similar could not be accomplished this way using 
python with a means of producing a callback. I really don't want 
anything too messy to accomplish this. Many thanks.


Regards,
David


Darryl Cousins wrote:

Hi David,

I've tried out the methods described here:
http://mail.python.org/pipermail/distutils-sig/2007-November/008489.html

and have plumped for the buildout.sh shell script which builds python
and then uses that to bootstrap and re-continue the buildout. It is
working well and I see that all eggs are then compiled with the compiled
python.

I had already begun using a Makefile to build/test/run the project so I
find it convenient to use a shell script in the mix.

Regards,
Darryl

On Thu, 2007-12-20 at 21:03 -0400, David Pratt wrote:
Hey Darryl. Our minds were in a similar place today. I finally had 
decided to move away from my system python starting on my mac and also 
having a spot of trouble. I put something simple together below to 
illustrate what happened in my instance.


Here the eggs built with the python would not install. I initially 
though it was the lxml I was compiling, but it was just happened to be 
the first part requiring the custom-python. Eggs were made, and I could 
see the temp folder also, but when was finalizing the install, it threw 
the error and without much verbosity, the tempfolder it was installing 
into disappeared.


Note, I did not include the custom-python as a part in the buildout. I 
tend to like the python24:location type of notation since it is fairly 
explicit when you have a number of pieces of software you are building.


Regards
David

[buildout]
develop = .
parts = python24
 interpreter
index = http://download.zope.org/ppix
log-level = DEBUG

[python24]
recipe = zc.recipe:cmmi
url = http://python.org/ftp/python/2.4.4/Python-2.4.4.tgz

[custom-python]
executable = ${python24:location}/bin/python

[interpreter]
recipe = zc.recipe.egg
interpreter = interpreter
eggs = Cheetah==1.0
python = custom-python


Here are the highlights:

Some verification it is using the right python...

Getting distribution for 'Cheetah==1.0'.
We have the best distribution that satisfies 'setuptools'.
Picked: setuptools = 0.6c7
Running easy_install:
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/parts/python24/bin/python 
-c from setuptools.command.easy_install import main; main() -mUNxd 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH -v 
/Users/davidpratt/Desktop/Buildouts/download-cache/dist/Cheetah-1.0.tar.gz

path=/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/setuptools-0.6c7-py2.4.egg

Processing Cheetah-1.0.tar.gz

Last few lines of output.. some time later.

changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/top_level.txt 
to 755
changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah 
to 755
changing mode of 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg/EGG-INFO/scripts/cheetah-compile 
to 755


Installed 
/Users/davidpratt/Desktop/Buildouts/osxdeploy2/eggs/tmprQq-MH/Cheetah-1.0-py2.4-macosx-10.3-ppc.egg


Because this distribution was installed --multi-version, before you can
import modules from this package in an application, you will need to
'import pkg_resources' and then use a 'require()' call similar to one of
these examples, in order to select the desired version:

 pkg_resources.require(Cheetah)  # latest installed version
 pkg_resources.require(Cheetah==1.0)  # this exact version
 pkg_resources.require(Cheetah=1.0)  # this version or higher


Note also that the installation directory must be on sys.path at runtime for
this to work.  (e.g. by being the application's script directory, by 
being on

PYTHONPATH, or by being added to sys.path by your code.)

While:
   Installing interpreter.
   Getting distribution for 'Cheetah==1.0'.
Error: Couldn't install: Cheetah 1.0
[mac-pg:~/Desktop/Buildouts/osxdeploy2] davidpra%

Jim Fulton wrote:

On Dec 20, 2007, at 6:03 PM, Darryl Cousins wrote:

I'm trying to created an isolated environment using buildout. I had
understood that defining a custom build python would ensure that all
eggs are installed and compiled with the custom python.

But when the buildout comes to install ZODB it is being compiled agains
the system python headers. (-I/usr/include/python2.4). Which is not what
I want, and at that point the custom python has not even been built.

Any thoughts.

[buildout]
python = custom-python
parts = python
custom-python


[python]
recipe = zc.recipe.cmmi
url = http://www.python.org/ftp/python/2.4.4/Python-2.4.4.tar.bz2