Ross Patterson <m...@rpatterson.net> writes:
> Marius Gedminas <mar...@pov.lt> writes:
>
>> On Sat, Jan 21, 2012 at 02:19:03AM -0800, Ross Patterson wrote:
>>> I moved this patch to a branch of zc.buildout:
>>>
>>> svn+ssh://svn.zope.org/repos/main/zc.buildout/branches/env-cache
[...snip...]
>> How can I test your branch? It doesn't seem to be enough to add
>>
>> [buildout]
>> develop += /path/to/your/branch/checkout
>> [versions]
>> zc.buildout =
>>
>> to my existing buildout.cfg.
>
> Not sure how. I've been patching eggs to do the 'timed' tests with
> real-world buildout. Maybe you could install the checkout as a develop
> egg to the python installation? Ick. Maybe you could edit bin/buildout
> and adjust the path for the buildout egg? Ick.
Here's how I just managed to test my real-world buildouts using my
zc.buildout and buildout.dumppickedversions checkouts and compare those
times against the released versions (1.4 and 0.4 respectively).
To get the buildout to use the zc.buildout checkout, the bootstrap.py
script needs to be convinced to use it when there is not buildout
environment controlling what dists it finds. As such you need to
install your zc.buildout checkout as a develop egg in the python
installation you are using for the buildout you want to test.
I recommend using a virtualenv so that when you install the develop egg
it only influences that specific buildout:
$ cd ~/src/my-buildout
$ virtualenv -p ~/src/buildout.python/python-2.6 --no-site-packages .
$ bin/python2.6 bootstrap.py -d
Then time your baseline with the existing buildout release:
$ bin/buildout -N && time bin/buildout -N
Then install the zc.buildout develop egg from where ever your checkout
it into the virtualenv python installation:
$ cd ~/src/zc.buildout
$ ~/src/my-buildout/bin/python2.6 setup.py develop
Then use that python with the bootstrap.py script from the checkout and
tell it to use the unreleased version with "-t":
$ cd ~/src/my-buildout
$ bin/python2.6 ~/src/zc.buildout/bootstrap/bootstrap.py -dt
versions:zc.buildout=
Now time with the zc.buildout checkout by overriding the version pin for
zc.buildout:
$ bin/buildout -N versions:zc.buildout= && time bin/buildout -N
versions:zc.buildout=
Now, time it with the buildout.dumppickedversions checkout by
overriding both version pins and adding a develop egg for
buildout.dumppickedversions. In the case of buildout.dumppickedversions
we *can* use buildout alone to make it use the develop egg and need not
install it into the virtualenv python installation:
$ bin/buildout -N versions:zc.buildout=
versions:buildout.dumppickedversions=
buildout:develop+=/home/me/src/buildout.dumppickedversions && time bin/buildout
-N versions:zc.buildout= versions:buildout.dumppickedversions=
buildout:develop+=/home/me/src/buildout.dumppickedversions
Finally, cleanup the zc.buildout develop egg:
$ bin/easy_install-2.6 -m zc.buildout
With this procedure, here are my timings with two different buildouts,
one that is development focused with lots of different parts with
slightly different to wildly different required dists, and one that is a
production buildout with many identical parts. I did all this after
completely clearing out my egg and download caches and running each
buildout once first to re-download all eggs so that results aren't
skewed by a large, out-of-date egg cache:
+--------------------------------------+-------------+------------+
| | development | production |
+--------------------------------------+-------------+------------+
| released eggs | 1m52.068s | 2m2.731s |
+--------------------------------------+-------------+------------+
| zc.buildout checkout | 0m56.079s | 1m3.781s |
+--------------------------------------+-------------+------------+
| buildout.dumppickedversions checkout | 0m16.453s | 0m20.752s |
+--------------------------------------+-------------+------------+
All told, roughly a 6-7 fold decrease in buildout runtime. This really
makes re-running buildout something I no longer feel an urge to avoid.
I'd really like to see this in releases soon so that people may feel
less buildout pain.
Ross
_______________________________________________
Distutils-SIG maillist - Distutils-SIG@python.org
http://mail.python.org/mailman/listinfo/distutils-sig
