The build system enhancements I proprosed some time ago is finished now.
Here are some pragmatic (quite unrepresentative/inaccurate, due cpufreq
and loaded machines) benchmarks:
Setup:
I disable ccache for all of this benchmarks, distcc is used as noted.
The compiler commandlines still always use the 'ccache distcc' prefix.
The tests are run on my laptop (slow encrypted disk) and all other hosts
are connected via WLan.
CC=ccache distcc g++-4.1
CXX=ccache distcc g++-4.1
CCACHE_DISABLE=true
$ rm * -rf
$ ../configure --program-suffix=_flatam --with-external-ffmpeg
1. old build system using SUBDIRS
1.1. using distcc over all machines here
DISTCC_HOSTS="10.20.20.20/2,lzo 10.20.60.10/3,lzo 10.20.10.40/1,lzo \
10.20.50.10/2,lzo"
Note: my laptop is already loaded with preprocessing and feeding the
cluster, there is no compilation on localhost.
1.1.1 full rebuild
$ time make -j 9
...
real 6m12.913s
user 1m40.237s
sys 0m32.071s
1.1.2. rebuild with few files in cinelerra touched
$ touch ../cinelerra/cache.*
$ time make -j 9
...
real 1m11.398s
user 0m29.755s
sys 0m6.763s
1.1.3. rebuild with a plugin touched
$ touch ../plugins/blur/blur.*
$ time make -j 9
...
real 0m8.623s
user 0m5.966s
sys 0m0.553s
1.2. build sequential no distcc hosts
DISTCC_HOSTS=localhost/1
1.2.1 full rebuild
$ time make
....
real 9m11.694s
user 7m33.710s
sys 0m38.491s
1.2.2. rebuild with few files in cinelerra touched
$ touch ../cinelerra/cache.*
$ time make
...
real 3m7.865s
user 2m39.326s
sys 0m9.649s
1.2.3. rebuild with a plugin touched
$ touch ../plugins/blur/blur.*
$ time make
...
real 0m9.239s
user 0m7.829s
sys 0m0.587s
2. the new build system using included makefiles
2.1. using distcc over all machines here
2.1.1 full rebuild
$ time make -j 9
....
real 3m13.020s
user 1m38.214s
sys 0m31.068s
2.1.2. rebuild with few files in cinelerra touched
$ touch ../cinelerra/cache.*
$ time make -j 9
...
real 1m9.546s
user 0m30.031s
sys 0m6.270s
2.1.3. rebuild with a plugin touched
$ touch ../plugins/blur/blur.*
$ time make -j 9
...
real 0m8.748s
user 0m6.326s
sys 0m0.317s
2.2. build sequential no distcc hosts
DISTCC_HOSTS=localhost/1
2.2.1 full rebuild
$ time make
....
real 9m3.112s
user 7m39.167s
sys 0m37.714s
2.2.2. rebuild with few files in cinelerra touched
$ touch ../cinelerra/cache.*
$ time make
...
real 3m10.850s
user 2m44.649s
sys 0m9.513s
2.2.3. rebuild with a plugin touched
$ touch ../plugins/blur/blur.*
$ time make
...
real 0m9.255s
user 0m8.629s
sys 0m0.323s
Conclusions:
* parallel builds are double as fast (mostly because plugins build
parallel now)
* rebuilds are not that improved like I hoped (maybe I touched the
wrong files) :(
* building on single processor is not improved (that wasn't expected
anyway)
* distcc rocks, but doesn't scale that well maybe perhaps of my wlan or
due the slow HD in my laptop.
* enableing ccache would give another speed boost but isn't useful for
this comparsions.
* Not measured here, but configure is faster since far less Makefiles
are generated.
* So far this is just a minimal translation, there is still room for
improvement.
Whats next:
Some of the issues I mentioned earlier are not yet fixed
* I only did *works-for-me* efforts so far, that is:
+ buildinfo will be unconditionally recreated
+ works only --with-external-ffmpeg (I didnt decided yet if to
keep SUBDIRS for ffmpeg or also to turn it into a include)
+ some (clean..:) targets are commented out
All or some of this work could be merged back into the SVN. There are
some fixes and changes to the sources too (garbled dependencies, path
fixes, libaffine factored out, ...), please review it! I'll prepare a
patch including what we want in SVN on request when we concluded what
shall go in there.
Christian
_______________________________________________
Cinelerra mailing list
Cinelerra@skolelinux.no
https://init.linpro.no/mailman/skolelinux.no/listinfo/cinelerra
