Re: speeding up buildworld/kernel
I have started using ccache as a result of this thread, the speed benefit for old hw is nice. p3-850 Cele This would be 3-4 hours for a buildworld and a make kernel, now it is 3234.877u 2318.444s 1:44:56.27 88.1%2580+1569k 79615+15476io 10553pf+0w K6-2 450 This would be 8-9 hours for a buildworld and a make kernel, now it is 5835.095u 3103.667s 3:29:42.86 71.0%-2168+1636k 140556+17495io 15489pf+0w This definitely makes it more feasible to use older hardware longer. Brian ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: speeding up buildworld/kernel
On Jan 25, 2008 1:50 AM, Brian [EMAIL PROTECTED] wrote: Hmm, 2 queries here. 1-wonder how much more gain would be gotten by using a speedy flash drive for the ccache folder. Actually you get the opposite. Here are my results with a USB 2.0 flash drive: cache directory /mnt/.ccache cache hit 12106 cache miss12 called for link 461 multiple source files 1 not a C/C++ file1228 unsupported compiler option7 files in cache122144 cache size 820.2 Mbytes max cache size 2.0 Gbytes 2h7m4.56s real 31m36.79s user 15m31.80s sys For reference, I've again rebuilt world and kernel. Ccache stats were the same as for the flash drive, but here's the time: 47m26.34s real 27m16.22s user 13m45.71s sys Flash drive is better than nothing at all, but much worse than using a hard drive. 2-I'm wondering about dependencies, like a change in x requires a recompile of y, but y doesnt look any different, is this smart enough to rebuild based on the dependency? Brian What do you mean by y doesn't look any different? If recompiling y results in the same object file, then there is no need to recompile it. That's all that ccache does. It considers all the variables that can possibly affect the contents of an object file. If those variables are the same as from a previously-cached run, then it returns the precompiled version of the file. More info is available at the ccache website: http://ccache.samba.org/ - Max ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
speeding up buildworld/kernel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com Free software != Free beer Blog: http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHmPHPQi2hk2LEXBARAofVAKD5MBoQ24Wal5CjKng5bUv8Pp2/mQCfcX6p NfCsz8egGQjn9KFPQ0Frths= =XeOG -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: speeding up buildworld/kernel
On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. - Max ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: speeding up buildworld/kernel
On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. You are indeed wrong. NO_CLEAN will work fine almost all the time - except in special circumstances. The few times it does not work one can always do a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value you can do the same thing for the ports tree. Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had problems originating with this. ccache is not very useful for buildworld, since among the first thing buildworld does is to build the compiler and then use the newly built compiler to compile the rest. I.e. the already installed compiler (which is the one ccache will handle) will not be used for most of the build thus removing almost all the advantage of ccache. It is supposed to be possible to use ccache for buildworld as well, but that would require a bit of hackery. As for speeding up the build even more there a couple of things that can be tried: You can add NO_PROFILE=true to make.conf if you do not need profiling libraries. Set CFLAGS/COPTFLAGS to -O instead of -O2. This should speed up the compiler a bit since it will no have to do as much work. This will make programs slightly less well optimized, but since the vast majority of the system binaries are not really CPU-bound anyway it is unlikely that any performance loss will be noticed. If you have more than one CPU-core in your machine (and an SMP-enabled kernel) you can use the -j flag to tell make to run several jobs in parallell. Just be aware that building with -j does get broken occasionaly and there is no promise that it will always be fixed quickly. If you do run into problems when building with -j, try without -j before sending any bug reports. -- Insert your favourite quote here. Erik Trulsson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: speeding up buildworld/kernel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Erik Trulsson wrote: On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. You are indeed wrong. NO_CLEAN will work fine almost all the time - except in special circumstances. The few times it does not work one can always do a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value you can do the same thing for the ports tree. Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had problems originating with this. ccache is not very useful for buildworld, since among the first thing buildworld does is to build the compiler and then use the newly built compiler to compile the rest. I.e. the already installed compiler (which is the one ccache will handle) will not be used for most of the build thus removing almost all the advantage of ccache. It is supposed to be possible to use ccache for buildworld as well, but that would require a bit of hackery. As for speeding up the build even more there a couple of things that can be tried: You can add NO_PROFILE=true to make.conf if you do not need profiling libraries. I thought most profiled libs had been removed in current but I will try this. I was also looking at NO_SHARED but my gut says this would cause a sigficant performence hit. Set CFLAGS/COPTFLAGS to -O instead of -O2. This should speed up the compiler a bit since it will no have to do as much work. This will make programs slightly less well optimized, but since the vast majority of the system binaries are not really CPU-bound anyway it is unlikely that any performance loss will be noticed. Do you have any numbers on this? If you have more than one CPU-core in your machine (and an SMP-enabled kernel) you can use the -j flag to tell make to run several jobs in parallell. Just be aware that building with -j does get broken occasionaly and there is no promise that it will always be fixed quickly. If you do run into problems when building with -j, try without -j before sending any bug reports. Since I like to run it in the background (i.e. while doing stuff on a different X screen) I usually don't use -j unless I am doing a bare metal install and then I typically do core*4+2 for it's value. - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com Free software != Free beer Blog: http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.4 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHmP9KQi2hk2LEXBARAu7zAJ4/sGAzpMFCZOKkZBVx/s07KTRw9gCgwF1m 6ee/hiJIvj8gyieoq/ZxIz0= =tnVh -END PGP SIGNATURE- ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: speeding up buildworld/kernel
On Jan 24, 2008 4:05 PM, Erik Trulsson [EMAIL PROTECTED] wrote: On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. You are indeed wrong. NO_CLEAN will work fine almost all the time - except in special circumstances. The few times it does not work one can always do a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value you can do the same thing for the ports tree. Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had problems originating with this. ccache is not very useful for buildworld, since among the first thing buildworld does is to build the compiler and then use the newly built compiler to compile the rest. I.e. the already installed compiler (which is the one ccache will handle) will not be used for most of the build thus removing almost all the advantage of ccache. It is supposed to be possible to use ccache for buildworld as well, but that would require a bit of hackery. That's not true. I just ran `make buildworld buildkernel` on my firewall. Here are ccache stats when the operation finished: [EMAIL PROTECTED] [/root]# ccache -s cache directory /srv/.ccache cache hit 12056 cache miss38 called for link 461 multiple source files 1 not a C/C++ file1228 unsupported compiler option7 files in cache117366 cache size 679.6 Mbytes max cache size 2.0 Gbytes Ccache is used through the entire build process and there is no hackery involved. Just follow the directions for changing the compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40 seconds. I don't recall exactly what it was without ccache, but I think it was around 3 hours. Just make sure that you set the CCACHE_HASH_COMPILER variable, otherwise it will assume that the compiler is different just because its modification time has changed. - Max ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: speeding up buildworld/kernel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Maxim Khitrov wrote: On Jan 24, 2008 4:05 PM, Erik Trulsson [EMAIL PROTECTED] wrote: On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. You are indeed wrong. NO_CLEAN will work fine almost all the time - except in special circumstances. The few times it does not work one can always do a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value you can do the same thing for the ports tree. Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had problems originating with this. ccache is not very useful for buildworld, since among the first thing buildworld does is to build the compiler and then use the newly built compiler to compile the rest. I.e. the already installed compiler (which is the one ccache will handle) will not be used for most of the build thus removing almost all the advantage of ccache. It is supposed to be possible to use ccache for buildworld as well, but that would require a bit of hackery. That's not true. I just ran `make buildworld buildkernel` on my firewall. Here are ccache stats when the operation finished: [EMAIL PROTECTED] [/root]# ccache -s cache directory /srv/.ccache cache hit 12056 cache miss 38 called for link 461 multiple source files 1 not a C/C++ file1228 unsupported compiler option7 files in cache117366 cache size 679.6 Mbytes max cache size 2.0 Gbytes Ccache is used through the entire build process and there is no hackery involved. Just follow the directions for changing the compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40 seconds. I don't recall exactly what it was without ccache, but I think it was around 3 hours. Just make sure that you set the CCACHE_HASH_COMPILER variable, otherwise it will assume that the compiler is different just because its modification time has changed. - Max I think Erik is correct here are some times (done in the order listed): After adding NO_PROFILE to make.conf: flosoft# cvs -q update -dP M lib/libc/stdlib/malloc.c flosoft# time make buildworld buildkernel installkernel installworld . . . 129.160u 49.686s 6:48.67 43.7%1001+2748k 16259+6155io 29699pf+0w After installing ccache (first run): flosoft# setenv CCACHE_HASH_COMPILER 1 flosoft# set CCACHE_HASH_COMPILER=1 flosoft# time make buildworld buildkernel installkernel installworld . . . 117.765u 46.502s 4:56.24 55.4%474+2667k 674+6151io 8269pf+0w flosoft# ccache -s cache directory /root/.ccache cache hit 0 cache miss 0 files in cache 0 cache size 0 Kbytes max cache size 976.6 Mbytes Second run: flosoft# time make buildworld buildkernel installkernel installworld . . . 118.318u 46.055s 4:46.64 57.3%475+2644k 251+6145io 6203pf+0w flosoft# !cc ccache -s cache directory /root/.ccache cache hit 0 cache miss 0 files in cache 0 cache size 0 Kbytes max cache size 976.6 Mbytes After clearing out /usr/obj (with ccache turned off): Note: Even though I didn't time without NO_PROFILE this time I have in the past on the same machine and got about 1 hour 5 mins (so not a big savings) flosoft# rm -rf /usr/objflosoft# time make buildworld buildkernel installkernel installworld . . . 2549.561u 387.975s 58:08.69 84.2%6352+7186k 27134+14972io 11234pf+0w Turning ccache back on (1st run): flosoft# rm -rf /usr/obj flosoft# setenv CCACHE_HASH_COMPILER 1 flosoft# set
Re: speeding up buildworld/kernel
On Jan 24, 2008 11:01 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Maxim Khitrov wrote: On Jan 24, 2008 4:05 PM, Erik Trulsson [EMAIL PROTECTED] wrote: On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. You are indeed wrong. NO_CLEAN will work fine almost all the time - except in special circumstances. The few times it does not work one can always do a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value you can do the same thing for the ports tree. Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had problems originating with this. ccache is not very useful for buildworld, since among the first thing buildworld does is to build the compiler and then use the newly built compiler to compile the rest. I.e. the already installed compiler (which is the one ccache will handle) will not be used for most of the build thus removing almost all the advantage of ccache. It is supposed to be possible to use ccache for buildworld as well, but that would require a bit of hackery. That's not true. I just ran `make buildworld buildkernel` on my firewall. Here are ccache stats when the operation finished: [EMAIL PROTECTED] [/root]# ccache -s cache directory /srv/.ccache cache hit 12056 cache miss 38 called for link 461 multiple source files 1 not a C/C++ file1228 unsupported compiler option7 files in cache117366 cache size 679.6 Mbytes max cache size 2.0 Gbytes Ccache is used through the entire build process and there is no hackery involved. Just follow the directions for changing the compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40 seconds. I don't recall exactly what it was without ccache, but I think it was around 3 hours. Just make sure that you set the CCACHE_HASH_COMPILER variable, otherwise it will assume that the compiler is different just because its modification time has changed. - Max No commands where issued on the terminal except the ones showed and ccache was installed with default settings (I didn't modify any files after doing make install for ccache) It doesn't work like that. You have to read /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and configure things properly before ccache is used for building the os. In /etc/make.conf you need to add the following: .if exists(/usr/local/libexec/ccache) !defined(NOCCACHE) \ (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*)) CC= /usr/local/libexec/ccache/world-cc CXX= /usr/local/libexec/ccache/world-c++ .endif Unless you actually want to use /root/.ccache (and have enough space for it), I would move that directory to some other partition. In my /etc/csh.cshrc I have this configuration: setenv CCACHE_DIR /srv/.ccache setenv CCACHE_PATH /usr/bin:/usr/local/bin setenv CCACHE_HASH_COMPILER 1 Once you've done all of this, rebuild the world. You can run ccache -s during that process. If the hit/miss numbers stay at 0 then ccache is not being used. Check your configuration and try again. The first run will be slower than normal (though not by much), because the cache is being populated for the first time. On the second run, however, clear ccache stats (ccache -z) and you should see the same results as I've posted above. With the exception of only a few files, just about everything should be obtained from the cache and not compiled from scratch. - Max ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to
Re: speeding up buildworld/kernel
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Maxim Khitrov wrote: On Jan 24, 2008 11:01 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Maxim Khitrov wrote: On Jan 24, 2008 4:05 PM, Erik Trulsson [EMAIL PROTECTED] wrote: On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote: On Jan 24, 2008 3:15 PM, Aryeh M. Friedman [EMAIL PROTECTED] wrote: I update my sources at least once a day and do buildworld/kernel just as often... It seems some stuff that needs not be recompiled is on every single run for example gcc and kerbos. I have NO_CLEAN in /etc/make.conf is there anything else I can do to speed stuff up... for ref here is my /etc/make.conf: CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8 - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers Developer, not business, friendly http://www.flosoft-systems.com I might be wrong, but NO_CLEAN seems like a bad idea except in special circumstances. Install ccache, but make sure you set CCACHE_HASH_COMPILER environment variable to 1. That will make sure that the cache stays valid if the compiler executable is overwritten by an identical copy (as it would be on installworld). When the compiler changes the cache will be repopulated on the next rebuild. You are indeed wrong. NO_CLEAN will work fine almost all the time - except in special circumstances. The few times it does not work one can always do a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value you can do the same thing for the ports tree. Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had problems originating with this. ccache is not very useful for buildworld, since among the first thing buildworld does is to build the compiler and then use the newly built compiler to compile the rest. I.e. the already installed compiler (which is the one ccache will handle) will not be used for most of the build thus removing almost all the advantage of ccache. It is supposed to be possible to use ccache for buildworld as well, but that would require a bit of hackery. That's not true. I just ran `make buildworld buildkernel` on my firewall. Here are ccache stats when the operation finished: [EMAIL PROTECTED] [/root]# ccache -s cache directory /srv/.ccache cache hit 12056 cache miss 38 called for link 461 multiple source files 1 not a C/C++ file1228 unsupported compiler option 7 files in cache117366 cache size 679.6 Mbytes max cache size 2.0 Gbytes Ccache is used through the entire build process and there is no hackery involved. Just follow the directions for changing the compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40 seconds. I don't recall exactly what it was without ccache, but I think it was around 3 hours. Just make sure that you set the CCACHE_HASH_COMPILER variable, otherwise it will assume that the compiler is different just because its modification time has changed. - Max No commands where issued on the terminal except the ones showed and ccache was installed with default settings (I didn't modify any files after doing make install for ccache) It doesn't work like that. You have to read /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and configure things properly before ccache is used for building the os. In /etc/make.conf you need to add the following: .if exists(/usr/local/libexec/ccache) !defined(NOCCACHE) \ (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*)) CC= /usr/local/libexec/ccache/world-cc CXX= /usr/local/libexec/ccache/world-c++ .endif Unless you actually want to use /root/.ccache (and have enough space for it), I would move that directory to some other partition. In my /etc/csh.cshrc I have this configuration: setenv CCACHE_DIR /srv/.ccache setenv CCACHE_PATH /usr/bin:/usr/local/bin setenv CCACHE_HASH_COMPILER 1 Once you've done all of this, rebuild the world. You can run ccache -s during that process. If the hit/miss numbers stay at 0 then ccache is not being used. Check your configuration and try again. The first run will be slower than normal (though not by much), because the cache is being populated for the first time. On the second run, however, clear ccache stats (ccache -z) and you should see the same results as I've posted above. With the exception of only a few files, just about everything should be obtained from the cache and not compiled from scratch. - Max Here are the results: First run (wo/ removing /usr/obj) using the above settings: 119.025u 46.448s 8:11.19 33.6%487+2711k 15027+6126io 28785pf+0w flosoft# ccache -s cache directory
Re: speeding up buildworld/kernel
Hmm, 2 queries here. 1-wonder how much more gain would be gotten by using a speedy flash drive for the ccache folder. 2-I'm wondering about dependencies, like a change in x requires a recompile of y, but y doesnt look any different, is this smart enough to rebuild based on the dependency? Brian ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
