Re: make -j$n buildworld : use of -j investigated
At 12:51 PM +0900 11/23/04, Rob wrote: Laurence Sanford wrote: Rob wrote: With these simple tests, I come to the conclusion that make -j$n buildworld is best with n = number of CPUs. Does that make sense? Rob. This is what I've been telling people and using myself for years. However, I've been shot down on this several times, so I just leave everyone alone and let them do their own thing. You and I will be getting it done a little faster though. Not really faster, but higher values do not make a difference, well, as long as the extra processes do not force the use of swap. Intensified swapping because of a high -j value slows down the build considerably. I don't understand why this is reason for debate. My test has obvious results on various of my PCs, and was very quickly done: I wrote a script with a loop that built the world again and again, doing a 'touch' to a file immediately before and after the build. Got all my data within a day or so. There are many things will effect what different people see for buildworld times on their own hardware. You also have to make some effort to make sure you're doing *exactly* the same thing on each test build, as your results may be skewed due to various caching that can go on. You're not the only one who has benchmarked this for yourself, and my guess is that everyone who does a serious benchmark will simply find out the right answer for *their* hardware. I have had dual- processor machines where buildworlds kept getting faster up to -j9, but even -j9 was only about 2% faster than -j4, so I stuck with -j4. In my case, I tried with -j values from 1 to 15. On my single processor i386 systems, I generally find that -j2 is faster than -j1. Anything more than -j2 and performance seems to get worse. On my single processor sparc64 system, the disks are so slow that there's no point doing -j at all. So my own rule of thumb is -j of 2*number-of-CPU's (but then most of my machines have plenty of memory), except for the Ultra-10. Also, did you do your benchmarking before or after the recent fixes to -j processing? I haven't redone them after that change, and I think it will be interesting to see what effect that has. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: make -j$n buildworld : use of -j investigated
Garance A Drosihn wrote: Also, did you do your benchmarking before or after the recent fixes to -j processing? I haven't redone them after that change, and I think it will be interesting to see what effect that has. These tests were done on 5.3-Stable as of Nov. 22nd. I believe these -j changes were applied to 6-Current. Or am I wrong? Rob. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: make -j$n buildworld : use of -j investigated
At 10:41 AM +0900 11/24/04, Rob wrote: Garance A Drosihn wrote: Also, did you do your benchmarking before or after the recent fixes to -j processing? I haven't redone them after that change, and I think it will be interesting to see what effect that has. These tests were done on 5.3-Stable as of Nov. 22nd. I believe these -j changes were applied to 6-Current. Or am I wrong? You are correct. My mistake, I overlooked what you said about this being on 5.3-stable. I am too focused on 6.x-current due to a few system builds that I've been doing. Still, I do think it will be interesting to see how the recent -j changes will effect the benchmarking. The way it worked before, you often getting more -j than you asked for, and I think that would have to be significant. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
make -j$n buildworld : use of -j investigated
Hi, I have tested following with FreeBSD 5.3-Stable. On several different PCs I have used make -j$n buildworld with $n ranging from 1 to 9. Although people suggest -j4 as optimal in general case, I have come to a very different conclusion: 1) single CPU with enough RAM (2 GHz, 512 MB) there's no significant speed up in the range -j1 to -j9. So -j1 is as good as -j9. 2) single CPU with little RAM (333 MHz, 64 MB) speed slows down rapidly from -j1 to -j9, because of intensive swapping. So -j1 performs best in this case. 3) dual CPU with enough RAM (2 x 800 MHz, 1GB) speed up by almost two from -j1 to -j2, but after that no noticeable speed up anymore. So -j2 is as good as -j9. With these simple tests, I come to the conclusion that make -j$n buildworld is best with n = number of CPUs. Does that make sense? Rob. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: make -j$n buildworld : use of -j investigated
Laurence Sanford wrote: Rob wrote: With these simple tests, I come to the conclusion that make -j$n buildworld is best with n = number of CPUs. Does that make sense? Rob. This is what I've been telling people and using myself for years. However, I've been shot down on this several times, so I just leave everyone alone and let them do their own thing. You and I will be getting it done a little faster though. Not really faster, but higher values do not make a difference, well, as long as the extra processes do not force the use of swap. Intensified swapping because of a high -j value slows down the build considerably. I don't understand why this is reason for debate. My test has obvious results on various of my PCs, and was very quickly done: I wrote a script with a loop that built the world again and again, doing a 'touch' to a file immediately before and after the build. Got all my data within a day or so. Rob. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: make -j$n buildworld : use of -j investigated
The idea behind the speedups by using nnumber of CPUs is that you use unused CPU cycles during disk activity. Obviously this works only on systems which use 'low CPU usage storage' such as SCSI, firewire or more prominently NFS for the sources. Of cause this also assumes you don't have any performance penalties due to increased swap usage... So in a 'normal' IDE environment it probably wont make a difference. Regarding your test, I'd say don't trust your results unless you have rebootet the machine betweeen the tests, otherwise the cache will mess up your results. Kind regards, Alex. Rob wrote: Laurence Sanford wrote: Rob wrote: With these simple tests, I come to the conclusion that make -j$n buildworld is best with n = number of CPUs. Does that make sense? Rob. This is what I've been telling people and using myself for years. However, I've been shot down on this several times, so I just leave everyone alone and let them do their own thing. You and I will be getting it done a little faster though. Not really faster, but higher values do not make a difference, well, as long as the extra processes do not force the use of swap. Intensified swapping because of a high -j value slows down the build considerably. I don't understand why this is reason for debate. My test has obvious results on various of my PCs, and was very quickly done: I wrote a script with a loop that built the world again and again, doing a 'touch' to a file immediately before and after the build. Got all my data within a day or so. Rob. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
