#include <hallo.h> * Guus Sliepen [Sun, Jan 04 2009, 10:45:23AM]: > On Sat, Jan 03, 2009 at 09:57:33PM +0100, Eduard Bloch wrote: > > > PS: I plan to hack it a little bit and use syssconf function on Debian > > systems to determine the real number of CPU cores (#x) since pigz's > > default value is 8 which is much more than home systems have nowadays, > > and the performance isn't getting (much) better with a constant number > > of idle threads, they just consume more memory. > > Although sysconf() can tell you the total number of cores and the number of > cores that are online in your system, it does not tell you how many cores are > available for your program. It's better to use sched_getaffinity() to get the > set of CPU (cores) available to your program. And if you know better than the > OS, use sched_setaffinity() to bind each thread to it's own core.
Sounds like a plan, but I don't feel very comfortable to do that in the Debian package. Let me explain why: - sched_setaffinity method seems to be Linux specific - though it might be beneficial to pinpoint each thread to the same CPU, I don't think (*assumption*) that the data resides long enough in the CPU cache anyway so we wouldn't win much - I have not used it before but I used the sysconf method in portable apps (i.e. in cloop-utils), sysconf method is also used in pbzip2 - it's hard to imagine environments with big difference between count(cores) and count(available cores) - the code change is minimal, see below @Marc: please check the -s option, it's called -S in gzip (upper case). What's the reason for the different case? Regards, Eduard. --- pigz-2.1.4.orig/pigz.c +++ pigz-2.1.4/pigz.c @@ -2795,8 +2795,13 @@ #ifdef NOTHREAD procs = 1; #else + +#ifdef _SC_NPROCESSORS_ONLN + procs=sysconf(_SC_NPROCESSORS_ONLN); +#else procs = 8; -#endif +#endif /* _SC_NPROCESSORS_ONLN */ +#endif /* NOTHREAD */ size = 131072UL; rsync = 0; /* don't do rsync blocking */ dict = 1; /* initialize dictionary each thread */ -- Naja, Garbage Collector eben. Holt den Müll sogar vom Himmel. (Heise Trollforum über Java in der Flugzeugsteuerung)
signature.asc
Description: Digital signature