On 24 Aug 2004, Jake McGuire <[EMAIL PROTECTED]> wrote: > What's the rationale for getting the CPU lock before starting to > preprocess?
The rationale is that "3 processes" means "3 processes", not "3 compilers." It covers linking too. Preprocessing is quite different to compilation: typically more IO/bus/VM-bound and pretty light on CPU. So you could argue that they have different limits on different machines. Something with a fast CPU and slow network filesystem might be relatively slow at preprocessing. But as a simple model, you want to run some small multiple of the number of CPUs on any machine. > Our situation was admittedly somewhat pathological, but in any case we > were seeing preprocessing taking 40%-80% of the time to compile a > particular file. We also had far more jobs scheduled than we had slots > in our hosts file, which was exacerbating the slow preprocessing as it > looks like distcc attempts to obtain (num_jobs - num_cpus) * num_cpus > locks per second. Which was pushing 5,000 at times, and this over NFS. > > Obviously we've got some internal IT issues to resolve, but is there a > reason why getting the lock can't be moved to after the preprocessing > step? I think the real issue here is that the poll-based locking is just not very efficient when there is contention. It was designed using file locks to be simple, portable and robust, and efficient for moderately low contention. I'm not surprised it dies when 5000 processes contend for a couple of locks. (SysV semaphores do badly when distcc is abruptly killed, which often happens to compilers.) A better design would let clients sleep while waiting. I have some ideas but suggestions would be welcome. I'm not sure that -j5000 is really a sensible scenario to support. Even if it did lock after running the preprocessor then running 5000 copies of cpp would probably melt your machine. You can try moving it and see if it does better. -- Martin
signature.asc
Description: Digital signature
__ distcc mailing list http://distcc.samba.org/ To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/distcc
