On Sat, Sep 30, 2006 at 03:49:51PM +0200, Michel Talon wrote: > Kris Kennaway said: > > > FYI, I'm not sure if the python version is parallelizable, but you do > > get a small benefit from using parallelized 'make index' builds (via > > INDEX_JOBS) on a typical SMP machine. > > My script runs parallel makes, of the order of ten simultaneous makes.
OK, FYI I see a performance drop if you use too much parallelism (10 is "far too much" on a typical system; ~3 is probably optimal for a HTT machine). > You can see them, running top at the same time, but of course they are very > transient, since usually one make lasts of the order of 0.1 s on my machine. > Anyways, still on my machine i have gained of the order of 20% wall clock time > by doing that. And it is only an hyperthreaded P4 (running in SMP mode). This > is why i was curious to know the effect of this idea on a machine with real > SMP and perhaps disks supporting tagged queuing, which should allow better > throughput for these simultaneous makes. I had, perhaps naively, assumed that > with a good machine one can divide time by 2. The only situation where you might reasonably expect to see something approaching a factor of 2 speed improvement is if your ports tree is on a striped or mirrored device on separate controllers, so 2 reads can be completely decoupled from each other in hardware. I've not tried this though. Kris
pgpxpvSFgXbUj.pgp
Description: PGP signature
