On 30/07/2013, at 16:39, Ben Noordhuis wrote: > On Tue, Jul 30, 2013 at 3:45 PM, Jorge Chamorro <[email protected]> > wrote: >> On 30/07/2013, at 11:52, Ben Noordhuis wrote: >>> >>> >>> Effective thread pool management is a topic of ongoing research but >>> it's hard. With many kinds of workloads, having more threads actually >>> _decreases_ the rate of throughput rather than increase it. I have >>> alternative thread pool implementations sitting in branches that speed >>> up some workloads 10x while slowing down others by that same amount or >>> more. :-( >> >> In the end, it can't go any faster than the disk, but the disk is much >> faster doing large sequential read/writes than random access ones (even on >> SSDs random access is much slower than sequential r/w), which (random >> access) is what ends up happening when dozens of threads fight for i/o in >> parallel: i/o troughput gone haywire. >> >> IOW, reading/writing a hundred files sequentially is *always* faster than >> reading/writing them in parallel (to the same disk, of course). > > That's a big part of it. > > Another aspect is that concurrent reads from the same file are > effectively unordered, destroying the operating system's ability to do > proper read-ahead. Serializing reads would go some way towards > addressing that but kills performance with yet other workloads. You > just can't win. :-/
A queue+thread pool per I/O *device* would speedup things under certain circumstances, istm, but perhaps it's not worth the effort because that circumstances don't happen so often. -- ( Jorge )(); -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
