On Tue, Jul 30, 2013 at 3:45 PM, Jorge Chamorro <[email protected]> wrote: > On 30/07/2013, at 11:52, Ben Noordhuis wrote: >> >> >> Effective thread pool management is a topic of ongoing research but >> it's hard. With many kinds of workloads, having more threads actually >> _decreases_ the rate of throughput rather than increase it. I have >> alternative thread pool implementations sitting in branches that speed >> up some workloads 10x while slowing down others by that same amount or >> more. :-( > > In the end, it can't go any faster than the disk, but the disk is much faster > doing large sequential read/writes than random access ones (even on SSDs > random access is much slower than sequential r/w), which (random access) is > what ends up happening when dozens of threads fight for i/o in parallel: i/o > troughput gone haywire. > > IOW, reading/writing a hundred files sequentially is *always* faster than > reading/writing them in parallel (to the same disk, of course).
That's a big part of it. Another aspect is that concurrent reads from the same file are effectively unordered, destroying the operating system's ability to do proper read-ahead. Serializing reads would go some way towards addressing that but kills performance with yet other workloads. You just can't win. :-/ -- -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
