On Fri, 10 Nov 2006, Serguei Osokine wrote:
Depends on what you're writing. For high-performance servers the disk is the main bottleneck, and you're expected to spend plenty of time waiting for the I/O to finish. The reason for this is that the disk seek time did not improve much in the past twenty years and is still on the order of 10 ms, whereas all the bandwidth and CPU frequency numbers have increased by several orders of magnitude.
i'm glad you brought this up. it's a huge point, and it's not limited to disks. latency has lagged bandwidth by orders of magnitude in memory, network connections, and CPU cores for decades.
david patterson [1], a berkeley cs prof who "wrote the book" on computer architecture, did a study [2] on this recently and concluded that bandwidth has been increasing with the *square* of latency since 1980 (!). the industry has only recently begun to appreciate this, even though it has huge implications for the way we build high-performance software.
there are standard ways to mitigate this, including caching, prefetching, and prediction. we could definitely use them more aggressively. in many latency sensitive areas at my work, we've given up on disks altogether and serve purely out of memory. that's clearly only a short term fix, though.
i've been itching to work on a project in this area for a long time, but haven't thought of an idea worth pursuing yet. any ideas?
[1] http://www.cs.berkeley.edu/~pattrsn/bio.html [2] http://216.239.63.104/search?q=cache:B5cT4nwYAsEJ:roc.cs.berkeley.edu/retreats/winter_04/posters/pattrsn_BWv1b.doc -Ryan -- http://snarfed.org/ _______________________________________________ p2p-hackers mailing list [email protected] http://lists.zooko.com/mailman/listinfo/p2p-hackers
