On Friday, November 10, 2006 Jeff Darcy wrote:
> For this very reason, it's probably OK to pretend it doesn't
> happen. When it does, it usually means the whole system's in
> pretty bad shape, and an unexpected pause in one application
> is probably the least of your worries.
Depends on what you're writing. For high-performance servers
the disk is the main bottleneck, and you're expected to spend plenty
of time waiting for the I/O to finish. The reason for this is that
the disk seek time did not improve much in the past twenty years
and is still on the order of 10 ms, whereas all the bandwidth and
CPU frequency numbers have increased by several orders of magnitude.
Thus your "per-spindle" data transfer rate can be anywhere between
2MB/s and 50MB/s (MB means megabytes here), depending on your data
organization, file system type, disk fragmentation, reading chunk
size, and a little bit on the disk model. RAIDs can raise this
number in proportion to the number of spindles (actual rotating
disk parts), more or less.
So typically you do not notice that until you go into high
bandwidth numbers, where things like "15 megabits per second per
spindle" start bothering you. Most of P2P applications are nowhere
close to this limit and can really pretend that it does not exist,
because these data transfer rates are in the dream range for a
typical P2P app. When you're deployed on a home computer and are
limited by something like network-wide average asymmetric uplink
rate, who cares. However, I would imagine that some vertical-market
LAN-based P2P could really hit this limit. Not sure whether anyone
is developing such stuff - P2P still tends to be WAN-based.
Best wishes -
S.Osokine.
10 Nov 2006.
P.S.: in case anyone is curious about these 15 megabits per spindle:
imagine yourself a web caching server that is serving 16-KB
files (on the average), all of which are different most of the
time, so most of them cannot be cached in RAM, and have to be
read from disk. These files are all over the disk, so reading
every one of them requires a disk seek, because you cannot
predict which file you'll need next. Thus you can read one
such file in about 10 ms, which is 1600 KB/s total, or about
15 mbits/sec. One of the worst-case disk performance scenarios,
actually.
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] Behalf Of Jeff Darcy
Sent: Friday, November 10, 2006 5:58 AM
To: theory and practice of decentralized computer networks
Subject: Re: [p2p-hackers] Cross-platform development
> I'll write some file i/o support for Twisted to be non-blocking but I
> still wonder about the following: does writing to a file ever block?
Yes. The data have to be written somewhere. Depending on the OS this
may be either the buffer or the page cache, but either way it's possible
that a write could block waiting for a resource to become available. A
fair number of system hangs come down to this, actually, because it's a
rare enough case that it's often not very well thought through or
tested. For this very reason, it's probably OK to pretend it doesn't
happen. When it does, it usually means the whole system's in pretty bad
shape, and an unexpected pause in one application is probably the least
of your worries.
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
