Hi Joe,
don't understand or believe this. If the server's local disks can
read/write at 300MB/s and the networking can run substantially faster than
100 MB/s, I don't see any constraint to faster operation. But perhaps
someone on this list can provide real-world data (or say why it can't
work).
.... ok, a number of different issues going on here
a) the 300 MB/s (SATA II, right?) is the max theoretical speed. You are
going to get something close to this in pure buffer to memory transactions in
specialized cases. Normally you will see 50-70 MB/s for these disks for large
block sequential reads. SATA also does a bit of interrupting... you need a
*good* SATA controller, or you will see your interrupt rate go up 10x in
heavy disk load times. Software RAID will increase this a bit as well.
The NFS server is using RAID controllers, in PCI-X busses. Many disks are
striped together. We've measured (locally) > 350 MB/s read and > 250
MB/sec write on these arrays.
b) If this is gigabit, you get about 110 MB/s max in best case scenarios,
with the wind at your packets, along with a nice gravitational potential, an
a good switch to direct packets by. If this is IB, you should be able to see
quite a bit higher, though your PCI is going to limit you. PCI-e is better
(and HTX is *awesome*).
This is either 10Gb/s or three or four channel-bonded Gb/s links. This
should give substantially faster than 100MB/sec.
Note that even PCI-X is 133 MHz x 8 bytes = 1064 MB/sec.
FWIW, we have customers with units we have built out that happily support
2-400 MB/s over NFS without complaining, over gigabit (multiple simultaneous
clients hammering on the server). There are multiple problems to overcome to
get this working correctly and efficiently.
I'd be interested to hear about the details of this server, on list or off
list.
Cheers,
Bruce
_______________________________________________
Beowulf mailing list, [email protected]
To change your subscription (digest mode or unsubscribe) visit
http://www.beowulf.org/mailman/listinfo/beowulf
