> Agreed. Just finished telling another vendor, "It's not high speed > storage unless it has an IB/RDMA interface". They love that. Except
what does RDMA have to do with anything? why would straight 10G ethernet not qualify? I suspect you're really saying that you want an efficient interface, as well as enough bandwidth, but that doesn't necessitate RDMA. > for some really edge cases, I can't imagine running IO over GbE for > anything more than trivial IO loads. well, it's a balance issue. if someone was using lots of Atom boards lashed into a cluster, 1Gb apiece might be pretty reasonable. but for fat nodes (let's say 48 cores), even 1 QDR IB pipe doesn't seem all that generous. as an interesting case in point, SeaMicro was in the news again with a 512 atom system: either 64 Gb links or 16 10G links. the former (.128 Gb/core) seems low even for atoms, but .3 Gb/core might be reasonable. > I am Curious if anyone is doing IO over IB to SRP targets or some > similar "Block Device" approach. The Integration into the filesystem by > Lustre/GPFS and others may be the best way to go, but we are not 100% > convinced yet. Any stories to share? you mean you _like_ block storage? how do you make a shared FS namespace out of it, manage locking, etc? regards, mark hahn. _______________________________________________ Beowulf mailing list, Beowulf@beowulf.org sponsored by Penguin Computing To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
