On Tue, Feb 14, 2006 at 02:42:21PM -0800, Harry Mangalam wrote: > We are building a small test cluster (5 nodes of 2x2dual-core opterons) for > doing some parallel file system research (probably mostly with pvfs2) and I'm > trying to gather info on a cost/time optimal setup. The cluster will also be > used for computation in atmospheric data reduction of fairly large files > (hence the interest in using a pfs). Most of the applications will be using > MPI-IO at some level (HDF5's parallel access, parallel netCDF). > It seems that more nodes will be better for I/O up to the point of saturation > of the backplane and/or interconnect. > > For the interconnect technology, the lower-latency technologies such as > myrinet and IB are preferred, but is it possible to bond ethernets (initially > 1GE, then later 10GE) to provide better thruput? Is it advisable to try this > or will the latency end up killing such an effort anyway?
If you channel bond you'll be looking at a max bandwidth of what, 200 MBytes/sec? You'll be hard-pressed to get that much disk performance, so you'll end up with a very unbalanced (network-heavy) solution. Unless you've got some high-end raid hardware on each server node, you'll only come close to 200 MBytes/sec if you consitently work from the cache. Possible, depending on your datatset. > Is there a downside to having IB on the motherboards anyway (besides the > cost)? I know nothing about IB hardware... ==rob -- Rob Latham Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF Argonne National Labs, IL USA B29D F333 664A 4280 315B _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
