On Thu, Jan 05, 2012 at 08:37:38PM -0600, Sebastian Good wrote: > We would like to be able to do > parallel reads or writes, from multiple mpi ranks, but want these ranks to > be on different nodes, with each node able to utilize its own local disk. > That is, instead of, say 50TB of data being in a single file on a network > file system, we might want to arrange for ~3TB to be on each of 20 nodes' > local file systems.
You need to be careful here. You are asking for both a distributed file system and a specific set of consistency semantics. You can get all kinds of distributed file systems but they will not provide the sort of consistency you require. Install something like PVFS on these nodes, if you really want to use node-local storage this way. Make each node both a client and a server in the PVFS environment. Presto: a fairly large file system volume with the kinds of consistency semantics you require for a parallel I/O workload. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
