On Wed, Jul 24, 2013 at 11:50:17PM +0200, Matthieu Dorier wrote: > Hi, > > > I'm benchmarking a 12-node PVFS installation right now using an MPI program > that generates various strided patterns supposed to stress the file system in > different ways and to trigger the collective buffering in Mpich. I'm using > 576 processes for that (24 24-core nodes). I was expecting the collective > buffering to perform better than having each process issue an independent > strided write, but it seems that disabling collective buffering leads to > better performance. > > > I'd be curious how PVFS2 optimizes its independent strided operations to have > such good performance?
list-io! The ROMIO pvfs driver will take non-contiguous requests and stitch them up into PVFS datatype I/O (except not full-fledged datatypes. I implemented list-io and left it at that). Collective buffering will be a good optimization, but there is overhead when exchanging the accesses and moving data to/from the aggregator processes. What's the whole point of collective I/O? to turn many small requests into one larger request. Well, list-io, describes the I/O with one PVFS_sys_io command (containing up to 64 pieces), effectively doing the same thing but without some of the overhead. This paper discusses the optimization in pvfs (v1, not pvfs v2, but the ideas are the same): http://dl.acm.org/citation.cfm?id=793304 And this paper discuses using that optimization in an MPI-IO driver: http://dl.acm.org/citation.cfm?id=791231.792478 (which we did for both pvfs v1 and v2) ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
