The PVFS system interface allows the distribution to be set on a per-file basis, the issue is doing it thru the MPI or the kernel interface. There was/is a patch for ROMIO that allows the distribution to be set thru MPI_INFO hints, to my knowledge that version of ROMIO never has been released (Julian Kunkel's offered the patch I believe). I also have a patch for setting the distribution that worked against Avery's ROMIO branch, but that too is still unreleased I believe.
Cheers, Brad On Fri, Jan 23, 2009 at 3:09 PM, Kyle Schochenmaier <[email protected]> wrote: > FSN - > > You are spot-on with your assesment of the striping in PVFS2. > Each file gets a round-robin striping based on the distribution > parameters of its directory. > To my knowledge, you cannot set things on a per-file basis, as the > distribution is done on a per-directory basis. > If I'm wrong, everyone please feel free to flame me ;-) > > I'm not sure why you would want to base your stripe size on the size > of the files, however, my guess is that you are trying to create a > 'balanced load.' > If this is the case, you can pick a generally good stripe size for the > majority of your files, though the defaults seem to work well enough. > Then pvfs2 will do the load balancing on its own. > The way that pvfs2 does its striping is to have the server > representing the 0th block of each file move in a random/round-robin > order. > So this spreads the file data load relatively evenly across all > servers in the fs. > > The only things you should worry about here is picking a stripe size > that is excessively small (< 4k?) as it will take with it a fairly > large overhead/performance hit on the servers as well as the network. > AND > If you are worried about disk space, try not to set it too big. I've > used 4MB stripes before and not had problems if dealing with large > files, however the 64KB default works pretty well. > We can always try to help tune/tweak things for performance if you > identify slowness. > > If you're trying to do something else with the stripe size > modifications, feel free to ignore the previous comments. > > ~Kyle > > Kyle Schochenmaier > > > > On Fri, Jan 23, 2009 at 12:13 PM, FileSystem Novice > <[email protected]> wrote: >> Hi, >> >> While trying to explore the PVFS2 striping features using the 'setfattr' and >> distribution parameters (like type of stripe, stripe-size, etc), the >> examples I came across mostly deal with setting the striping distribution >> parameters (simple-stripe, etc) on a directory basis. So if we set the >> distribution parameters for a directory, all files created on that directory >> (after the striping distribution parameters have been set) will be striped >> identically. >> >> But let's say I want to stripe a file A in directory X using a simple-stripe >> and strip length of 64KB and want to stripe a different file B in the same >> directory X using a simple-stripe and strip length of 5KB, etc (since each >> data file may have different record lengths, I would like to stripe them >> based on a multiple of the record lenght)... would I achieve this by setting >> each file's extended attributes with the desired values ? >> Is it possible to do striping in PVFS2 on a per-file basis as mentioned >> above and if so, are there links to any documents / examples out there? >> >> Thanks a lot for your help in advance, >> Novice >> >> >> _______________________________________________ >> Pvfs2-users mailing list >> [email protected] >> http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users >> >> > _______________________________________________ > Pvfs2-users mailing list > [email protected] > http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users > _______________________________________________ Pvfs2-users mailing list [email protected] http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
