On 2012-02-09, at 6:20 AM, Jack David wrote: > In the output of "lsf getstripe <filename> | <dirname>", the obdidx > denotes the OST index (I assume). > > Consider the following output: > > lmm_stripe_count: 2 > lmm_stripe_size: 1048576 > lmm_stripe_offset: 1 > obdidx objid objid group > 1 2 0x2 0 > 0 3 0x3 0 > > where I have a setup consisting of two OSTs. If I have more than two > OSTs, is it possible that I get the obdidx values out of order? Or the > obdidx values will always be linear? > > For example, in above output, the values are linear (like 1, 0 - and > this pattern will be repeated while storing the data I assume). If I > have 4 OSTs, can the values be non-linear? Something like 2,0,1,3 or > 2,1,3,0 (or any pattern for that matter)??
Typically the ordering will be linear, but this depends on a number of different factors: - what order the OSTs were created in: without --index=N the OST order depends on the order in which they were first mounted, so using --index is always recommended, and will be mandatory in the future - the distribution of OSTs among OSS nodes: the MDS object allocator will normally select one OST from each OSS before allocating another object from a different OST on the same OSS - the space available on each OST: when OST free space is imbalanced the OSTs will be selected in part based on how full they are > My assumption on how the data is stored on OSTs: > Based upon the values of obdidx, each OST will store a stripe_size > worth data into the objid (a file under ldiskfs volume of that OST) in > rotation. So if I get the obdidx like 2,1,3,0 and stripe_size if 1MB, > then the data will be stored in following order: > > 1st MB: 2nd OST > 2nd MB: 1st OST > 3rdMB: 3rd OST > 4thMB: 0th OST > 5th MB: 2nd OST (Again - repeating the pattern) > 6th MB: 1st OST > > Is this understanding correct?? I hope I am clear on my question. Correct. The data is strictly round-robin on the objects once they are allocated to a file. Cheers, Andreas -- Andreas Dilger Whamcloud, Inc. Principal Engineer http://www.whamcloud.com/ _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
