Justin, Do you have holes in your OST list?
Meaning, is this OST something like: lfs df for me shows: … lustre-OST001b_UUID 15378613900 8631157344 5966252368 59% /lustre[OST:27] lustre-OST0029_UUID 7689826700 5611663868 1687542044 77% /lustre[OST:41] … Note, there are no OST:28,29,… and no OST001c,1d,… I can lfs find files on lustre-OST001b_UUID but never on lustre-OST0029_UUID. I did some reconfiguring of OSTs a while back and holes got into my lustre system, and the OST after hole is never found with lfs find. lfs getstripe shows that the files are indeed there and on the OST indexed as 41. -mb On May 21, 2012, at 4:32 AM, Ross, Justin wrote: > I am stuck on an issue with an OST. I am in the process of replacing some > disks in the Lustre file system under a few OSTs. The process has been to > drain the OSTs by marking the OST inactive, lfs find, copy to a new name, > move to the old name. It has been working very well for all of the OSTs. I > have one particular OST that has about 4.5TB of data on it and lfs find > doesn't find any files on it at all. My assumption was that Lustre doesn't > know about any files on that OST. I assumed a full fsck is in order. But > the real weird catch is that just watching the OST it is in use. I have set > it inactive so no new files are written to it. There are a good bit of reads > coming from that OST. I watch the OSS with iostat and all of the other OSTs > on that system are idle, already drained. This one OST has plenty of read > activity. So it seems that Lustre does know about files on this OST, but > just not with lfs find... Any ideas how to find these files and move them? > I have mounted the OST with ldiskfs and everything looks ok. There are > plenty of files there. > > I have file striping set to one OST. I am running 1.8.5. I will upgrade to > 1.8.7 as soon as I can, but am curious if anyone else has seen anything like > this. I can't find any similar threads. > > Thanks, > Justin > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss -- +----------------------------------------------- | Michael Barnes | | Thomas Jefferson National Accelerator Facility | Scientific Computing Group | 12000 Jefferson Ave. | Newport News, VA 23606 | (757) 269-7634 +----------------------------------------------- _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
