Strangely (although I'm sure it's not related) I have seen the exact same behavior on my Lustre cluster in the last month or so. I have also never seen this before, and to the best of my knowledge there is no change in usage patterns.
I'm running 1.6.7.2 on the servers. Ron Jerome National Research Council Canada. -----Original Message----- From: [email protected] on behalf of Thomas Roth Sent: Fri 10/30/2009 2:07 PM To: [email protected] Subject: [Lustre-discuss] Bad distribution of files among OSTs Hi all, in our 196 OST - Cluster, the previously perfect distribution of files among the OSTs is not working anymore, since ~ 2 weeks. The filling for most OSTs is between 57% and 62%, but some (~10) have risen up to 94%. I'm trying to fix that by having these OSTs deactivated on the MDT and finding and migrating away data from them, but it seems I'm not fast enough and it's a ongoing problem - I've just deactivated another OST with threatening 67%. Our qos_prio_free is at the default 90%. Our OST's sizes are between 2.3TB and 4.5TB. We use striping level 1, so it would be possible to fill up an OST by just creating a 2TB file. However, I'm not aware of any such gigafiles (using robinhood to get a picture of our file system). In addition, our user's behavior should not have changed recently. In August, the entire cluster had filled up to almost 80% in a neatly even distribution among the OSTs, so we extended the cluster by more OSTs, migrating data to even the filling between old and new ones. This also succeeded, and up to October there was no indication of something not working. There are no error message in the logs that would point to some OSTs being favored ;-) So, what could be the cause of this misdistribution? Regards, Thomas _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
_______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
