Dear All, We are running a lustre-1.6.6 on Linux vanilla kernel 2.6.22.19. Recently we suffer a serious un-balanced OSTs problem. If I run the command:
# /opt/lustre-1.8/bin/lfs df UUID 1K-blocks Used Available Use% Mounted on cwarp-MDT0000_UUID 119627860 659184 112132220 0% /mnt/src[MDT:0] cwarp-OST0000_UUID 1441859112 1343925184 24691744 93% /mnt/src[OST:0] cwarp-OST0001_UUID 1441859128 799739136 568869616 55% /mnt/src[OST:1] cwarp-OST0002_UUID 1441859128 643666316 724950624 44% /mnt/src[OST:2] cwarp-OST0003_UUID 1441859112 745288308 623015556 51% /mnt/src[OST:3] cwarp-OST0004_UUID 1441859128 654020352 714567920 45% /mnt/src[OST:4] cwarp-OST0005_UUID 1441859128 658416232 709949996 45% /mnt/src[OST:5] filesystem summary: 8651154736 4845055528 3366045456 56% /mnt/src It is clear that the cwarp-OST0000_UUID is almost full, but the other OSTs are still quite empty. No matter what, we are trying to do something in order to re-balance the OSTs. Our procedure is (which is indicated in the Lustre manual): 1. In MDS, disable the cwarp-OST0000 so that newly created file will go to other OSTs: echo 0 > /proc/fs/lustre/osc/cwarp-OST0000-osc/active 2. In one of the client node, we copy and rename files, in hope that some files can be pull out of the cwarp-OST0000 and then go to other OSTs: cp /path/to/some/files /path/to/some/files.tmp mv /path/to/some/files.tmp /path/to/some/files However, this way seems does not help too much. I guess this is because we do not pull out the files which locate in cwarp-OST0000 exactly. Actually we are not sure whether this is the correct way to solve this problem. Any suggestion to re-balance or to prevent un-balanced OSTs are very appriciated. Best Regards, T.H.Hsieh _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
