Greetings, I am looking for more information regarding the "size on MDS" feature as it exists for lustre-1.8.x. Testing on our system (which started out as 1.6.6 and is now 1.8.x) indicates that there are many files which do not have the size information stored on the MDT. So, my basic question: under what conditions will the "size hint" attribute be updated? Is there any way to force the MDT to query the OSTs and update it's information?
More details and background information... We are currently running: Client: CentOS-5.5, kernel-2.6.18-194.11.4.el5, lustre-1.8.4 Server: CentOS-5.3, kernel-2.6.18-164.15.1.el5, lustre-1.8.3.ddn4.1 with no immediate plans to upgrade to lustre-2.x. I found the high level design here: http://wiki.lustre.org/images/9/94/Size_on_mds-hld.pdf but that seems to be targeted at 2.0.x for full implementation. I also discovered this old thread on the topic: http://lists.lustre.org/pipermail/lustre-discuss/2009-July/011149.html in which Andreas wrote "In Lustre 1.6.7 the approximate file size started to be stored on the MDT inodes..." and "This size is not actively updated for pre-existing files...". (It seems as though this is still true for 1.8.x.) The end goal of this is to facilitate efficient checks of disk usage on a per-directory basis (essentially we want "volume based quotas"). I'm hoping to run something once a day on the MDS like the following: lvcreate -s -p r -n mdt_snap /dev/mdt mount -t ldiskfs -o ro /dev/mdt_snap /mnt/snap cd /mnt/snap/ROOT du --apparent-size ./* > volume_usage.log cd / umount /mnt/snap lvremove /dev/mdt_snap Since the data is going to be up to one day old anyway, I don't really mind that the file size is "approximate", but it does have to be reasonably close. With the MDT LVM snapshot method I can check the whole 300TB file system in about 3 hours, whereas checking from a client takes weeks. Here is why I am relatively certain that the size-on-MDS attributes are not updated (lightly edited): [r...@mds0 ~]# ls -l /mnt/snap/ROOT/test/rollover/user_acct_file -rw-r--r-- 1 9999 9000 0 Mar 23 2010 /mnt/snap/ROOT/test/rollover/user_acct_file [r...@mds0 ~]# du /mnt/snap/ROOT/test/rollover/user_acct_file 0 /mnt/snap/ROOT/test/rollover/user_acct_file [r...@mds0 ~]# du --apparent-size /mnt/snap/ROOT/test/rollover/user_acct_file 0 /mnt/snap/ROOT/test/rollover/user_acct_file [r...@c448 ~]# ls -l /mnt/lfs0/test/rollover/user_acct_file -rw-r--r-- 1 user group 184435207 Mar 23 2010 /mnt/lfs0/test/rollover/user_acct_file [r...@c448 ~]# du /mnt/lfs0/test/rollover/user_acct_file 180120 /mnt/lfs0/test/rollover/user_acct_file [r...@c448 ~]# du --apparent-size /mnt/lfs0/test/rollover/user_acct_file 180113 /mnt/lfs0/test/rollover/user_acct_file Thanks very much for any answers or suggestions you can provide! -Nathan _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
