Andreas Dilger wrote: > On Mar 06, 2009 07:26 -0600, Nirmal Seenu wrote: > >> While trying to figure out the reason for LVM2 snapshots failing on our >> MDT server I found that there are a lot of sparse files on the MDT >> volume. The file size as seen from a ls command output on the MDT is >> same as the real file size. The tar runs for a few hours at this point >> even if try to use the --spare option in the tar command. >> > > All of the files on the MDT are sparse. The data lives on the OSTs. > > >> The total MDT partition usage itself is about 500MB (as reported by df) >> and it used to take me less than 10 minutes to create a LVM2 snapshot >> and tar it up when I was running the servers using Lustre 1.6.5 with no >> quota enabled. >> >> I recently upgraded my Lustre servers to 1.6.7 and tried to enable quota >> on the MDT and OST by doing the following commands: >> > > Also, the quota file is a huge sparse file, size proportional to the > highest UID in use. > Quota files are sparse files, but they aren't *only* size proportional to the highest UID in use. It has some relationship with distribution of uid/gid. For example, 1. if the system only has one user whose uid=1 or uid=33554432, sizes of user quota file will be same. 2. if the system only has two users whose uid=(1, 2) or uid=(1, 33554432), the latter will take three more blocks. This thing is quite like ext3's indirect blocks.
BTW, there are two kinds of quota files in lustre: 1. admin quota files(only exists on mdt) 2. operation quota files(exist on mdt and osts) For 1, its size is relative to the number of users who have quota limitation; for 2, its size is relative to the number of users who are in use in filesystem. Landen _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
