On 26 Oct 2010, at 07:04, Andreas Dilger wrote:
> On 2010-10-26, at 05:36, Ashley Pittman wrote:
>> There is a lot of documentation out there to help in choosing the best inode
>> size for a filesystem however none that I can find about measuring the
>> actually used space in inodes on an existing filesystem. As an example I
>> have a existing Lustre filesystem where the MDT has a inode size is 512
>> bytes, the filesystem makes heavy use of ACLs however and MDT performance is
>> below what is expected - the suspicion is that we are overflowing the 512
>> allowance and having to do two lookups for each file access. On a related
>> note what happens when there is not enough space in the inode for all the
>> data, is a second inode used or is a block allocated?
>
> If you mount the MDT filesystem locally and run "ls -ls {file}", or "find
> -type f -ls" on the filesystem it will print out the blocks used for each
> inode. For regular files on the MDT, the number of blocks should always be
> 0, unless there is an xattr block.
>
> That will at least tell you whether the xattr data has exceeded the in-inode
> space, but will not necessarily tell you the actual number of bytes used by
> each xattr.
>
>> Is there a way to measure the space used by the inode for a given file, we
>> are looking at debugfs and the stat command which is giving us some data but
>> it's not clear how to parse this information and calculate the space used.
>> Some info is shown in the "Blockcount:" field in the stat output but this
>> increases in multiples of 8 and I only see a listing of blocks when it's
>> value is 16 or more. There are also the "File ACL" and "Directory ACL"
>> fields but these are not always non-zero even when ACLs are used?
>
> You could also do something like "getfattr -e hex -d -m acl.* {file}| wc -c"
> on each inode (command from memory, probably needs some correction), but I'm
> not 100% sure whether this would allow extracting the ACLs.
>
>> The specific FS I'm thinking about typically has 28 ACL on a file and there
>> are 30 OSTS but most files have a default stripe-count of 1,
>
> With 28 ACLs, this would be 32 + 4 + 8 * 28 = 260 bytes, which is _maybe_ too
> big for the in-inode xattr space. With a 512-byte inode, 128 bytes are used
> for the standard inode data, 28 bytes for the extra inode data (nanosecond
> timestamps, etc) and 32 + 32 + 24 = 88 bytes for the LOV EA, leaving 268
> bytes for other data. Numbers might be slightly off (depending on alignment)
> so it may not be enough space for the ACL data.
>
>> Ideally I'd like a way of seeing a size-distrobution for all inodes in a
>> filesystem though so I could use this as the basis for choosing the inode
>> size.
Thank you, that's the kind of thing I was looking for. As we are seeing inodes
with a blockcount of more than zero I know we are overflowing but it doesn't
sound like it's by much so we'll move one step up the scale to 1024 rather than
two steps up which would have taken us to a somewhat spacious 2048 bytes.
Ashley.
--
Ashley Pittman, Bath, UK.
Padb - A parallel job inspection tool for cluster computing
http://padb.pittman.org.uk
_______________________________________________
Lustre-discuss mailing list
[email protected]
http://lists.lustre.org/mailman/listinfo/lustre-discuss
