Hello Kums, Thank you; I could verify that the data of small files is in the inode. In the table below you see the filesize and the result of the tsdbfs query. Up to 3k all data is in the inode.
The result of stat calls on small files is very puzzling. When data is in the inode stat reports one block of 512 bytes used, even for 3k of data. I don’t see how this would affect any of our applications, so it’s just something to note. I append some output and the script to generate it, just for completeness. Cheers, Heiner # stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel 0 indirectionLevel=DIRECT status=USERFILE. <empty file is special> 1 indirectionLevel=INODE status=USERFILE 16 indirectionLevel=INODE status=USERFILE 512 indirectionLevel=INODE status=USERFILE 1k indirectionLevel=INODE status=USERFILE 2k indirectionLevel=INODE status=USERFILE 3k indirectionLevel=INODE status=USERFILE. < up to 3k data is in inode > 4k indirectionLevel=DIRECT status=USERFILE 16k indirectionLevel=DIRECT status=USERFILE 64k indirectionLevel=DIRECT status=USERFILE 1M indirectionLevel=DIRECT status=USERFILE 2M indirectionLevel=DIRECT status=USERFILE Stat output # stat -c “%n size: %s allocated: %b*%B” # stat -c %n size: %s allocated: %b*%B 0 size: 0 allocated: 0*512 1 size: 1 allocated: 1*512 16 size: 16 allocated: 1*512 512 size: 512 allocated: 1*512 1k size: 1024 allocated: 1*512 2k size: 2048 allocated: 1*512 3k size: 3072 allocated: 1*512 < 3k file and all data in inode: stat reports 1*512 allocated/used) 4k size: 4096 allocated: 64*512 16k size: 16384 allocated: 64*512. < as expected, 32k subblock size > 64k size: 65536 allocated: 128*512 1M size: 1048576 allocated: 2048*512 2M size: 2097152 allocated: 4096*512 The script: # test-data-in-inode.sh sizes="0 1 16 512 1k 2k 3k 4k 16k 64k 1M 2M" echo create files for s in $sizes do head -c $s /dev/zero > $s done echo sleep 20. # give gpfs some time to update metadata sleep 20 echo echo "# ls -ls $sizes" ls -ls $sizes echo echo "# stat -c %n size: %s allocated: %b*%B" stat -c "%n size: %s allocated: %b*%B" $sizes echo echo '# stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel' for f in $sizes do echo -n $f stat -c "inode %i" $f | tsdbfs fs1301 | grep indirectionLevel done From: <[email protected]> on behalf of Kumaran Rajaram <[email protected]> Reply to: gpfsug main discussion list <[email protected]> Date: Wednesday, 17 July 2019 at 14:38 To: gpfsug main discussion list <[email protected]> Subject: Re: [gpfsug-discuss] How to prove that data is in inode Hi, >> How can I prove that data of a small file is stored in the inode (and not on >> a data nsd)? You may use echo "inode file_inode_number" | tsdbfs fs_device | grep indirectionLevel and if it points to INODE, then the file is stored in the inodes # 4K Inode Size # mmlsfs gpfs3a | grep 'Inode size' -i 4096 Inode size in bytes # Small file # ls -l /mnt/gpfs3a/hello.txt -rw-r--r-- 1 root root 6 Jul 17 08:32 /mnt/gpfs3a/hello.txt # ls -i /mnt/gpfs3a/hello.txt 91649 /mnt/gpfs3a/hello.txt #File is inlined within Inode # echo "inode 91649" | tsdbfs gpfs3a | grep indirectionLevel indirectionLevel=INODE status=USERFILE Regards, -Kums [Inactive hide details for "Billich Heinrich Rainer (ID SD)" ---07/17/2019 07:49:56 AM---Hello, How can I prove that data of a]"Billich Heinrich Rainer (ID SD)" ---07/17/2019 07:49:56 AM---Hello, How can I prove that data of a small file is stored in the inode (and not on a data nsd)? From: "Billich Heinrich Rainer (ID SD)" <[email protected]> To: gpfsug main discussion list <[email protected]> Date: 07/17/2019 07:49 AM Subject: [EXTERNAL] [gpfsug-discuss] How to prove that data is in inode Sent by: [email protected] ________________________________ Hello, How can I prove that data of a small file is stored in the inode (and not on a data nsd)? We have a filesystem with 4k inodes on Scale 5.0.2 , but it seems there is no file data in the inodes? I would expect that 'stat' reports 'Blocks: 0' for a small file, but I see 'Blocks:1'. Cheers, Heiner I tried []# rm -f test; echo hello > test []# ls -ls test 1 -rw-r--r-- 1 root root 6 Jul 17 13:11 test [root@testnas13ems01 test]# stat test File: ‘test’ Size: 6 Blocks: 1 IO Block: 1048576 regular file Device: 2dh/45d Inode: 353314 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2019-07-17 13:11:03.037049000 +0200 Modify: 2019-07-17 13:11:03.037331000 +0200 Change: 2019-07-17 13:11:03.037259319 +0200 Birth: - [root@testnas13ems01 test]# du test 1 test [root@testnas13ems01 test]# du -b test 6 test [root@testnas13ems01 test]# Filesystem # mmlsfs f**** flag value description ------------------- ------------------------ ----------------------------------- -f 32768 Minimum fragment (subblock) size in bytes -i 4096 Inode size in bytes -I 32768 Indirect block size in bytes -m 1 Default number of metadata replicas -M 2 Maximum number of metadata replicas -r 1 Default number of data replicas -R 2 Maximum number of data replicas -j cluster Block allocation type -D nfs4 File locking semantics in effect -k nfs4 ACL semantics in effect -n 32 Estimated number of nodes that will mount file system -B 1048576 Block size -Q user;group;fileset Quotas accounting enabled user;group;fileset Quotas enforced user;group;fileset Default quotas enabled --perfileset-quota Yes Per-fileset quota enforcement --filesetdf Yes Fileset df enabled? -V 20.01 (5.0.2.0) Current file system version 15.01 (4.2.0.0) Original file system version --create-time ***** 2017 File system creation time -z No Is DMAPI enabled? -L 33554432 Logfile size -E Yes Exact mtime mount option -S relatime Suppress atime mount option -K whenpossible Strict replica allocation option --fastea Yes Fast external attributes enabled? --encryption No Encryption enabled? --inode-limit 1294592 Maximum number of inodes in all inode spaces --log-replicas 0 Number of log replicas --is4KAligned Yes is4KAligned? --rapid-repair Yes rapidRepair enabled? --write-cache-threshold 0 HAWC Threshold (max 65536) --subblocks-per-full-block 32 Number of subblocks per full block -P system;data Disk storage pools in file system --file-audit-log No File Audit Logging enabled? --maintenance-mode No Maintenance Mode enabled? -d ****** -A yes Automatic mount option -o nfssync,nodev Additional mount options -T /**** Default mount point --mount-priority 0 Mount priority -- ======================= Heinrich Billich ETH Zürich Informatikdienste Tel.: +41 44 632 72 56 [email protected] ======================== _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
