Hi, Yes, you could use "mmdf" to obtain file-system "usage" across the NSDs (comprising the file-system).
If you want to obtain "data block distribution corresponding to a file across the NSDs", then there is a utility "mmgetlocation" in /usr/lpp/mmfs/samples/fpo that can be used to get file-data-blocks to NSD mapping. Example: # File-system comprises of single storage pool, all NSDs configured as dataAndMetadata, -m 1 -r 1, FS block-size=2MiB # mmlsfs gpfs1b | grep 'Block size' -B 2097152 Block size # The file-system is comprised of 10 x dataAndMetadata NSDs # mmlsdisk gpfs1b | grep DMD | wc -l 10 # Create a sample file that is 40MiB (20 data blocks) /mnt/sw/benchmarks/gpfsperf/gpfsperf create seq -r 2m -n 40m /mnt/gpfs1b/temp_dir/lf.s.1 # File size is 40 MiB (09:52:49) c25m3n07:~ # ls -lh /mnt/gpfs1b/temp_dir/lf.s.1 -rw-r--r-- 1 root root 40M Mar 17 09:52 /mnt/gpfs1b/temp_dir/lf.s.1 (09:52:54) c25m3n07:~ # du -sh /mnt/gpfs1b/temp_dir/lf.s.1 40M /mnt/gpfs1b/temp_dir/lf.s.1 # Verified through mmgetlocation that the file data blocks is uniformly striped across all the dataAndMetadata NSDs, with each NSD containing 2 file data blocks # In the output below, "DMD_NSDX" is name of the NSDs. (09:53:00) c25m3n07:~ # /usr/lpp/mmfs/samples/fpo/mmgetlocation -f /mnt/gpfs1b/temp_dir/lf.s.1 [FILE INFO] ------------------------------------------------------------------------ blockSize 2 MB blockGroupFactor 1 metadataBlockSize 2M writeAffinityDepth 0 flags: data replication: 1 max 2 storage pool name: system metadata replication: 1 max 2 Chunk 0 (offset 0) is located at disks: [ DMD_NSD09 c25m3n07-ib,c25m3n08-ib ] Chunk 1 (offset 2097152) is located at disks: [ DMD_NSD10 c25m3n08-ib,c25m3n07-ib ] Chunk 2 (offset 4194304) is located at disks: [ DMD_NSD01 c25m3n07-ib,c25m3n08-ib ] Chunk 3 (offset 6291456) is located at disks: [ DMD_NSD02 c25m3n08-ib,c25m3n07-ib ] Chunk 4 (offset 8388608) is located at disks: [ DMD_NSD03 c25m3n07-ib,c25m3n08-ib ] Chunk 5 (offset 10485760) is located at disks: [ DMD_NSD04 c25m3n08-ib,c25m3n07-ib ] Chunk 6 (offset 12582912) is located at disks: [ DMD_NSD05 c25m3n07-ib,c25m3n08-ib ] Chunk 7 (offset 14680064) is located at disks: [ DMD_NSD06 c25m3n08-ib,c25m3n07-ib ] Chunk 8 (offset 16777216) is located at disks: [ DMD_NSD07 c25m3n07-ib,c25m3n08-ib ] Chunk 9 (offset 18874368) is located at disks: [ DMD_NSD08 c25m3n08-ib,c25m3n07-ib ] Chunk 10 (offset 20971520) is located at disks: [ DMD_NSD09 c25m3n07-ib,c25m3n08-ib ] Chunk 11 (offset 23068672) is located at disks: [ DMD_NSD10 c25m3n08-ib,c25m3n07-ib ] Chunk 12 (offset 25165824) is located at disks: [ DMD_NSD01 c25m3n07-ib,c25m3n08-ib ] Chunk 13 (offset 27262976) is located at disks: [ DMD_NSD02 c25m3n08-ib,c25m3n07-ib ] Chunk 14 (offset 29360128) is located at disks: [ DMD_NSD03 c25m3n07-ib,c25m3n08-ib ] Chunk 15 (offset 31457280) is located at disks: [ DMD_NSD04 c25m3n08-ib,c25m3n07-ib ] Chunk 16 (offset 33554432) is located at disks: [ DMD_NSD05 c25m3n07-ib,c25m3n08-ib ] Chunk 17 (offset 35651584) is located at disks: [ DMD_NSD06 c25m3n08-ib,c25m3n07-ib ] Chunk 18 (offset 37748736) is located at disks: [ DMD_NSD07 c25m3n07-ib,c25m3n08-ib ] Chunk 19 (offset 39845888) is located at disks: [ DMD_NSD08 c25m3n08-ib,c25m3n07-ib ] [SUMMARY INFO] ---------------------------------------------------------------------------------------------------------- Replica num Nodename TotalChunkst Replica 1 : c25m3n07-ib,c25m3n08-ib: Total : 10 Replica 1 : c25m3n08-ib,c25m3n07-ib: Total : 10 Best Regards, -Kums From: <greg.lehm...@csiro.au> To: <gpfsug-discuss@spectrumscale.org> Date: 03/29/2017 08:00 PM Subject: Re: [gpfsug-discuss] question on viewing block distribution across NSDs Sent by: gpfsug-discuss-boun...@spectrumscale.org I was going to keep mmdf in mind, not gpfs.snap. I will now also keep in mind that mmdf can have an impact as at present we have spinning disk for metadata. The system I am playing around on is not production yet, so I am safe for the moment. Thanks again. From: gpfsug-discuss-boun...@spectrumscale.org [ mailto:gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP] Sent: Thursday, 30 March 2017 9:55 AM To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] question on viewing block distribution across NSDs I don't necessarily think you need to run a snap prior, just the output of mmdf should be enough. Something to keep in mind that I should have said before-- an mmdf can be stressful on your system particularly if you have spinning disk for your metadata. We're fortunate enough to have all flash for our metadata and I tend to take it for granted some times :) From: greg.lehm...@csiro.au Sent: 3/29/17, 19:52 To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] question on viewing block distribution across NSDs Thanks. I don’t have a snap. I’ll keep that in mind for next time I do this. From: mailto:gpfsug-discuss-boun...@spectrumscale.org] On Behalf Of Knister, Aaron S. (GSFC-606.2)[COMPUTER SCIENCE CORP] Sent: Thursday, 30 March 2017 9:45 AM To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> Subject: Re: [gpfsug-discuss] question on viewing block distribution across NSDs Hi Greg, You could run an mmdf which will show you how full each NSD is. I'm not sure how to look back in time though to see the fs before the restripe. Do you perhaps have a gpfs.snap you took somewhat recently before the restripe? Maybe an internaldump in /tmp/mmfs somewhere? From: greg.lehm...@csiro.au Sent: 3/29/17, 19:21 To: gpfsug main discussion list Subject: [gpfsug-discuss] question on viewing block distribution across NSDs Hi All, I added some NSDs to an existing filesystem and ran mmrestripefs. I was sort of curious to see what the distribution looked like before and after the restripe. Is there any way of looking at it? Cheers, Greg._______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss