Just to clarify - its 2M block size, so 64k subblock size. Regards,
Tomer Perry Scalable I/O Development (Spectrum Scale) email: [email protected] 1 Azrieli Center, Tel Aviv 67021, Israel Global Tel: +1 720 3422758 Israel Tel: +972 3 9188625 Mobile: +972 52 2554625 From: "Tomer Perry" <[email protected]> To: gpfsug main discussion list <[email protected]> Date: 10/04/2019 23:11 Subject: Re: [gpfsug-discuss] Follow-up: ESS File systems Sent by: [email protected] Its also important to look into the actual space "wasted" by the "subblock mismatch". For example, a snip from a filehist output I've found somewhere: File%ile represents the cummulative percentage of files. Space%ile represents the cummulative percentage of total space used. AvlSpc%ile represents the cummulative percentage used of total available space. Histogram of files <= one 2M block in size Subblocks Count File%ile Space%ile AvlSpc%ile --------- -------- ---------- ---------- ---------- 0 1297314 2.65% 0.00% 0.00% 1 34014892 72.11% 0.74% 0.59% 2 2217365 76.64% 0.84% 0.67% 3 1967998 80.66% 0.96% 0.77% 4 798170 82.29% 1.03% 0.83% 5 1518258 85.39% 1.20% 0.96% 6 581539 86.58% 1.27% 1.02% 7 659969 87.93% 1.37% 1.10% 8 1178798 90.33% 1.58% 1.27% 9 189220 90.72% 1.62% 1.30% 10 130197 90.98% 1.64% 1.32% So, 72% of the files are smaller then 1 subblock ( 2M in the above case BTW). If, for example, we'll double it - we will "waste" ~76% of the files, and if we'll push it to 16M it will be ~90% of the files... But, we really care about capacity, right? So, going into the 16M extreme, we'll "waste" 1.58% of the capacity ( worst case of course). So, if it will give you ( highly depends on the workload of course) 4X the performance ( just for the sake of discussion) - will it be OK to pay the 1.5% "premium" ? Regards, Tomer Perry Scalable I/O Development (Spectrum Scale) email: [email protected] 1 Azrieli Center, Tel Aviv 67021, Israel Global Tel: +1 720 3422758 Israel Tel: +972 3 9188625 Mobile: +972 52 2554625 From: "Marc A Kaplan" <[email protected]> To: gpfsug main discussion list <[email protected]> Date: 10/04/2019 20:57 Subject: Re: [gpfsug-discuss] Follow-up: ESS File systems Sent by: [email protected] If you're into pondering some more tweaks: -i InodeSize is tunable system pool : --metadata-block-size is tunable separately from -B blocksize On ESS you might want to use different block size and error correcting codes for (v)disks that hold system pool. Generally I think you'd want to set up system pool for best performance for relatively short reads and updates. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=mLPyKeOa1gNDrORvEXBgMw&m=qbhRxpvXiJPC72GAztszQ27LP3W7o1nmJYNV1rP2k2U&s=T5j2wkoj3NuxnK-RAMPlSc9vYHIViTOe8hGF68u5VsU&e=
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
