> "It has a significant performance penalty for small files in large > block size filesystems"
Aaron, Below are mdtest results for a test we ran for CORAL - file size was 32k. We have not gone back and ran the test on a file system formatted without > 32 subblocks. We'll do that at some point... -Bill -- started at 10/28/2017 17:51:38 -- mdtest-1.9.3 was launched with 228 total task(s) on 12 node(s) Command line used: /tmp/mdtest-binary-dir/mdtest -d /ibm/fs2-16m-10/mdtest-60000 -i 3 -n 294912 -w 32768 -C -F -r -p 360 -u -y Path: /ibm/fs2-16m-10 FS: 128.1 TiB Used FS: 0.3% Inodes: 476.8 Mi Used Inodes: 0.0% 228 tasks, 67239936 files SUMMARY: (of 3 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 51953.498 50558.517 51423.221 616.643 File stat : 0.000 0.000 0.000 0.000 File read : 0.000 0.000 0.000 0.000 File removal : 96746.376 92149.535 94658.774 1900.187 Tree creation : 1.588 0.070 0.599 0.700 Tree removal : 0.213 0.034 0.097 0.082 -- finished at 10/28/2017 19:51:54 -- Bill Hartner IBM Systems Scalable I/O Development Austin, Texas [email protected] home office 512-784-0980 [email protected] wrote on 11/29/2017 04:41:48 PM: > From: Aaron Knister <[email protected]> > To: gpfsug main discussion list <[email protected]> > Date: 11/29/2017 04:42 PM > Subject: Re: [gpfsug-discuss] Online data migration tool > Sent by: [email protected] > > Thanks, Nikhil. Most of that was consistent with my understnading, > however I was under the impression that the >32 subblocks code is > required to achieve the touted 50k file creates/second that Sven has > talked about a bunch of times: > > http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf > http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_- > _Sven_Oehme_-_News_from_Research.pdf > http://files.gpfsug.org/presentations/2016/SC16/12_- > _Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf > from those presentations regarding 32 subblocks: > > "It has a significant performance penalty for small files in large > block size filesystems" > although I'm not clear on the specific definition of "large". Many > filesystems I encounter only have a 1M block size so it may not > matter there, although that same presentation clearly shows the > benefit of larger block sizes which is yet *another* thing for which > a migration tool would be helpful. > -Aaron > > On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <[email protected]> wrote: > Hi, > > I would like to clarify migration path to 5.0.0 from 4.X.X clusters. > For all Spectrum Scale clusters that are currently at 4.X.X, it is > possible to migrate to 5.0.0 with no offline data migration and no > need to move data. Once these clusters are at 5.0.0, they will > benefit from the performance improvements, new features (such as > file audit logging), and various enhancements that are included in 5.0.0. > > That being said, there is one enhancement that will not be applied > to these clusters, and that is the increased number of sub-blocks > per block for small file allocation. This means that for file > systems with a large block size and a lot of small files, the > overall space utilization will be the same it currently is in 4.X.X. > Since file systems created at 4.X.X and earlier used a block size > that kept this allocation in mind, there should be very little > impact on existing file systems. > > Outside of that one particular function, the remainder of the > performance improvements, metadata improvements, updated > compatibility, new functionality, and all of the other enhancements > will be immediately available to you once you complete the upgrade > to 5.0.0 -- with no need to reformat, move data, or take your data offline. > > I hope that clarifies things a little and makes the upgrade path > more accessible. > > Please let me know if there are any other questions or concerns. > > Thank you, > Nikhil Khandelwal > Spectrum Scale Development > Client Adoption > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url? > u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx- > siA1ZOg&r=Ew59QH6nxuyx6oTs7a8AYX7kKG3gaWUGDGo5ZZr3wQ4&m=DHoqgBeMFgcM0LpXEI0VCYvvb8ollct5aSYUDln2t68&s=iOxGm-853L_W0XkB3jGsGzCTVlSYUvANOTSewcR_Ue8&e=
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
