Oh? I specifically remember Sven talking about the >32 subblocks on the context of file creation speed in addition to space efficiency. If what you’re saying is true, then why do those charts show that feature in the context of file creation performance and specifically mention it as a performance bottleneck? Are the slides incorrect or am I just reading them wrong?
Sent from my iPhone > On Nov 30, 2017, at 10:05, Lyle Gayne <lga...@us.ibm.com> wrote: > > Aaron, > that is a misunderstanding. The new feature for larger numbers of sub-blocks > (varying by block size) has nothing to do with the 50K creates per second or > many other performance patterns in GPFS. > > The improved create (and other metadata ops) rates came from identifying and > mitigating various locking bottlenecks and optimizing the code paths > specifically involved in those ops. > > Thanks > Lyle > > > <graycol.gif>Aaron Knister ---11/29/2017 05:42:26 PM---Thanks, Nikhil. Most > of that was consistent with my understnading, however I was under the > impressio > > From: Aaron Knister <aaron.knis...@gmail.com> > To: gpfsug main discussion list <gpfsug-discuss@spectrumscale.org> > Date: 11/29/2017 05:42 PM > Subject: Re: [gpfsug-discuss] Online data migration tool > Sent by: gpfsug-discuss-boun...@spectrumscale.org > > > > > Thanks, Nikhil. Most of that was consistent with my understnading, however I > was under the impression that the >32 subblocks code is required to achieve > the touted 50k file creates/second that Sven has talked about a bunch of > times: > > http://files.gpfsug.org/presentations/2017/Manchester/08_Research_Topics.pdf > http://files.gpfsug.org/presentations/2017/Ehningen/31_-_SSUG17DE_-_Sven_Oehme_-_News_from_Research.pdf > http://files.gpfsug.org/presentations/2016/SC16/12_-_Sven_Oehme_Dean_Hildebrand_-_News_from_IBM_Research.pdf > > from those presentations regarding 32 subblocks: > > "It has a significant performance penalty for small files in large block size > filesystems" > > although I'm not clear on the specific definition of "large". Many > filesystems I encounter only have a 1M block size so it may not matter there, > although that same presentation clearly shows the benefit of larger block > sizes which is yet *another* thing for which a migration tool would be > helpful. > > -Aaron > > > On Wed, Nov 29, 2017 at 2:08 PM, Nikhil Khandelwal <nikh...@us.ibm.com> wrote: > Hi, > > I would like to clarify migration path to 5.0.0 from 4.X.X clusters. For all > Spectrum Scale clusters that are currently at 4.X.X, it is possible to > migrate to 5.0.0 with no offline data migration and no need to move data. > Once these clusters are at 5.0.0, they will benefit from the performance > improvements, new features (such as file audit logging), and various > enhancements that are included in 5.0.0. > > That being said, there is one enhancement that will not be applied to these > clusters, and that is the increased number of sub-blocks per block for small > file allocation. This means that for file systems with a large block size and > a lot of small files, the overall space utilization will be the same it > currently is in 4.X.X. Since file systems created at 4.X.X and earlier used a > block size that kept this allocation in mind, there should be very little > impact on existing file systems. > > Outside of that one particular function, the remainder of the performance > improvements, metadata improvements, updated compatibility, new > functionality, and all of the other enhancements will be immediately > available to you once you complete the upgrade to 5.0.0 -- with no need to > reformat, move data, or take your data offline. > > I hope that clarifies things a little and makes the upgrade path more > accessible. > > Please let me know if there are any other questions or concerns. > > Thank you, > Nikhil Khandelwal > Spectrum Scale Development > Client Adoption > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=IbxtjdkPAM2Sbon4Lbbi4w&m=irBRNHjLNBazoPW27vuMTJGyZjdo_8yqZZNkY7RRh5I&s=8nZVi2Wp8LPbXo0Pg6ItJv6GEOk5jINHR05MY_H7a4w&e= > > >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
