thats not how GPFS aehm Scale works :-) each client has pre-allocated inodes in memory and creating files is a matter of spooling records. yes, eventually you need to destage this to the disk, but that happens only every few seconds and given this i/os are usually very colocated so good storage cache technology can reduce i/os to physical media significant.
to proof the point look at this numbers : -- started at 10/17/2017 14:29:13 -- mdtest-1.9.3 was launched with 110 total task(s) on 11 node(s) Command line used: /ghome/oehmes/mpi/bin/mdtest-pcmpi9131-existingdir -d /ibm/fs2-16m-09/shared/mdtest-ec -i 1 -n 10000 -F -w 0 -Z -p 8 -N 11 -u Path: /ibm/fs2-16m-09/shared FS: 128.1 TiB Used FS: 0.2% Inodes: 476.8 Mi Used Inodes: 0.0% 110 tasks, 1100000 files SUMMARY: (of 1 iterations) Operation Max Min Mean Std Dev --------- --- --- ---- ------- File creation : 444221.343 444221.343 444221.343 0.000 File stat : 6704498.841 6704498.841 6704498.841 0.000 File read : 3859105.596 3859105.596 3859105.596 0.000 File removal : 409336.606 409336.606 409336.606 0.000 Tree creation : 5.344 5.344 5.344 0.000 Tree removal : 1.145 1.145 1.145 0.000 -- finished at 10/17/2017 14:29:27 -- this is a run against a 16mb blocksize filesystem with only spinning disks (just one GL6 ESS) , not a single SSD and as you can see , this system on 11 nodes produces 444k creates / second something far above and beyond of what drives can do. and yes i know this stuff is all very complicated and not easy to explain :-) sven On Thu, Dec 21, 2017 at 8:35 PM <[email protected]> wrote: > On Thu, 21 Dec 2017 16:38:27 +0000, Sven Oehme said: > > > size. so if you now go to a 16MB blocksize and you have just 50 iops @ > 2MB > > each you can write ~800 MB/sec with the exact same setup and same size > > small writes, that's a factor of 8 . > > That's assuming your metadata storage is able to handle > open/read/write/close > on enough small files per second to push 800MB/sec. If you're talking > 128K subblocks, > you're going to need some 6,400 small files per second to fill that pipe... > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
