I wanted to know, how does mmap interact with GPFS pagepool with respect to
Does the efficiency depend on the mmap read size and the block-size of the
filesystem even if all the data is cached in pagepool?
GPFS 22.214.171.124 and CentOS7.
Here is what i observed:
I was testing a user script that uses mmap to read from 100M to 500MB files.
The above files are stored on 3 different filesystems.
Compute nodes - 10G pagepool and 5G seqdiscardthreshold.
1. 4M block size GPFS filesystem, with separate metadata and data. Data on Near
line and metadata on SSDs
2. 1M block size GPFS filesystem as a AFM cache cluster, "with all the required
files fully cached" from the above GPFS cluster as home. Data and Metadata
together on SSDs
3. 16M block size GPFS filesystem, with separate metadata and data. Data on
Near line and metadata on SSDs
When i run the script first time for “each" filesystem:
I see that GPFS reads from the files, and caches into the pagepool as it reads,
from mmdiag -- iohist
When i run the second time, i see that there are no IO requests from the
compute node to GPFS NSD servers, which is expected since all the data from the
3 filesystems is cached.
However - the time taken for the script to run for the files in the 3 different
filesystems is different - although i know that they are just
"mmapping"/reading from pagepool/cache and not from disk.
Here is the difference in time, for IO just from pagepool:
20s 4M block size
15s 1M block size
40S 16M block size.
Why do i see a difference when trying to mmap reads from different block-size
filesystems, although i see that the IO requests are not hitting disks and just
I am willing to share the strace output and mmdiag outputs if needed.
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org