I thought read caching was disabled by default, as the kernel's default handling of pages was better. It's been awhile since I looked at those test results.
On 8/3/16, 11:32 AM, "lustre-discuss on behalf of Mohr Jr, Richard Frank (Rick Mohr)" <[email protected] on behalf of [email protected]> wrote: >Do you have the Lustre read caching feature enabled? I think it should >be on by default, but you might want to check. If the files are only 20 >KB, then I would think the Lustre OSS nodes could keep them in memory >most of the time to speed up access (unless of course this is a metadata >bottleneck as Oliver suggested.) Do your OSS nodes have a lot of memory? > Do you know what your typical memory usage is on the OSS nodes? > >-- >Rick Mohr >Senior HPC System Administrator >National Institute for Computational Sciences >http://www.nics.tennessee.edu > > >> On Jul 28, 2016, at 10:19 PM, Riccardo Veraldi >><[email protected]> wrote: >> >> Hello, >> >> I have a lustre cluster on rhel7, 6 OSS each of them has 3 OSTs and 1 >>MDS. >> >> I am using lustre on ZFS. >> While write performances are excellent also on smaller files, I find >>there is a drop down in performance >> on reading 20KB files. Performance can go as low as 200MB/sec or even >>less. >> I am talking about random reads and random stride reads. >> I did the following to try to improve things: >> € disabled lnet debug messages: >> € sysctl -w lnet.debug=0 >> € increased dirty cache >> € lctl set_param osc.lutrefs\*.max_dirty_mb=256 >> € increased number of RPC in flight >> € for i in `ls >>/proc/fs/lustre/osc/lustrefs-OST00*/max_rpcs_in_flight`; do echo 32 > >>$i; done >> it did not improve reading 20KB file performances. >> I have to say in advance I did not set up any striping because I will >>have no more than 6 concurrent reads and writes, >> so striping is not that much important for me. >> Here the problem is that one single random read of a 20KB file is >>around 190MB/s and this is really disappointing. >> I am open to any suggestion. >> I made sure it is not a ZFS problem, on the OSSs ZFS is performing like >>a charm locally. >> thank you >> >> >> Riccardo >> >> >> _______________________________________________ >> lustre-discuss mailing list >> [email protected] >> http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > >_______________________________________________ >lustre-discuss mailing list >[email protected] >http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
