Hi Stephane, thank you! With ashift=9 reads are much better (1.7 GB/s vs 0.5 GB/s). However reads are still 5-10 times slower in the mixed read/write workflows. I'm continuing tuning/benchmarking and will keep you updated.
Thanks, Alex 2018-03-30 20:00 GMT+03:00 Stephane Thiell <[email protected]>: > Hi Alex, > > I’m no ZFS expert, but for a new project I recently faced some read > performance issues too when doing some zfs 0.7 testing, but not at all as bad > as you… so I feel sorry for you as you seem to have done quite a good work so > far… so perhaps some ideas: > > - check that arc is fully enabled (primarycache=all) > - restraint arc memory usage (zfs_arc_min = zfs_arc_max = 50% of the RAM is > what we usually use) > - try different values of ashift (unlikely but try to keep the default 9, > and/or try 11, 13 or more but it has a volume cost) > - turn off readahead but you already did that... > - bump zfs_vdev_cache_max to 131072 > - bump zfs_read_chunk_size to 1310720 > - bump zfs_vdev_cache_bshift to 17 > - bump zfs_vdev_async_read_min_active to 8 > - bump zfs_vdev_async_read_max_active to 32 > - bump zfs_vdev_sync_read_max_active to 32 > > /!\ These tunings are not to take as is and are meant for performance > testing, plus some of them might be obsolete with 0.7, but we still use a few > of these on our ZFS on Linux systems. > > As a rule of thumb, do not explicitly tune if you don’t really need it. > > To get the best read performance from nearline drives with Lustre (without > the use of any SSD), we went with mdraid/ldiskfs and we’re VERY happy with > that. With the same hardware, a ZFS backend will definitively provide better > writes though. > > Good luck... > > Stephane > > > _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
