Hi all, We've a feature performance.parallel-readdir [1] that is known to improve performance of readdir operations [2][3][4]. The option is especially useful when distribute scale is relatively large (>10) and is known to improve performance of readdir operations even on smaller scale of distribute count 1 [4].
However, this option is not enabled by default. I am here proposing to make this as a default feature. But, there are some important things to be addressed in readdir-ahead (which is core part of parallel-readdir), before we can do so: To summarize issues with readdir-ahead: * There seems to be one prominent problem of missing dentries with parallel-readdir. There was one problem discussed on tech-list just yesterday. I've heard about this recurrently earlier too. Not sure whether this is the problem of missing unlink/rmdir/create etc fops (see below) in readdir-ahead. ATM, no RCA. * fixes to maintain stat-consistency in dentries pre-fetched have not made into downstream yet (though merged upstream [5]). * readdir-ahead doesn't implement directory modification fops like rmdir/create/symlink/link/unlink/rename. This means cache won't be updated wiith newer content, even on single mount till its consumed by application or purged. * dht linkto-files should store relative positions of subvolumes instead of absolute subvolume name, so that changes to immediate child won't render them stale. * Features parallel-readdir depends on to be working should be enabled automatically even though they were off earlier when parallel-readdir is enabled [6]. I've listed important known issues above. But we can discuss which are the blockers for making this feature as a default. Thoughts? [1] http://review.gluster.org/#/c/16090/ [2] https://events.static.linuxfound.org/sites/events/files/slides/Gluster_DirPerf_Vault2017_0.pdf (sections on small directory) [3] https://bugzilla.redhat.com/show_bug.cgi?id=1628807#c35 <https://bugzilla.redhat.com/show_bug.cgi?id=1628807#c34> [4] https://www.spinics.net/lists/gluster-users/msg34956.html [5] http://review.gluster.org/#/c/glusterfs/+/20639/ [6] https://bugzilla.redhat.com/show_bug.cgi?id=1631406 regards, Raghavendra
_______________________________________________ Gluster-devel mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-devel
