On Fri, Sep 21, 2018 at 11:25 PM Raghavendra Gowdappa <[email protected]> wrote:
> Hi all, > > We've a feature performance.parallel-readdir [1] that is known to improve > performance of readdir operations [2][3][4]. The option is especially > useful when distribute scale is relatively large (>10) and is known to > improve performance of readdir operations even on smaller scale of > distribute count 1 [4]. > > However, this option is not enabled by default. I am here proposing to > make this as a default feature. > > But, there are some important things to be addressed in readdir-ahead > (which is core part of parallel-readdir), before we can do so: > > To summarize issues with readdir-ahead: > * There seems to be one prominent problem of missing dentries with > parallel-readdir. There was one problem discussed on tech-list just > yesterday. I've heard about this recurrently earlier too. Not sure whether > this is the problem of missing unlink/rmdir/create etc fops (see below) in > readdir-ahead. ATM, no RCA. > * fixes to maintain stat-consistency in dentries pre-fetched have not made > into downstream yet (though merged upstream [5]). > * readdir-ahead doesn't implement directory modification fops like > rmdir/create/symlink/link/unlink/rename. This means cache won't be updated > wiith newer content, even on single mount till its consumed by application > or purged. > * dht linkto-files should store relative positions of subvolumes instead > of absolute subvolume name, so that changes to immediate child won't render > them stale. > * Features parallel-readdir depends on to be working should be enabled > automatically even though they were off earlier when parallel-readdir is > enabled [6]. > > I've listed important known issues above. But we can discuss which are the > blockers for making this feature as a default. > > Thoughts? > > [1] http://review.gluster.org/#/c/16090/ > [2] > https://events.static.linuxfound.org/sites/events/files/slides/Gluster_DirPerf_Vault2017_0.pdf > (sections on small directory) > [3] https://bugzilla.redhat.com/show_bug.cgi?id=1628807#c35 > <https://bugzilla.redhat.com/show_bug.cgi?id=1628807#c34> > [4] https://www.spinics.net/lists/gluster-users/msg34956.html > [5] http://review.gluster.org/#/c/glusterfs/+/20639/ > [6] https://bugzilla.redhat.com/show_bug.cgi?id=1631406 > > regards, > Raghavendra > >
_______________________________________________ Gluster-devel mailing list [email protected] https://lists.gluster.org/mailman/listinfo/gluster-devel
