On Tue, Feb 6, 2018 at 8:15 PM, Vijay Bellur <vbel...@redhat.com> wrote:
> > > On Sun, Feb 4, 2018 at 3:39 AM, Raghavendra Gowdappa <rgowd...@redhat.com> > wrote: > >> All, >> >> One of our users pointed out to the documentation that glusterfs is not >> good for storing "Structured data" [1], while discussing an issue [2]. > > > > As far as I remember, the content around structured data in the Install > Guide is from a FAQ that was being circulated in Gluster, Inc. indicating > the startup's market positioning. Most of that was based on not wanting to > get into performance based comparisons of storage systems that are > frequently seen in the structured data space. > > >> Does any of you have more context on the feasibility of storing >> "structured data" on Glusterfs? Is one of the reasons for such a suggestion >> "staleness of metadata" as encountered in bugs like [3]? >> > > > There are challenges that distributed storage systems face when exposed to > applications that were written for a local filesystem interface. We have > encountered problems with applications like tar [4] that are not in the > realm of "Structured data". If we look at the common theme across all these > problems, it is related to metadata & read after write consistency issues > with the default translator stack that gets exposed on the client side. > While the default stack is optimal for other scenarios, it does seem that a > category of applications needing strict metadata consistency is not well > served by that. We have observed that disabling a few performance > translators and tuning cache timeouts for VFS/FUSE have helped to overcome > some of them. The WIP effort on timestamp consistency across the translator > stack, patches that have been merged as a result of the bugs that you > mention & other fixes for outstanding issues should certainly help in > catering to these workloads better with the file interface. > > There are deployments that I have come across where glusterfs is used for > storing structured data. gluster-block & qemu-libgfapi overcome the > metadata consistency problem by exposing a file as a block device & by > disabling most of the performance translators in the default stack. > Workloads that have been deemed problematic with the file interface for the > reasons alluded above, function well with the block interface. > I agree that gluster-block due to its usage of a subset of glusterfs fops (mostly reads/writes I guess), runs into less number of consistency issues. However, as you've mentioned we seem to disable perf xlator stack in our tests/use-cases till now. Note that perf xlator stack is one of worst offenders as far as the metadata consistency is concerned (relatively less scenarios of data inconsistency). So, I wonder, * what would be the scenario if we enable perf xlator stack for gluster-block? * Is performance on gluster-block satisfactory so that we don't need these xlators? - Or is it that these xlators are not useful for the workload usually run on gluster-block (For random read/write workload, read/write caching xlators offer less or no advantage)? - Or theoretically the workload is ought to benefit from perf xlators, but we don't see them in our results (there are open bugs to this effect)? I am asking these questions to ascertain priority on fixing perf xlators for (meta)data inconsistencies. If we offer a different solution for these workloads, the need for fixing these issues will be less. I feel that we have come a long way from the time the install guide was > written and an update for removing the "staleness of content" might be in > order there :-). > > Regards, > Vijay > > [4] https://bugzilla.redhat.com/show_bug.cgi?id=1058526 > > >> >> [1] http://docs.gluster.org/en/latest/Install-Guide/Overview/ >> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1512691 >> [3] https://bugzilla.redhat.com/show_bug.cgi?id=1390050 >> >> regards, >> Raghavendra >> _______________________________________________ >> Gluster-devel mailing list >> Gluster-devel@gluster.org >> http://lists.gluster.org/mailman/listinfo/gluster-devel >> > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-devel > -- Raghavendra G
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-devel