Now that there are sufficient detail in place, could a Gluster team
member file a RHBZ and post it back to this thread?
On Wed, Mar 20, 2019 at 2:51 AM Jim Kinney <jim.kin...@gmail.com> wrote:
>
> Volume Name: home
> Type: Replicate
> Volume ID: 5367adb1-99fc-44c3-98c4-71f7a41e628a
> Status: Started
> Snapshot Count: 0
> Number of Bricks: 1 x 2 = 2
> Transport-type: tcp,rdma
> Bricks:
> Brick1: bmidata1:/data/glusterfs/home/brick/brick
> Brick2: bmidata2:/data/glusterfs/home/brick/brick
> Options Reconfigured:
> performance.client-io-threads: off
> storage.build-pgfid: on
> cluster.self-heal-daemon: enable
> performance.readdir-ahead: off
> nfs.disable: off
>
>
> There are 11 other volumes and all are similar.
>
>
> On Tue, 2019-03-19 at 13:59 -0700, Vijay Bellur wrote:
>
> Thank you for the reproducer! Can you please let us know the output of
> `gluster volume info`?
>
> Regards,
> Vijay
>
> On Tue, Mar 19, 2019 at 12:53 PM Jim Kinney <jim.kin...@gmail.com> wrote:
>
> This python will fail when writing to a file in a glusterfs fuse mounted
> directory.
>
> import mmap
>
> # write a simple example file
> with open("hello.txt", "wb") as f:
> f.write("Hello Python!\n")
>
> with open("hello.txt", "r+b") as f:
> # memory-map the file, size 0 means whole file
> mm = mmap.mmap(f.fileno(), 0)
> # read content via standard file methods
> print mm.readline() # prints "Hello Python!"
> # read content via slice notation
> print mm[:5] # prints "Hello"
> # update content using slice notation;
> # note that new content must have same size
> mm[6:] = " world!\n"
> # ... and read again using standard file methods
> mm.seek(0)
> print mm.readline() # prints "Hello world!"
> # close the map
> mm.close()
>
>
>
>
>
>
>
> On Tue, 2019-03-19 at 12:06 -0400, Jim Kinney wrote:
>
> Native mount issue with multiple clients (centos7 glusterfs 3.12).
>
> Seems to hit python 2.7 and 3+. User tries to open file(s) for write on long
> process and system eventually times out.
>
> Switching to NFS stops the error.
>
> No bug notice yet. Too many pans on the fire :-(
>
> On Tue, 2019-03-19 at 18:42 +0530, Amar Tumballi Suryanarayan wrote:
>
> Hi Jim,
>
> On Tue, Mar 19, 2019 at 6:21 PM Jim Kinney <jim.kin...@gmail.com> wrote:
>
>
> Issues with glusterfs fuse mounts cause issues with python file open for
> write. We have to use nfs to avoid this.
>
> Really want to see better back-end tools to facilitate cleaning up of
> glusterfs failures. If system is going to use hard linked ID, need a mapping
> of id to file to fix things. That option is now on for all exports. It should
> be the default If a host is down and users delete files by the thousands,
> gluster _never_ catches up. Finding path names for ids across even a 40TB
> mount, much less the 200+TB one, is a slow process. A network outage of 2
> minutes and one system didn't get the call to recursively delete several
> dozen directories each with several thousand files.
>
>
>
> Are you talking about some issues in geo-replication module or some other
> application using native mount? Happy to take the discussion forward about
> these issues.
>
> Are there any bugs open on this?
>
> Thanks,
> Amar
>
>
>
>
> nfs
> On March 19, 2019 8:09:01 AM EDT, Hans Henrik Happe <ha...@nbi.dk> wrote:
>
> Hi,
>
> Looking into something else I fell over this proposal. Being a shop that are
> going into "Leaving GlusterFS" mode, I thought I would give my two cents.
>
> While being partially an HPC shop with a few Lustre filesystems, we chose
> GlusterFS for an archiving solution (2-3 PB), because we could find files in
> the underlying ZFS filesystems if GlusterFS went sour.
>
> We have used the access to the underlying files plenty, because of the
> continuous instability of GlusterFS'. Meanwhile, Lustre have been almost
> effortless to run and mainly for that reason we are planning to move away
> from GlusterFS.
>
> Reading this proposal kind of underlined that "Leaving GluserFS" is the right
> thing to do. While I never understood why GlusterFS has been in feature crazy
> mode instead of stabilizing mode, taking away crucial features I don't get.
> With RoCE, RDMA is getting mainstream. Quotas are very useful, even though
> the current implementation are not perfect. Tiering also makes so much sense,
> but, for large files, not on a per-file level.
>
> To be honest we only use quotas. We got scared of trying out new performance
> features that potentially would open up a new back of issues.
>
> Sorry for being such a buzzkill. I really wanted it to be different.
>
> Cheers,
> Hans Henrik
>
> On 19/07/2018 08.56, Amar Tumballi wrote:
>
> Hi all,
>
> Over last 12 years of Gluster, we have developed many features, and continue
> to support most of it till now. But along the way, we have figured out better
> methods of doing things. Also we are not actively maintaining some of these
> features.
>
> We are now thinking of cleaning up some of these ‘unsupported’ features, and
> mark them as ‘SunSet’ (i.e., would be totally taken out of codebase in
> following releases) in next upcoming release, v5.0. The release notes will
> provide options for smoothly migrating to the supported configurations.
>
> If you are using any of these features, do let us know, so that we can help
> you with ‘migration’.. Also, we are happy to guide new developers to work on
> those components which are not actively being maintained by current set of
> developers.
>
> List of features hitting sunset:
>
> ‘cluster/stripe’ translator:
>
> This translator was developed very early in the evolution of GlusterFS, and
> addressed one of the very common question of Distributed FS, which is “What
> happens if one of my file is bigger than the available brick. Say, I have 2
> TB hard drive, exported in glusterfs, my file is 3 TB”. While it solved the
> purpose, it was very hard to handle failure scenarios, and give a real good
> experience to our users with this feature. Over the time, Gluster solved the
> problem with it’s ‘Shard’ feature, which solves the problem in much better
> way, and provides much better solution with existing well supported stack.
> Hence the proposal for Deprecation.
>
> If you are using this feature, then do write to us, as it needs a proper
> migration from existing volume to a new full supported volume type before you
> upgrade.
>
> ‘storage/bd’ translator:
>
> This feature got into the code base 5 years back with this patch[1]. Plan was
> to use a block device directly as a brick, which would help to handle
> disk-image storage much easily in glusterfs.
>
> As the feature is not getting more contribution, and we are not seeing any
> user traction on this, would like to propose for Deprecation.
>
> If you are using the feature, plan to move to a supported gluster volume
> configuration, and have your setup ‘supported’ before upgrading to your new
> gluster version.
>
> ‘RDMA’ transport support:
>
> Gluster started supporting RDMA while ib-verbs was still new, and very
> high-end infra around that time were using Infiniband. Engineers did work
> with Mellanox, and got the technology into GlusterFS for better data
> migration, data copy. While current day kernels support very good speed with
> IPoIB module itself, and there are no more bandwidth for experts in these
> area to maintain the feature, we recommend migrating over to TCP (IP based)
> network for your volume.
>
> If you are successfully using RDMA transport, do get in touch with us to
> prioritize the migration plan for your volume. Plan is to work on this after
> the release, so by version 6.0, we will have a cleaner transport code, which
> just needs to support one type.
>
> ‘Tiering’ feature
>
> Gluster’s tiering feature which was planned to be providing an option to keep
> your ‘hot’ data in different location than your cold data, so one can get
> better performance. While we saw some users for the feature, it needs much
> more attention to be completely bug free. At the time, we are not having any
> active maintainers for the feature, and hence suggesting to take it out of
> the ‘supported’ tag.
>
> If you are willing to take it up, and maintain it, do let us know, and we are
> happy to assist you.
>
> If you are already using tiering feature, before upgrading, make sure to do
> gluster volume tier detach all the bricks before upgrading to next release.
> Also, we recommend you to use features like dmcache on your LVM setup to get
> best performance from bricks.
>
> ‘Quota’
>
> This is a call out for ‘Quota’ feature, to let you all know that it will be
> ‘no new development’ state. While this feature is ‘actively’ in use by many
> people, the challenges we have in accounting mechanisms involved, has made it
> hard to achieve good performance with the feature. Also, the amount of
> extended attribute get/set operations while using the feature is not very
> ideal. Hence we recommend our users to move towards setting quota on backend
> bricks directly (ie, XFS project quota), or to use different volumes for
> different directories etc.
>
> As the feature wouldn’t be deprecated immediately, the feature doesn’t need a
> migration plan when you upgrade to newer version, but if you are a new user,
> we wouldn’t recommend setting quota feature. By the release dates, we will be
> publishing our best alternatives guide for gluster’s current quota feature.
>
> Note that if you want to contribute to the feature, we have project quota
> based issue open[2] Happy to get contributions, and help in getting a newer
> approach to Quota.
>
>
> ________________________________
>
> These are our set of initial features which we propose to take out of ‘fully’
> supported features. While we are in the process of making the user/developer
> experience of the project much better with providing well maintained
> codebase, we may come up with few more set of features which we may possibly
> consider to move out of support, and hence keep watching this space.
>
> [1] - http://review.gluster.org/4809
>
> [2] - https://github.com/gluster/glusterfs/issues/184
>
> Regards,
>
> Vijay, Shyam, Amar
>
>
>
> _______________________________________________
>
> Gluster-users mailing list
>
> Gluster-users@gluster.org
>
>
> https://lists.glus
>
>
> --
>
>
> Sent from my Android device with K-9 Mail. All tyopes are thumb related and
> reflect authenticity.ter.org/mailman/listinfo/gluster-users
>
>
>
> --
>
>
> James P. Kinney III
>
>
> Every time you stop a school, you will have to build a jail. What you
>
> gain at one end you lose at the other. It's like feeding a dog on his
>
> own tail. It won't fatten the dog.
>
> - Speech 11/23/1900 Mark Twain
>
>
> http://heretothereideas.blogspot.com/
>
> --
>
>
> James P. Kinney III
>
>
> Every time you stop a school, you will have to build a jail. What you
>
> gain at one end you lose at the other. It's like feeding a dog on his
>
> own tail. It won't fatten the dog.
>
> - Speech 11/23/1900 Mark Twain
>
>
> http://heretothereideas.blogspot.com/
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
>
> --
>
> James P. Kinney III
>
> Every time you stop a school, you will have to build a jail. What you
> gain at one end you lose at the other. It's like feeding a dog on his
> own tail. It won't fatten the dog.
> - Speech 11/23/1900 Mark Twain
>
> http://heretothereideas.blogspot.com/
>
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-users
