Re: [Gluster-users] Sharding on 7.4 - filesizes may be wrong

On 4/1/20 6:20 PM, Claus Jeppesen wrote: Has anyone else seen a phenomenon like this ? Well, this one (observed on 8dev git) may be related: # qemu-img convert gluster://192.168.111.2/TEST/qcow-32G.qcow2 gluster://192.168.111.3/TEST/out-32G.raw [2020-04-02 09:38:08.968750] E

Re: [Gluster-users] sharding in glusterfs

Hi, Apologies for the late reply. My email filters are messed up, I missed reading this. Answers to questions around shard algorithm inline ... On Sun, Sep 30, 2018 at 9:54 PM Ashayam Gupta wrote: > Hi Pranith, > > Thanks for you reply, it would be helpful if you can please help us with > the

Re: [Gluster-users] sharding in glusterfs

On Sun, Sep 30, 2018 at 9:54 PM Ashayam Gupta wrote: > Hi Pranith, > > Thanks for you reply, it would be helpful if you can please help us with > the following issues with respect to sharding. > The gluster version we are using is *glusterfs 4.1.4 *on Ubuntu 18.04.1 > LTS > > >-

Re: [Gluster-users] sharding in glusterfs

Hi Pranith, Thanks for you reply, it would be helpful if you can please help us with the following issues with respect to sharding. The gluster version we are using is *glusterfs 4.1.4 *on Ubuntu 18.04.1 LTS - *Shards-Creation Algo*: We were interested in understanding the way in which

Re: [Gluster-users] sharding in glusterfs

On Wed, Sep 19, 2018 at 11:37 AM Ashayam Gupta wrote: > Please find our workload details as requested by you : > > * Only 1 write-mount point as of now > * Read-Mount : Since we auto-scale our machines this can be as big as > 300-400 machines during peak times > * >" multiple concurrent reads

Re: [Gluster-users] sharding in glusterfs

Please find our workload details as requested by you : * Only 1 write-mount point as of now * Read-Mount : Since we auto-scale our machines this can be as big as 300-400 machines during peak times * >" multiple concurrent reads means that Reads will not happen until the file is completely written

Re: [Gluster-users] sharding in glusterfs

On Mon, Sep 17, 2018 at 4:14 AM Ashayam Gupta wrote: > Hi All, > > We are currently using glusterfs for storing large files with write-once > and multiple concurrent reads, and were interested in understanding one of > the features of glusterfs called sharding for our use case. > > So far from

Re: [Gluster-users] sharding in glusterfs

Did you try disperse volume? It may work your workload I think. We are using disperse volumes for archive workloads with 2GB files and I did not encounter any problems. On Mon, Sep 17, 2018 at 1:43 AM Ashayam Gupta wrote: > > Hi All, > > We are currently using glusterfs for storing large files

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

<ihalli...@ndevix.com>; "gluster-user" <gluster-users@gluster.org>; "Nithya Balachandran" <nbala...@redhat.com> Sent: 3/26/2018 2:37:21 AM Subject: Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids Ian, Do you've

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

-quorum-type: server > cluster.data-self-heal-algorithm: full > performance.client-io-threads: off > server.allow-insecure: on > client.event-threads: 8 > storage.owner-gid: 36 > storage.owner-uid: 36 > server.event-threads: 16 > features.shard-block-size: 5GB > features.shard: o

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

"Raghavendra Gowdappa" <rgowd...@redhat.com> To: "Krutika Dhananjay" <kdhan...@redhat.com> Cc: "Ian Halliday" <ihalli...@ndevix.com>; "gluster-user" <gluster-users@gluster.org>; "Nithya Balachandran" <nbala...@redhat.com&g

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

-- Original Message -- From: "Raghavendra Gowdappa" <rgowd...@redhat.com> To: "Krutika Dhananjay" <kdhan...@redhat.com> Cc: "Ian Halliday" <ihalli...@ndevix.com>; "gluster-user" <gluster-users@gluster.org>; "Nithya Balachandran&quo

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

Ian, Do you've a reproducer for this bug? If not a specific one, a general outline of what operations where done on the file will help. regards, Raghavendra On Mon, Mar 26, 2018 at 12:55 PM, Raghavendra Gowdappa wrote: > > > On Mon, Mar 26, 2018 at 12:40 PM, Krutika

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

On Mon, Mar 26, 2018 at 12:40 PM, Krutika Dhananjay wrote: > The gfid mismatch here is between the shard and its "link-to" file, the > creation of which happens at a layer below that of shard translator on the > stack. > > Adding DHT devs to take a look. > Thanks Krutika. I

Re: [Gluster-users] Sharding problem - multiple shard copies with mismatching gfids

The gfid mismatch here is between the shard and its "link-to" file, the creation of which happens at a layer below that of shard translator on the stack. Adding DHT devs to take a look. -Krutika On Mon, Mar 26, 2018 at 1:09 AM, Ian Halliday wrote: > Hello all, > > We are

Re: [Gluster-users] Sharding option for distributed volumes

Hello Ji-Hyeon, Thanks, is that option available in 3.12 gluster release? because we're still on 3.8 and just playing around latest version in order to have our solution migrated. Thank you! 9/21/17 2:26 PM, Ji-Hyeon Gim пишет: Hello Pavel! In my opinion, you need to check

Re: [Gluster-users] Sharding/Local Gluster Volumes

On Thu, Apr 6, 2017 at 7:09 AM, Holger Rojahn wrote: > Hi, > > i ask a question several Days ago ... > in short: Is it Possible to have 5 Bricks with Sharding enabled and > replica count 3 to ensure that all files are on 3 of 5 bricks ? > Manual says i can add any count of

Re: [Gluster-users] Sharding/Local Gluster Volumes

Hi, If you want replica 3, you must have a multiple of 3 bricks. So no, you can't use 5 bricks for a replica 3, that's one of the things gluster can't do unfortunatly. On Thu, Apr 06, 2017 at 01:09:32PM +0200, Holger Rojahn wrote: > Hi, > > i ask a question several Days ago ... > in short: Is

Re: [Gluster-users] Sharding?

> On 10 Mar 2017, at 12:05, Krutika Dhananjay wrote: > > On Fri, Mar 10, 2017 at 4:09 PM, Cedric Lemarchand > wrote: > > > On 10 Mar 2017, at 10:33, Alessandro Briosi > > wrote:

Re: [Gluster-users] Sharding?

2017-03-10 11:39 GMT+01:00 Cedric Lemarchand : > I am still asking myself how such bug could happen on a clustered storage > software, where adding bricks is a base feature for scalable solution, like > Gluster. Or maybe is it that STM releases are really under tested

Re: [Gluster-users] Sharding?

On Fri, Mar 10, 2017 at 4:09 PM, Cedric Lemarchand wrote: > > > On 10 Mar 2017, at 10:33, Alessandro Briosi wrote: > > > > Il 10/03/2017 10:28, Kevin Lemonnier ha scritto: > >>> I haven't done any test yet, but I was under the impression that > >>> sharding

Re: [Gluster-users] Sharding?

> On 10 Mar 2017, at 10:33, Alessandro Briosi wrote: > > Il 10/03/2017 10:28, Kevin Lemonnier ha scritto: >>> I haven't done any test yet, but I was under the impression that >>> sharding feature isn't so stable/mature yet. >>> In the remote of my mind I remember reading

Re: [Gluster-users] Sharding?

On Fri, Mar 10, 2017 at 3:03 PM, Alessandro Briosi wrote: > Il 10/03/2017 10:28, Kevin Lemonnier ha scritto: > > I haven't done any test yet, but I was under the impression that > sharding feature isn't so stable/mature yet. > In the remote of my mind I remember reading

Re: [Gluster-users] Sharding?

> I'm using 3.8.9, so I suppose all known bugs have been fixed there (also > the one with adding briks) Can't comment on that, I just saw they fixed it, not sure in which version. I'd wait for someone who knows to confirm that before going into production if adding bricks is something you'll need

Re: [Gluster-users] Sharding?

Il 10/03/2017 10:28, Kevin Lemonnier ha scritto: >> I haven't done any test yet, but I was under the impression that >> sharding feature isn't so stable/mature yet. >> In the remote of my mind I remember reading something about a >> bug/situation which caused data corruption. >> Can someone

Re: [Gluster-users] Sharding?

> I haven't done any test yet, but I was under the impression that > sharding feature isn't so stable/mature yet. > In the remote of my mind I remember reading something about a > bug/situation which caused data corruption. > Can someone confirm that sharding is stable enough to be used in >

Re: [Gluster-users] Sharding?

Il 09/03/2017 17:17, Vijay Bellur ha scritto: > > > On Thu, Mar 9, 2017 at 11:10 AM, Kevin Lemonnier > wrote: > > > I've seen the term sharding pop up on the list a number of times > but I > > haven't found any documentation or

Re: [Gluster-users] Sharding?

Hi folks, This chapter on sharding and how to configure it went into the RHGS 3.1 Administration Guide some time ago: https://access.redhat.com/documentation/en-US/Red_Hat_Storage/3.1/html/Administration_Guide/chap-Managing_Sharding.html If there's anything in here that isn't clear, let me know

Re: [Gluster-users] Sharding?

On Thu, Mar 9, 2017 at 11:10 AM, Kevin Lemonnier wrote: > > I've seen the term sharding pop up on the list a number of times but I > > haven't found any documentation or explanation of what it is. Would > someone > > please enlighten me? > > It's a way to split the files

Re: [Gluster-users] Sharding?

> I've seen the term sharding pop up on the list a number of times but I > haven't found any documentation or explanation of what it is. Would someone > please enlighten me? It's a way to split the files you put on the volume. With a shard size of 64 MB for example, the biggest file on the volume

Re: [Gluster-users] Sharding - what next?

On 16/12/15 22:59, Krutika Dhananjay wrote: I guess I did not make myself clear. Apologies. I meant to say that printing a single list of counts aggregated from all bricks can be tricky and is susceptible to the possibility of same entry getting counted multiple times if the inode needs a heal

Re: [Gluster-users] Sharding - what next?

- Original Message - > From: "Lindsay Mathieson" > To: "Krutika Dhananjay" > Cc: "Gluster Devel" , "gluster-users" > > Sent: Wednesday, December 16, 2015 6:56:03 AM > Subject: Re:

Re: [Gluster-users] Sharding - what next?

Hi, late reply again ... On 10/12/2015 5:33 PM, Krutika Dhananjay wrote: There is a 'heal-info summary' command that is under review, written by Mohammed Ashiq @ http://review.gluster.org/#/c/12154/3 which prints the number of files that are yet to be healed. It could perhaps be enhanced to

Re: [Gluster-users] Sharding - what next?

Hi Guys, sorry for the late reply, my attention tends to be somewhat sporadic due to work and the large number of rescue dogs/cats I care for :) On 3/12/2015 8:34 PM, Krutika Dhananjay wrote: We would love to hear from you on what you think of the feature and where it could be improved.

Re: [Gluster-users] Sharding - what next?

- Original Message - > From: "Lindsay Mathieson" > To: "Krutika Dhananjay" , "Gluster Devel" > , "gluster-users" > Sent: Wednesday, December 9, 2015 6:48:40 PM > Subject: Re: Sharding