Re: [Gluster-users] Rebalance taking > 2 months

2018-08-01 Thread Nithya Balachandran
On 31 July 2018 at 22:17, Rusty Bower wrote: > Is it possible to pause the rebalance to get those number? I'm hesitant to > stop the rebalance and have to redo the entire thing again. > > I'm afraid not. Rebalance will start from the beginning if you do so. > On Tue, Jul 31, 2018 at 11:40

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-31 Thread Rusty Bower
Is it possible to pause the rebalance to get those number? I'm hesitant to stop the rebalance and have to redo the entire thing again. On Tue, Jul 31, 2018 at 11:40 AM, Nithya Balachandran wrote: > > > On 31 July 2018 at 19:44, Rusty Bower wrote: > >> I'll figure out what hasn't been

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-31 Thread Nithya Balachandran
On 31 July 2018 at 19:44, Rusty Bower wrote: > I'll figure out what hasn't been rebalanced yet and run the script. > > There's only a single client accessing this gluster volume, and while the > rebalance is taking place, the I am only able to read/write to the volume > at around 3MB/s. If I log

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-31 Thread Rusty Bower
I'll figure out what hasn't been rebalanced yet and run the script. There's only a single client accessing this gluster volume, and while the rebalance is taking place, the I am only able to read/write to the volume at around 3MB/s. If I log onto one of the bricks, I can read/write to the

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-31 Thread Nithya Balachandran
Hi Rusty, A rebalance involves 2 steps: 1. Setting a new layout on a directory 2. Migrating any files inside that directory that hash to a different subvol based on the new layout set in step 1. A few things to keep in mind : - Any new content created on this volume will currently

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-30 Thread Nithya Balachandran
I have not documented this yet - I will send you the steps tomorrow. Regards, Nithya On 30 July 2018 at 20:23, Rusty Bower wrote: > That would be awesome. Where can I find these? > > Rusty > > Sent from my iPhone > > On Jul 30, 2018, at 03:40, Nithya Balachandran > wrote: > > Hi Rusty, > >

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-30 Thread Rusty Bower
That would be awesome. Where can I find these? Rusty Sent from my iPhone > On Jul 30, 2018, at 03:40, Nithya Balachandran wrote: > > Hi Rusty, > > Sorry for the delay getting back to you. I had a quick look at the rebalance > logs - it looks like the estimates are based on the time taken to

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-30 Thread Nithya Balachandran
Hi Rusty, Sorry for the delay getting back to you. I had a quick look at the rebalance logs - it looks like the estimates are based on the time taken to rebalance the smaller files. We do have a scripting option where we can use virtual xattrs to trigger file migration from a mount point. That

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-27 Thread Rusty Bower
Just wanted to ping this to see if you guys had any thoughts, or other scripts I can run for this stuff. It's still predicting another 90 days to rebalance this, and performance is basically garbage while it rebalances. Rusty On Mon, Jul 23, 2018 at 10:19 AM, Rusty Bower wrote: > datanode03 is

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-23 Thread Rusty Bower
datanode03 is the newest brick the bricks had gotten pretty full, which I think might be part of the issue: - datanode01 /dev/sda1 51T 48T 3.3T 94% /mnt/data - datanode02 /dev/sda1 51T 48T 3.4T 94% /mnt/data - datanode03 /dev/md0 128T 4.6T

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-23 Thread Nithya Balachandran
Hi Rusty, Sorry I took so long to get back to you. Which is the newly added brick? I see datanode02 has not picked up any files for migration which is odd. How full are the individual bricks (df -h ) output. Is each of your bricks in a separate partition? Can you send me the rebalance logs from

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-16 Thread Rusty Bower
Thanks for the reply Nithya. 1. glusterfs 4.1.1 2. Volume Name: data Type: Distribute Volume ID: 294d95ce-0ff3-4df9-bd8c-a52fc50442ba Status: Started Snapshot Count: 0 Number of Bricks: 3 Transport-type: tcp Bricks: Brick1: datanode01:/mnt/data/bricks/data Brick2:

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-16 Thread Nithya Balachandran
If possible, please send the rebalance logs as well. On 16 July 2018 at 10:14, Nithya Balachandran wrote: > Hi Rusty, > > We need the following information: > >1. The exact gluster version you are running >2. gluster volume info >3. gluster rebalance status >4. Information on

Re: [Gluster-users] Rebalance taking > 2 months

2018-07-15 Thread Nithya Balachandran
Hi Rusty, We need the following information: 1. The exact gluster version you are running 2. gluster volume info 3. gluster rebalance status 4. Information on the directory structure and file locations on your volume. 5. How many levels of directories 6. How many files and

[Gluster-users] Rebalance taking > 2 months

2018-07-15 Thread Rusty Bower
Hey folks, I just added a new brick to my existing gluster volume, but *gluster volume rebalance data status* is telling me the following: Estimated time left for rebalance to complete : > 2 months. Please try again later. I already did a fix-mapping, but this thing is absolutely crawling