Re: [Gluster-devel] Rebalance improvement design

2015-05-04 Thread Susant Palai
Comments inline - Original Message - > From: "Benjamin Turner" > To: "Susant Palai" > Cc: "Vijay Bellur" , "Gluster Devel" > > Sent: Monday, May 4, 2015 8:58:13 PM > Subject: Re: [Gluster-devel] Rebalance improvement desig

Re: [Gluster-devel] Rebalance improvement design

2015-05-04 Thread Benjamin Turner
of local crawl mostly as only 81GB is migrated out > of 500GB. > > Thanks, > Susant > > - Original Message - > > From: "Benjamin Turner" > > To: "Vijay Bellur" > > Cc: "Gluster Devel" > > Sent: Monday, May 4, 2015 5:1

Re: [Gluster-devel] Rebalance improvement design

2015-05-04 Thread Susant Palai
local crawl mostly as only 81GB is migrated out of 500GB. Thanks, Susant - Original Message - > From: "Benjamin Turner" > To: "Vijay Bellur" > Cc: "Gluster Devel" > Sent: Monday, May 4, 2015 5:18:13 PM > Subject: Re: [Gluster-devel] Reba

Re: [Gluster-devel] Rebalance improvement design

2015-05-04 Thread Benjamin Turner
Thanks Vijay! I forgot to upgrade the kernel(thinp 6.6 perf bug gah) before I created this data set, so its a bit smaller: total threads = 16 total files = 7,060,700 (64 kb files, 100 files per dir) total data = 430.951 GB 88.26% of requested files processed, minimum is 70.00 10101.355737 sec

Re: [Gluster-devel] Rebalance improvement design

2015-05-03 Thread Vijay Bellur
On 05/01/2015 10:23 AM, Benjamin Turner wrote: Ok I have all my data created and I just started the rebalance. One thing to not in the client log I see the following spamming: [root@gqac006 ~]# cat /var/log/glusterfs/gluster-mount-.log | wc -l 394042 [2015-05-01 00:47:55.591150] I [MSGID: 1090

Re: [Gluster-devel] Rebalance improvement design

2015-05-03 Thread Benjamin Turner
>>> I'll have a run on the patch started tomorrow. >>> >>> -b >>> >>> On Wed, Apr 29, 2015 at 12:51 PM, Nithya Balachandran < >>> nbala...@redhat.com> wrote: >>> >>>> >>>> Doh my mistake, I t

Re: [Gluster-devel] Rebalance improvement design

2015-05-01 Thread Ravishankar N
; > > > > > From: "Benjamin Turner" mailto:bennytu...@gmail.com>> > > > > > > To: "Nithya Balachandran" mailto:nbala...@redhat.com>> > > > > > > Cc: "Susant Palai&quo

Re: [Gluster-devel] Rebalance improvement design

2015-04-30 Thread Benjamin Turner
merged? >>> >>> Yes, it would be very useful to have this run as the baseline. The patch >>> has just been merged in master. It should be backported to 3.7 in a day or >>> so. >>> >>> Regards, >>> Nithya >>> >>> >>> > &

Re: [Gluster-devel] Rebalance improvement design

2015-04-30 Thread Benjamin Turner
22, 2015 at 1:10 AM, Nithya Balachandran >> > > > > > >> > > > > > wrote: >> > > > > > >> > > > > > > That sounds great. Thanks. >> > > > > > > >> > > > > > > Rega

Re: [Gluster-devel] Rebalance improvement design

2015-04-29 Thread Benjamin Turner
now, I'll have some feedback for you > > this > > > > > > > week. > > > > > > > > > > > > > > -b > > > > > > > > > > > > > > On Tue, Apr 21, 2015 at 11:36 AM, Nithya Balachandran > &g

Re: [Gluster-devel] Rebalance improvement design

2015-04-29 Thread Nithya Balachandran
nal Message - > > > > > > From: "Benjamin Turner" > > > > > > To: "Nithya Balachandran" > > > > > > Cc: "Susant Palai" , "Gluster Devel" < > > > > > > gluster-devel@gluster.org> >

Re: [Gluster-devel] Rebalance improvement design

2015-04-29 Thread Shyam
.@redhat.com>> > To: "Benjamin Turner" mailto:bennytu...@gmail.com>>, "Nithya Balachandran" mailto:nbala...@redhat.com>> > Cc: "Shyamsundar Ranganathan" mailto:srang...@redhat.com>> > Sent: Wednesday, April 29, 2015 1:22:02 PM

Re: [Gluster-devel] Rebalance improvement design

2015-04-29 Thread Benjamin Turner
--- Original Message - > > From: "Susant Palai" > > To: "Benjamin Turner" , "Nithya Balachandran" < > nbala...@redhat.com> > > Cc: "Shyamsundar Ranganathan" > > Sent: Wednesday, April 29, 2015 1:22:02 PM > > Su

Re: [Gluster-devel] Rebalance improvement design

2015-04-29 Thread Susant Palai
min Turner" , "Nithya Balachandran" > > Cc: "Shyamsundar Ranganathan" > Sent: Wednesday, April 29, 2015 1:22:02 PM > Subject: Re: [Gluster-devel] Rebalance improvement design > > This is how it looks for 2000 file. each 1MB. Done rebalance on 2*2 + 2. &

Re: [Gluster-devel] Rebalance improvement design

2015-04-29 Thread Susant Palai
test with the same configuration on a small data set. > > Thanks, > Susant > > > - Original Message - > > From: "Benjamin Turner" > > To: "Nithya Balachandran" > > Cc: "Susant Palai" , "Gluster Devel"

Re: [Gluster-devel] Rebalance improvement design

2015-04-28 Thread Susant Palai
rom: "Benjamin Turner" > To: "Nithya Balachandran" > Cc: "Susant Palai" , "Gluster Devel" > > Sent: Wednesday, April 29, 2015 2:13:05 AM > Subject: Re: [Gluster-devel] Rebalance improvement design > > I am not seeing the performance you were.

Re: [Gluster-devel] Rebalance improvement design

2015-04-28 Thread Nithya Balachandran
Hi Ben, Is it ok if we take a look at your systems? Regards, Nithya - Original Message - From: "Benjamin Turner" To: "Nithya Balachandran" Cc: "Susant Palai" , "Gluster Devel" Sent: Wednesday, 29 April, 2015 2:13:05 AM Subject: Re: [Gluster-d

Re: [Gluster-devel] Rebalance improvement design

2015-04-28 Thread Benjamin Turner
< > gluster-devel@gluster.org> > Sent: Wednesday, 22 April, 2015 12:14:14 AM > Subject: Re: [Gluster-devel] Rebalance improvement design > > I am setting up a test env now, I'll have some feedback for you this week. > > -b > > On Tue, Apr 21, 2015 at 11:36 AM, Ni

Re: [Gluster-devel] Rebalance improvement design

2015-04-21 Thread Nithya Balachandran
That sounds great. Thanks. Regards, Nithya - Original Message - From: "Benjamin Turner" To: "Nithya Balachandran" Cc: "Susant Palai" , "Gluster Devel" Sent: Wednesday, 22 April, 2015 12:14:14 AM Subject: Re: [Gluster-devel] Rebalance improvemen

Re: [Gluster-devel] Rebalance improvement design

2015-04-21 Thread Benjamin Turner
Susant Palai" > To: "Benjamin Turner" > Cc: "Gluster Devel" > Sent: Monday, April 13, 2015 9:55:07 AM > Subject: Re: [Gluster-devel] Rebalance improvement design > > Hi Ben, > Uploaded a new patch here: http://review.gluster.org/#/c/9657/. We can >

Re: [Gluster-devel] Rebalance improvement design

2015-04-21 Thread Nithya Balachandran
Hi Ben, Did you get a chance to try this out? Regards, Nithya - Original Message - From: "Susant Palai" To: "Benjamin Turner" Cc: "Gluster Devel" Sent: Monday, April 13, 2015 9:55:07 AM Subject: Re: [Gluster-devel] Rebalance improvement design Hi Be

Re: [Gluster-devel] Rebalance improvement design

2015-04-12 Thread Susant Palai
-- From: "Benjamin Turner" To: "Vijay Bellur" Cc: "Susant Palai" , "Gluster Devel" Sent: Thursday, 9 April, 2015 2:10:30 AM Subject: Re: [Gluster-devel] Rebalance improvement design I have some rebalance perf regression stuff I have been working on, is there

Re: [Gluster-devel] Rebalance improvement design

2015-04-09 Thread Susant Palai
ot;Vijay Bellur" Cc: "Susant Palai" , "Gluster Devel" Sent: Thursday, 9 April, 2015 2:10:30 AM Subject: Re: [Gluster-devel] Rebalance improvement design I have some rebalance perf regression stuff I have been working on, is there an RPM with these patches anywhere so

Re: [Gluster-devel] Rebalance improvement design

2015-04-08 Thread Benjamin Turner
I have some rebalance perf regression stuff I have been working on, is there an RPM with these patches anywhere so that I can try it on my systems? If not I'll just build from: git fetch git://review.gluster.org/glusterfs refs/changes/57/9657/8 && git cherry-pick FETCH_HEAD I will have _at_least

Re: [Gluster-devel] Rebalance improvement design

2015-04-07 Thread Vijay Bellur
On 04/07/2015 03:08 PM, Susant Palai wrote: Here is one test performed on a 300GB data set and around 100%(1/2 the time) improvement was seen. [root@gprfs031 ~]# gluster v i Volume Name: rbperf Type: Distribute Volume ID: 35562662-337e-4923-b862-d0bbb0748003 Status: Started Number of Bricks: 4

Re: [Gluster-devel] Rebalance improvement design

2015-04-07 Thread Susant Palai
" To: "Susant Palai" , "Gluster Devel" Sent: Monday, 6 April, 2015 7:39:27 PM Subject: Re: [Gluster-devel] Rebalance improvement design On 03/31/2015 01:49 PM, Susant Palai wrote: > Hi, > Posted patch for rebalance improvement here: > http://review.gluster.org

Re: [Gluster-devel] Rebalance improvement design

2015-04-06 Thread Vijay Bellur
On 03/31/2015 01:49 PM, Susant Palai wrote: Hi, Posted patch for rebalance improvement here: http://review.gluster.org/#/c/9657/ . You can find the feature page here: http://www.gluster.org/community/documentation/index.php/Features/improve_rebalance_performance The current patch address t

[Gluster-devel] Rebalance improvement design

2015-03-31 Thread Susant Palai
Hi, Posted patch for rebalance improvement here: http://review.gluster.org/#/c/9657/ . You can find the feature page here: http://www.gluster.org/community/documentation/index.php/Features/improve_rebalance_performance The current patch address two part of the design proposed. 1. Rebalance mu