Re: [Gluster-devel] [Gluster-users] Rebalance improvement.

2020-08-03 Thread Susant Palai 
On Mon, Aug 3, 2020 at 2:53 PM sankarshan 
wrote:

> On Mon, 3 Aug 2020 at 12:47, Susant Palai  wrote:
> >
> > Centos Users can add the following repo and install the build from the
> master branch to try out the feature. [Testing purpose only, not ready for
> consumption in production env.]
> >
> > [gluster-nightly-master] baseurl=
> http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/
> gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly
> builds (master branch)
> >
> > A summary of perf numbers from our test lab :
> >
>
> Are these numbers impacted by sizing of the machine instance/hardware?
> What is the configuration on which these numbers were recorded?
>

There were 4 bricks(one brick per node) to begin with. Post creation of
directories, 2 more bricks(from two more nodes) were added.

Disks - HDD
Network - 10Gbps ethernet link
cores -  24

Numbers will definitely vary according to disk, network configuration.


>
> > DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth
> - 100 (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%
> >
> > Susant
>
> --
> sankarshan mukhopadhyay
> 
> ___
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
>
>
>
>
> Gluster-devel mailing list
> Gluster-devel@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
>
___

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Rebalance improvement.

2020-08-03 Thread sankarshan 
On Mon, 3 Aug 2020 at 12:47, Susant Palai  wrote:
>
> Centos Users can add the following repo and install the build from the master 
> branch to try out the feature. [Testing purpose only, not ready for 
> consumption in production env.]
>
> [gluster-nightly-master] 
> baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ 
> gpgcheck=0 keepalive=1 enabled=1 repo_gpgcheck = 0 name=Gluster Nightly 
> builds (master branch)
>
> A summary of perf numbers from our test lab :
>

Are these numbers impacted by sizing of the machine instance/hardware?
What is the configuration on which these numbers were recorded?

> DirSize - 1Million Old New %diff Depth - 100 (Run 1) 353 74 +377% Depth - 100 
> (Run 2) 348 72 +377~% Depth - 50 246 122 +100% Depth - 3 174 114 +52%
>
> Susant

-- 
sankarshan mukhopadhyay

___

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Rebalance improvement.

2020-08-03 Thread Susant Palai 


> On 03-Aug-2020, at 13:58, Aravinda VK  wrote:
> 
> Interesting numbers. Thanks for the effort.
> 
> What is the unit of old/new numbers? seconds? 

Minutes. 

> 
>> On 03-Aug-2020, at 12:47 PM, Susant Palai > > wrote:
>> 
>> Centos Users can add the following repo and install the build from the 
>> master branch to try out the feature. [Testing purpose only, not ready for 
>> consumption in production env.]
>> 
>> [gluster-nightly-master]
>> baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ 
>> 
>> gpgcheck=0
>> keepalive=1
>> enabled=1
>> repo_gpgcheck = 0
>> name=Gluster Nightly builds (master branch)
>> 
>> A summary of perf numbers from our test lab :
>> 
>> DirSize - 1Million   Old New %diff
>> Depth- 100 (Run 1)   353 74  +377%
>> Depth- 100 (Run 2)   348 72  +377~%
>> Depth- 50246 122 +100%
>> Depth- 3 174 114 +52%
>> 
>> Susant
>> 
>> 
>> On Mon, Aug 3, 2020 at 11:16 AM Susant Palai > > wrote:
>> Hi,
>> Recently, we have pushed some performance improvements for Rebalance 
>> Crawl which used to consume a significant amount of time, out of the entire 
>> rebalance process.
>> 
>> 
>> The patch [1] is recently merged in upstream and may land as an experimental 
>> feature in the upcoming upstream release.
>> 
>> The improvement currently works only for pure-distribute Volume. (which can 
>> be expanded).
>> 
>> 
>> Things to look forward to in future :
>>  - Parallel Crawl in Rebalance
>>  - Global Layout
>> 
>> Once these improvements are in place, we would be able to reduce the overall 
>> rebalance time by a significant time.
>> 
>> Would request our community to try out the feature and give us feedback.
>> 
>> More information regarding the same will follow.
>> 
>> 
>> Thanks & Regards,
>> Susant Palai
>> 
>> 
>> [1] https://review.gluster.org/#/c/glusterfs/+/24443/ 
>> 
>> 
>> 
>> 
>> Community Meeting Calendar:
>> 
>> Schedule -
>> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
>> Bridge: https://bluejeans.com/441850968 
>> 
>> Gluster-users mailing list
>> gluster-us...@gluster.org 
>> https://lists.gluster.org/mailman/listinfo/gluster-users
> 
> Aravinda Vishwanathapura
> https://kadalu.io 
> 
> 
> 

___

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel

Re: [Gluster-devel] [Gluster-users] Rebalance improvement.

2020-08-03 Thread Aravinda VK 
Interesting numbers. Thanks for the effort.

What is the unit of old/new numbers? seconds? 

> On 03-Aug-2020, at 12:47 PM, Susant Palai  wrote:
> 
> Centos Users can add the following repo and install the build from the master 
> branch to try out the feature. [Testing purpose only, not ready for 
> consumption in production env.]
> 
> [gluster-nightly-master]
> baseurl=http://artifacts.ci.centos.org/gluster/nightly/master/7/x86_64/ 
> 
> gpgcheck=0
> keepalive=1
> enabled=1
> repo_gpgcheck = 0
> name=Gluster Nightly builds (master branch)
> 
> A summary of perf numbers from our test lab :
> 
> DirSize - 1MillionOld New %diff
> Depth - 100 (Run 1)   353 74  +377%
> Depth - 100 (Run 2)   348 72  +377~%
> Depth - 50246 122 +100%
> Depth - 3 174 114 +52%
> 
> Susant
> 
> 
> On Mon, Aug 3, 2020 at 11:16 AM Susant Palai  > wrote:
> Hi,
> Recently, we have pushed some performance improvements for Rebalance 
> Crawl which used to consume a significant amount of time, out of the entire 
> rebalance process.
> 
> 
> The patch [1] is recently merged in upstream and may land as an experimental 
> feature in the upcoming upstream release.
> 
> The improvement currently works only for pure-distribute Volume. (which can 
> be expanded).
> 
> 
> Things to look forward to in future :
>  - Parallel Crawl in Rebalance
>  - Global Layout
> 
> Once these improvements are in place, we would be able to reduce the overall 
> rebalance time by a significant time.
> 
> Would request our community to try out the feature and give us feedback.
> 
> More information regarding the same will follow.
> 
> 
> Thanks & Regards,
> Susant Palai
> 
> 
> [1] https://review.gluster.org/#/c/glusterfs/+/24443/ 
> 
> 
> 
> 
> Community Meeting Calendar:
> 
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
> 
> Gluster-users mailing list
> gluster-us...@gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users

Aravinda Vishwanathapura
https://kadalu.io



___

Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968




Gluster-devel mailing list
Gluster-devel@gluster.org
https://lists.gluster.org/mailman/listinfo/gluster-devel