[jira] [Updated] (HBASE-17786) Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload)
[
https://issues.apache.org/jira/browse/HBASE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Busbey updated HBASE-17786:
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks for this [~uagashe]
(thanks for the additional review [~stack])
> Create LoadBalancer perf-tests (test balancer algorithm decoupled from
> workload)
>
>
> Key: HBASE-17786
> URL: https://issues.apache.org/jira/browse/HBASE-17786
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, proc-v2
>Reporter: stack
>Assignee: Umesh Agashe
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17786.001.patch, HBASE-17786.002.patch,
> HBASE-17786.002.patch
>
>
> (Below is a quote from [~mbertozzi] taken from an internal issue that I'm
> moving out here)
> Add perf tools and keep monitored balancer performance (a BalancerPE-type
> thing).
> Most of the balancers should be instantiable without requiring a
> mini-cluster, and it easy to create tons of RegionInfo and ServerNames with a
> for loop.
> The balancer is just creating a map RegionInfo:ServerName.
> There are two methods to test roundRobinAssignment() and retainAssignment()
> {code}
> Map roundRobinAssignment(
> List regions,
> List servers
> ) throws HBaseIOException;
> Map retainAssignment(
> Map regions,
> List servers
> ) throws HBaseIOException;
> {code}
> There are a bunch of obvious optimization that everyone can see just by
> looking at the code. (like replacing array with set when we do
> contains/remove operations). It will be nice to have a baseline and start
> improving from there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17786) Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload)
[
https://issues.apache.org/jira/browse/HBASE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Umesh Agashe updated HBASE-17786:
-
Attachment: HBASE-17786.002.patch
re-uploading to trigger build.
> Create LoadBalancer perf-tests (test balancer algorithm decoupled from
> workload)
>
>
> Key: HBASE-17786
> URL: https://issues.apache.org/jira/browse/HBASE-17786
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, proc-v2
>Reporter: stack
>Assignee: Umesh Agashe
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17786.001.patch, HBASE-17786.002.patch,
> HBASE-17786.002.patch
>
>
> (Below is a quote from [~mbertozzi] taken from an internal issue that I'm
> moving out here)
> Add perf tools and keep monitored balancer performance (a BalancerPE-type
> thing).
> Most of the balancers should be instantiable without requiring a
> mini-cluster, and it easy to create tons of RegionInfo and ServerNames with a
> for loop.
> The balancer is just creating a map RegionInfo:ServerName.
> There are two methods to test roundRobinAssignment() and retainAssignment()
> {code}
> Map roundRobinAssignment(
> List regions,
> List servers
> ) throws HBaseIOException;
> Map retainAssignment(
> Map regions,
> List servers
> ) throws HBaseIOException;
> {code}
> There are a bunch of obvious optimization that everyone can see just by
> looking at the code. (like replacing array with set when we do
> contains/remove operations). It will be nice to have a baseline and start
> improving from there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17786) Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload)
[
https://issues.apache.org/jira/browse/HBASE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Umesh Agashe updated HBASE-17786:
-
Release Note:
$ bin/hbase
org.apache.hadoop.hbase.master.balancer.LoadBalancerPerformanceEvaluation -help
usage: hbase
org.apache.hadoop.hbase.master.balancer.LoadBalancerPerformanceEvaluation
Options:
-regions Number of regions to consider by load balancer.
Default: 100
-servers Number of servers to consider by load balancer.
Default: 1000
-load_balancerType of Load Balancer to use. Default:
org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer
> Create LoadBalancer perf-tests (test balancer algorithm decoupled from
> workload)
>
>
> Key: HBASE-17786
> URL: https://issues.apache.org/jira/browse/HBASE-17786
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, proc-v2
>Reporter: stack
>Assignee: Umesh Agashe
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17786.001.patch, HBASE-17786.002.patch
>
>
> (Below is a quote from [~mbertozzi] taken from an internal issue that I'm
> moving out here)
> Add perf tools and keep monitored balancer performance (a BalancerPE-type
> thing).
> Most of the balancers should be instantiable without requiring a
> mini-cluster, and it easy to create tons of RegionInfo and ServerNames with a
> for loop.
> The balancer is just creating a map RegionInfo:ServerName.
> There are two methods to test roundRobinAssignment() and retainAssignment()
> {code}
> Map roundRobinAssignment(
> List regions,
> List servers
> ) throws HBaseIOException;
> Map retainAssignment(
> Map regions,
> List servers
> ) throws HBaseIOException;
> {code}
> There are a bunch of obvious optimization that everyone can see just by
> looking at the code. (like replacing array with set when we do
> contains/remove operations). It will be nice to have a baseline and start
> improving from there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17786) Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload)
[
https://issues.apache.org/jira/browse/HBASE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Umesh Agashe updated HBASE-17786:
-
Attachment: HBASE-17786.002.patch
Thanks [~busbey] for the review. I have done the changes per your comments.
Please see the 002 patch.
> Create LoadBalancer perf-tests (test balancer algorithm decoupled from
> workload)
>
>
> Key: HBASE-17786
> URL: https://issues.apache.org/jira/browse/HBASE-17786
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, proc-v2
>Reporter: stack
>Assignee: Umesh Agashe
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17786.001.patch, HBASE-17786.002.patch
>
>
> (Below is a quote from [~mbertozzi] taken from an internal issue that I'm
> moving out here)
> Add perf tools and keep monitored balancer performance (a BalancerPE-type
> thing).
> Most of the balancers should be instantiable without requiring a
> mini-cluster, and it easy to create tons of RegionInfo and ServerNames with a
> for loop.
> The balancer is just creating a map RegionInfo:ServerName.
> There are two methods to test roundRobinAssignment() and retainAssignment()
> {code}
> Map roundRobinAssignment(
> List regions,
> List servers
> ) throws HBaseIOException;
> Map retainAssignment(
> Map regions,
> List servers
> ) throws HBaseIOException;
> {code}
> There are a bunch of obvious optimization that everyone can see just by
> looking at the code. (like replacing array with set when we do
> contains/remove operations). It will be nice to have a baseline and start
> improving from there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17786) Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload)
[
https://issues.apache.org/jira/browse/HBASE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Umesh Agashe updated HBASE-17786:
-
Status: Patch Available (was: In Progress)
Added perf-test tool for benchmarking different LoadBalancer implementations.
Tested from command line.
> Create LoadBalancer perf-tests (test balancer algorithm decoupled from
> workload)
>
>
> Key: HBASE-17786
> URL: https://issues.apache.org/jira/browse/HBASE-17786
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, proc-v2
>Reporter: stack
>Assignee: Umesh Agashe
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17786.001.patch
>
>
> (Below is a quote from [~mbertozzi] taken from an internal issue that I'm
> moving out here)
> Add perf tools and keep monitored balancer performance (a BalancerPE-type
> thing).
> Most of the balancers should be instantiable without requiring a
> mini-cluster, and it easy to create tons of RegionInfo and ServerNames with a
> for loop.
> The balancer is just creating a map RegionInfo:ServerName.
> There are two methods to test roundRobinAssignment() and retainAssignment()
> {code}
> Map roundRobinAssignment(
> List regions,
> List servers
> ) throws HBaseIOException;
> Map retainAssignment(
> Map regions,
> List servers
> ) throws HBaseIOException;
> {code}
> There are a bunch of obvious optimization that everyone can see just by
> looking at the code. (like replacing array with set when we do
> contains/remove operations). It will be nice to have a baseline and start
> improving from there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (HBASE-17786) Create LoadBalancer perf-tests (test balancer algorithm decoupled from workload)
[
https://issues.apache.org/jira/browse/HBASE-17786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Umesh Agashe updated HBASE-17786:
-
Attachment: HBASE-17786.001.patch
Perf-test to tool for LoadBalancer algorithms.
> Create LoadBalancer perf-tests (test balancer algorithm decoupled from
> workload)
>
>
> Key: HBASE-17786
> URL: https://issues.apache.org/jira/browse/HBASE-17786
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, proc-v2
>Reporter: stack
>Assignee: Umesh Agashe
> Labels: beginner
> Fix For: 2.0.0
>
> Attachments: HBASE-17786.001.patch
>
>
> (Below is a quote from [~mbertozzi] taken from an internal issue that I'm
> moving out here)
> Add perf tools and keep monitored balancer performance (a BalancerPE-type
> thing).
> Most of the balancers should be instantiable without requiring a
> mini-cluster, and it easy to create tons of RegionInfo and ServerNames with a
> for loop.
> The balancer is just creating a map RegionInfo:ServerName.
> There are two methods to test roundRobinAssignment() and retainAssignment()
> {code}
> Map roundRobinAssignment(
> List regions,
> List servers
> ) throws HBaseIOException;
> Map retainAssignment(
> Map regions,
> List servers
> ) throws HBaseIOException;
> {code}
> There are a bunch of obvious optimization that everyone can see just by
> looking at the code. (like replacing array with set when we do
> contains/remove operations). It will be nice to have a baseline and start
> improving from there.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
