[
https://issues.apache.org/jira/browse/HBASE-10827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhaojianbo updated HBASE-10827:
-------------------------------
Description:
In our online cluster, usually there are multiple ethernet cards in one
machine, one for outer network, one for inner network. But the current version
of HBase can not use all of them which waste the network bandwidth of one
ethernet card. If we make HBase use multiple ethernet cards concurrently, the
performance of HBase will be improved.
So I did the work, and test a simple scenario:
8 clients scan the same region data from a different machine with two ethernet
cards.(machine of regionserver also with two ethernet cards)
The Environment is:
* I start HBase cluster with a master, a regionserver, a zookeeper in a machine.
* HDFS cluster with a Namenode, a datanode, a secondary namenode is also
started in the same machine.
* 8 client run on different machine.
* all data local
* 22GB data size
I measure the performance before and after the optimization.
The results are:
||client||time before optimization||time after optimization||
| 8 | 1665.07s | 1242.45s |
The patch is uploaded. What I did is the following:
# create new RPC getAllServerAddress which obtain all the addresses of
regionserver
# client call the RPC to obtain the addresses, choose one of them randomly,
validate the address and use the address as the regionLocation address
# add a cache serverAddressMap to avoid redundant RPC.
was:
In our online cluster, usually there are multiple ethernet cards in one
machine, one for outer network, one for inner network. But the current version
of HBase can not use all of them which waste the network bandwidth of one
ethernet card. If we make HBase use multiple ethernet cards concurrently, the
performance of HBase will be improved.
So I did the work, and test a simple scenario:
8 clients scan the same region data from a different machine with two ethernet
cards.(machine of regionserver also with two ethernet cards)
The Environment is:
* I start HBase cluster with a master, a regionserver, a zookeeper in a machine.
* HDFS cluster with a Namenode, a datanode, a secondary namenode is also
started in the same machine.
* 8 client run on different machine.
* all data local
* 22GB data size
I measure the performance before and after the optimization.
The results are:
||client||time before optimization||time after optimization||
| 8 | 1702.03s | 1242.45s |
The patch is uploaded. What I did is the following:
# create new RPC getAllServerAddress which obtain all the addresses of
regionserver
# client call the RPC to obtain the addresses, choose one of them randomly,
validate the address and use the address as the regionLocation address
# add a cache serverAddressMap to avoid redundant RPC.
> Making HBase use multiple ethernet cards will improve the performance
> ---------------------------------------------------------------------
>
> Key: HBASE-10827
> URL: https://issues.apache.org/jira/browse/HBASE-10827
> Project: HBase
> Issue Type: New Feature
> Affects Versions: 0.99.0
> Reporter: zhaojianbo
> Attachments: HBASE-10827-0.98-branch.patch
>
>
> In our online cluster, usually there are multiple ethernet cards in one
> machine, one for outer network, one for inner network. But the current
> version of HBase can not use all of them which waste the network bandwidth of
> one ethernet card. If we make HBase use multiple ethernet cards concurrently,
> the performance of HBase will be improved.
> So I did the work, and test a simple scenario:
> 8 clients scan the same region data from a different machine with two
> ethernet cards.(machine of regionserver also with two ethernet cards)
> The Environment is:
> * I start HBase cluster with a master, a regionserver, a zookeeper in a
> machine.
> * HDFS cluster with a Namenode, a datanode, a secondary namenode is also
> started in the same machine.
> * 8 client run on different machine.
> * all data local
> * 22GB data size
> I measure the performance before and after the optimization.
> The results are:
> ||client||time before optimization||time after optimization||
> | 8 | 1665.07s | 1242.45s |
> The patch is uploaded. What I did is the following:
> # create new RPC getAllServerAddress which obtain all the addresses of
> regionserver
> # client call the RPC to obtain the addresses, choose one of them randomly,
> validate the address and use the address as the regionLocation address
> # add a cache serverAddressMap to avoid redundant RPC.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
