[
https://issues.apache.org/jira/browse/HBASE-18124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liubangchen updated HBASE-18124:
--------------------------------
Description:
Hbase only have one way to locate hmaster or hregionserver not like hdfs has
two way to locate datanode used by name or hostname.
I'm a engineer of cloud computing , and I'm in charge of to make Hbase as a
cloud service.
Our Hbase cloud service architecture shown as follows: 1.jpg
1.VM
User’s Hbase client work in vm and use virtual ip address to access hbase
cluster.
2.NAT
Network Address Translation, vip(Virtual Network Address) to pip (Physical
Network Address)
3. HbaseCluster Service
HbaseCluster Service work in physical network
Problem
Vm use vip to communicate with hbase cluster, but hbase have only one way
to communicate with each server using host which is setting by parameters
hbase.regionserver.hostname or hbase.master.hostname. When HMaster startup will
publish master address and meta region server address in zookeeper, then the
address is pip(Physical Network Address) . Because hbase cluster work in
physical network. But the problem is that when vm get the address from
zookeeper is pip, not vip. If I set host as vip, it will has problems for hbase
cluster communication inside. so it needs two ways for communication setting by
parameters.
Solution
1. protocol extend
change strcut ServerName as
{code}
message ServerName {
required string host_name = 1;
optional uint32 port = 2;
optional uint64 start_code = 3;
optional string name=4; //new field
}
{code}
It will be publish in zookeeper. We can choose host_name or name
configured by parameters hbase.client.use.hostname
2. metatable extend
Add a column to hbase:meta named info:namelocation . So the original
column info:server configured with hbase.regionserver.hostname, and the new
column info:namelocation configured with hbase.regionserver.servername
3. hbase-server
When regionserver startup, configured hbase.regionserver.hostname as
pip and configured hbase.regionserver.servername as vip. Then
hbase.regionserver.hostname will be writed in ServerName's host_name, and
hbase.regionserver.servername will be writed in ServerName's name.When hmaster
startup, configured hbase.hmaster.hostname as pip and configured
hbase.hmaster.servername as vip. Then hbase.hmaster.hostname will be writed in
ServerName's host_name, and hbase.hmaster.servername will be writed in
ServerName's name.
4. hbase-client
Add a parameters named hbase.client.use.hostname to use vip or pip.
This patch is base on Hbase-1.3.0
was:
Hbase only have one way to locate hmaster or hregionserver not like hdfs has
two way to locate datanode use by name or hostname.
I’m a engineer of cloud computing , and I’m in charge of to make Hbase as a
cloud service,when we make hbase as a cloud service we need hbase support
other way to support locate hmaster or hregionserver
Our Hbase cloud service architectue shown as follows 1.jpg
1.VM
User’s Hbase client work in vm and use virtual ip address to access hbase
cluster.
2.NAT
Network Address Translation, vip(Virtual Network Address) to pip (Physical
Network Address)
3. HbaseCluster Service
HbaseCluster Service work in physical network
Problem
1. View on vm
On vm side vm use vip to communication,but hbase have only one way to
communication use struct named
ServerName. When Hmaster startup will store master address and meta
region server address in zookeeper,
then the address is pip(Physical Network Address) because hbase
cluster work in physical network . when vm
get the address from zookeeper will not work because vm use vip to
communication,one way to solve this is to
make physical machine host as vip like 192.168.0.1,but is not better to
make this.
2. View on Physical machine
Physical machine use pip to communication
Solution
1. protocol extend change proto message to below:
{code}
message ServerName {
required string host_name = 1;
optional uint32 port = 2;
optional uint64 start_code = 3;
optional string name=4;
}
{code}
add a filed named name like hdfs’s datablock location
2. metatable extend
add column to hbase:meta named info:namelocation
3. hbase-server
add params
{code}
hbase.regionserver.servername
<property>
<name>hbase.regionserver.servername</name>
<value>10.0.1.1</value>
</property>
{code}
to regionserver namelocation
add params
{code}
hbase.master.servername
<property>
<name>hbase.master.servername</name>
<value>10.0.1.2</value>
</property>
{code}
to set master namelocation
4. hbase-client
add params
{code}
hbase.client.use.hostname
<property>
<name>hbase.client.use.hostname</name>
<value>true</value>
</property>
{code}
to choose which address to use
This patch is base on Hbase-1.3.0
> Make Hbase Communication Support Virtual Network
> ------------------------------------------------
>
> Key: HBASE-18124
> URL: https://issues.apache.org/jira/browse/HBASE-18124
> Project: HBase
> Issue Type: New Feature
> Components: Client, hbase, master
> Reporter: liubangchen
> Assignee: liubangchen
> Attachments: 1.jpg, HBASE-18124.patch, HBASE-18124.pdf
>
>
> Hbase only have one way to locate hmaster or hregionserver not like hdfs has
> two way to locate datanode used by name or hostname.
> I'm a engineer of cloud computing , and I'm in charge of to make Hbase as a
> cloud service.
> Our Hbase cloud service architecture shown as follows: 1.jpg
> 1.VM
> User’s Hbase client work in vm and use virtual ip address to access hbase
> cluster.
> 2.NAT
> Network Address Translation, vip(Virtual Network Address) to pip
> (Physical Network Address)
> 3. HbaseCluster Service
> HbaseCluster Service work in physical network
> Problem
> Vm use vip to communicate with hbase cluster, but hbase have only one
> way to communicate with each server using host which is setting by parameters
> hbase.regionserver.hostname or hbase.master.hostname. When HMaster startup
> will publish master address and meta region server address in zookeeper,
> then the address is pip(Physical Network Address) . Because hbase cluster
> work in physical network. But the problem is that when vm get the address
> from zookeeper is pip, not vip. If I set host as vip, it will has problems
> for hbase cluster communication inside. so it needs two ways for
> communication setting by parameters.
> Solution
> 1. protocol extend
> change strcut ServerName as
> {code}
> message ServerName {
> required string host_name = 1;
> optional uint32 port = 2;
> optional uint64 start_code = 3;
> optional string name=4; //new field
> }
> {code}
> It will be publish in zookeeper. We can choose host_name or name
> configured by parameters hbase.client.use.hostname
> 2. metatable extend
> Add a column to hbase:meta named info:namelocation . So the original
> column info:server configured with hbase.regionserver.hostname, and the new
> column info:namelocation configured with hbase.regionserver.servername
> 3. hbase-server
> When regionserver startup, configured hbase.regionserver.hostname as
> pip and configured hbase.regionserver.servername as vip. Then
> hbase.regionserver.hostname will be writed in ServerName's host_name, and
> hbase.regionserver.servername will be writed in ServerName's name.When
> hmaster startup, configured hbase.hmaster.hostname as pip and configured
> hbase.hmaster.servername as vip. Then hbase.hmaster.hostname will be writed
> in ServerName's host_name, and hbase.hmaster.servername will be writed in
> ServerName's name.
> 4. hbase-client
> Add a parameters named hbase.client.use.hostname to use vip or pip.
> This patch is base on Hbase-1.3.0
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
