[
https://issues.apache.org/jira/browse/HBASE-29762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18045550#comment-18045550
]
Balazs Meszaros commented on HBASE-29762:
-----------------------------------------
{quote}In our company, we do some magic on the switch so all K8s pods are
accessible by other machines outside the k8s cluster, so we need to do nothing
for support k8s deployed HBase cluster.
{quote}
You can add some networking magic to the cluster, however these magics are
kubernetes implementation specific as far as I know. There can be multiple
_standard_ solutions for this issue, we have tried the followings:
{*}Istio{*}: Istio has several features, we can use istio-gateway for HBase. It
can work in Layer4 (TCP) layer, so no need to modify the HBase protocol itself.
In this operation mode, Istio can be configured to open several ports (1 port
for 1 RS) and based on a routing logic, it forwards the requests to the right
RS. Let's assume the Istio gateway address is {{{}10.10.10.10{}}}, then the RS
server addresses for external clients should be {{{}10.10.10.10:10000{}}},
{{{}10.10.10.10:10001{}}}, ... based on the configuration. The drawback of this
approach is it opens a new TCP connection inside the cluster for every incoming
connection, so the performance will be slightly slower.
{*}NodePort services{*}: it is also possible to create a {{NodePort}} service
for every RS. So let's assume a RS is running on host {{10.10.10.10}} then the
external RS address will be {{10.10.10.10:31234}} for example. The benefit of
this approach is it possibly using DNAT in this case, so the performance should
be better than the previous case.
{*}LoadBalancer services{*}: it is also possible to use {{LoadBalancer}}
instead of {{{}NodePort{}}}. In this case you might need N Load Balancer IP
address if you have N RegionServers, however the port number can be {{16020}}
in this case. So with some DNS magic outside of kubernetes you might use the
"same" addresses for external and internal clients.
This issue tries to address *Istio* and *NodePort* deployments. These
approaches are common in one thing. The external address will be different from
the internal address.
{quote}And as you said, a pod's ip/hostname can not be used outside the k8s
cluster, then I do not get the point why we can fix this by adding a
translation in client connection header? How does the client write the
hbase-site.xml for connecting to the cluster?
{quote}
The translation between the external and internal addresses will be done by a
co-processor (HBASE-29766). It ensures the returned addresses by client-meta
RPC endpoint and {{hbase:meta}} scans return the external addresses. This
translation will be inactive by default, but if the client has
{{hbase.client.header.kubernetes}} connection header for example (HBASE-29765),
then the co-processor will be active. So the {{hbase-site.xml}} should contain
{{hbase.client.header.kubernetes=true}} for example for external clients.
{quote}Which node does the client connect to when setting up the connection?
{quote}
Since every HBase node can be a bootstrap node, it can use any of them using
their external address.
> Support external client connection to kubernetized HBase cluster
> ----------------------------------------------------------------
>
> Key: HBASE-29762
> URL: https://issues.apache.org/jira/browse/HBASE-29762
> Project: HBase
> Issue Type: New Feature
> Reporter: Balazs Meszaros
> Assignee: Balazs Meszaros
> Priority: Major
>
> When HBase is deployed to kubernetes, kubernetes assigns its own hostnames to
> HBase pods. It works fine while the clients are in the same kubernetes
> environment, but can be an issue if they are coming from outside the
> kubernetes environment. Kubernetes offers several methods for exposing
> services like Istio, Load Balancers, ... Any method can be used, but these
> methods share one thing: the external addresses are different from the
> internal addresses.
> The goal is to support external HBase client connection to kubernetized HBase
> clusters. The idea is to implement a co-processor which translates the
> internal addresses to external addresses. This translation is optional based
> on a client connection header (which is coming from a configuration property).
> This task sums up these efforts:
> * [HBASE-29763] Implement co-processor host for client-meta (Zookeeper-less
> HBase discovery) service.
> * [HBASE-29764] Make client connection header attributes accessible inside
> co-processors.
> * [HBASE-29765] Make client connection header attributes configurable.
> * [HBASE-29766] Implement a co-processor which intercepts client-meta calls,
> {{hbase:meta}} table gets/scans and replaces internal addresses with external
> addresses.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
