[
https://issues.apache.org/jira/browse/HBASE-26196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shinya Yoshida updated HBASE-26196:
-----------------------------------
Description:
We introduced support to generate hfile with good locality for a remote cluster
even in HBASE-25608.
I realized we need to override other configurations for the remote cluster in
addition to the zookeeper cluster key.
For example, read from a non-secure cluster and write hfiles for a secure
cluster.
In this case, we use TableInputFormat for non-secure cluster with
hbase.security.authentication=simple in job configuration.
So HFileOutputFormat failed to connect to remote secure cluster because
requires hbase.security.authentication=kerberos in job conf.
Thus let's introduce configuration override for remote-cluster-aware
HFileOutputFormat locality-sensitive feature.
-Another example is to read from a secure cluster (A) and write hfiles for
another secure cluster (B) and we use different principal for each cluster.-
-For instance, we use cluster-a/[email protected] for A and
cluster-b/[email protected] for B.-
-Then we need to override MASTER_KRB_PRINCIPAL and REGIONSERVER_KRB_PRINCIPAL
using cluster-b/[email protected] to connect cluster B.-
^ This is not truth, we use token based digest auth in MR job, so this should
be fine
was:
We introduced support to generate hfile with good locality for a remote cluster
even in HBASE-25608.
I realized we need to override other configurations for the remote cluster in
addition to the zookeeper cluster key.
For example, read from a non-secure cluster and write hfiles for a secure
cluster.
In this case, we use TableInputFormat for non-secure cluster with
hbase.security.authentication=simple in job configuration.
So HFileOutputFormat failed to connect to remote secure cluster because
requires hbase.security.authentication=kerberos in job conf.
Another example is to read from a secure cluster (A) and write hfiles for
another secure cluster (B) and we use different principal for each cluster.
For instance, we use cluster-a/[email protected] for A and
cluster-b/[email protected] for B.
Then we need to override MASTER_KRB_PRINCIPAL and REGIONSERVER_KRB_PRINCIPAL
using cluster-b/[email protected] to connect cluster B.
Thus let's introduce configuration override for remote-cluster-aware
HFileOutputFormat locality-sensitive feature.
> Support configuration override for remote cluster of HFileOutputFormat
> locality sensitive
> -----------------------------------------------------------------------------------------
>
> Key: HBASE-26196
> URL: https://issues.apache.org/jira/browse/HBASE-26196
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 1.8.0, 3.0.0-alpha-2, 2.4.5
> Reporter: Shinya Yoshida
> Assignee: Shinya Yoshida
> Priority: Major
>
> We introduced support to generate hfile with good locality for a remote
> cluster even in HBASE-25608.
> I realized we need to override other configurations for the remote cluster in
> addition to the zookeeper cluster key.
> For example, read from a non-secure cluster and write hfiles for a secure
> cluster.
> In this case, we use TableInputFormat for non-secure cluster with
> hbase.security.authentication=simple in job configuration.
> So HFileOutputFormat failed to connect to remote secure cluster because
> requires hbase.security.authentication=kerberos in job conf.
>
> Thus let's introduce configuration override for remote-cluster-aware
> HFileOutputFormat locality-sensitive feature.
>
> -Another example is to read from a secure cluster (A) and write hfiles for
> another secure cluster (B) and we use different principal for each cluster.-
> -For instance, we use cluster-a/[email protected] for A and
> cluster-b/[email protected] for B.-
> -Then we need to override MASTER_KRB_PRINCIPAL and
> REGIONSERVER_KRB_PRINCIPAL using cluster-b/[email protected] to connect
> cluster B.-
> ^ This is not truth, we use token based digest auth in MR job, so this should
> be fine
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
