[
https://issues.apache.org/jira/browse/HBASE-14821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Helmling updated HBASE-14821:
----------------------------------
Attachment: HBASE-14821.patch
This patch adds support for overriding arbitrary config properties on the
destination cluster side when obtaining a delegation token for the destination
clsuter, and when initializing TableOutputFormat. I've been testing this to
override the kerberos principal names for a destination cluster, but it could
be used for other config properties as well.
This adds:
* a static {{HBaseConfiguration.subset(Configuration,String)}} method to extra
the set of configuration properties matching a key prefix, with the prefix
removed from the keys in the returned subset.
* test for the above
* use of the subset method in {{TableMapReduceUtil.initCredentials()}} and
{{TableOutputFormat.setConf()}}, using a prefix of "hbase.mapred.output."
This allows jobs using {{TableOutputFormat}} to use a different set of
configuration values on the output side than those used on the input side.
There are a few caveats with the current implementation that are less than
ideal:
* the prefix "hbase.mapred.output." makes for verbose keys (ie.
"hbase.mapred.output.hbase.regionserver.kerberos.principal"), but it's
consistent with other property keys used by TableOutputFormat.
* There are now two ways of specifying values for the existing TOF keys
matching this prefix -- the current short version (hbase.mapred.output.quorum)
and the new long version (hbase.mapred.output.hbase.zookeeper.quorum). This is
duplicative, but I don't see any reason to remove the short versions since it
would break back compatibility and docs.
* This also means that the short versions get copied to the subset-ed config
with odd names ("quorum", "quorum.port", "rs.class", "rs.impl"), though I don't
think this harms anything.
I did _not_ apply these changes to the o.a.h.h.mapred versions of
TableMapReduceUtil and TableOutputFormat, since those currently lack any of the
existing configuration overrides.
> CopyTable should allow overriding more config properties for peer cluster
> -------------------------------------------------------------------------
>
> Key: HBASE-14821
> URL: https://issues.apache.org/jira/browse/HBASE-14821
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
> Reporter: Gary Helmling
> Assignee: Gary Helmling
> Fix For: 2.0.0, 1.2.0, 1.3.0
>
> Attachments: HBASE-14821.patch
>
>
> When using CopyTable across two separate clusters, you can specify the ZK
> quorum for the destination cluster, but not much else in configuration
> overrides. This can be a problem when the cluster configurations differ,
> such as when using security with different configurations for server
> principals.
> We should provide a general way to override configuration properties for the
> peer / destination cluster. One option would be to allow use of a prefix for
> command line properties ("peer.property."). Properties matching this prefix
> will be stripped and merged to the peer configuration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
