bharathv commented on a change in pull request #1039: HBASE-23647: Make
MasterRegistry the default impl.
URL: https://github.com/apache/hbase/pull/1039#discussion_r370965852
##########
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/HBaseConfiguration.java
##########
@@ -268,14 +268,19 @@ public static Configuration
createClusterConf(Configuration baseConf, String clu
* used to communicate with distant clusters
* @param conf configuration object to configure
* @param key string that contains the 3 required configuratins
- * @throws IOException
*/
private static void applyClusterKeyToConf(Configuration conf, String key)
- throws IOException{
+ throws IOException {
ZKConfig.ZKClusterKey zkClusterKey = ZKConfig.transformClusterKey(key);
conf.set(HConstants.ZOOKEEPER_QUORUM, zkClusterKey.getQuorumString());
conf.setInt(HConstants.ZOOKEEPER_CLIENT_PORT,
zkClusterKey.getClientPort());
conf.set(HConstants.ZOOKEEPER_ZNODE_PARENT, zkClusterKey.getZnodeParent());
+ // Without the right registry, the above configs are useless. Also, we
don't use setClass()
+ // here because the ConnectionRegistry* classes are not resolvable from
this module.
+ // This will be broken if ZkConnectionRegistry class gets renamed or
moved. Is there a better
+ // way?
Review comment:
Hmm. Good point.
TableMapReduceUtil: This happens only in initCredentialsForCluster() or if
hbase.mapred.output.quorum is specified. Basically it only happens for a "peer"
cluster. (same for TableOutputFormat, unless QUORUM_ADDRESS for a target is
specified this doesn't happen). Overall, I think all MR jobs running on a
single cluster will use master registry. I think that answers the
MR/Spark/Flink usecases.
Now coming to MR jobs spanning multiple clusters (source and target) Ex:
VerifyReplication / ExportSnapshot/SyncTable/TableOutputFormat etc
I think these need to be rewritten with new config params like
hbase.[source|target].master.addrs for clients to pass the addresses so that
they can use master registry.
What do you think? Should we rewrite them with new configs or maintain
compatibility and keep using zkregistry?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
