[
https://issues.apache.org/jira/browse/HDFS-2979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2979:
---------------------------------
Attachment: HDFS-2979-HDFS-1623.patch
Thanks a lot for the review, Eli. Here's an updated patch which addresses your
feedback.
In the course of creating this patch, I also discovered a problem with the NN's
mutation of conf objects passed to it. The NN calls initializeGenericKeys in a
few places, which causes conf objects passed to it to be mutated. This causes a
problem if a client provides a configuration object to an NN that it doesn't
expect to be mutated, and basically results in a misconfigured conf object from
the POV of the client. To fix this, I made the NN make copies of conf objects
provided to it in a few places. This could obviously be broken out into a
separate JIRA, but it's pretty small so I figured I'd just do it here.
bq. The new getNNUris method seems overloaded, eg it only gets a URI for each
configured NN or a logical URI for an HA NS if if you pass it keys. But always
returns the URI of the default fs, which may be an alias for one of the NN URIs
already in the set. Let's make this just return the NN URIs for the keys like
getNNServiceRpcAddresses and people who just want the default URI from the conf
can use getDefaultUri instead of calling this w/o keys. People who want both
(can't think of any who should) can add the default URI to the set.
OK, done.
bq. Worth mentioning in the javadoc that for HA NNs, unlike federated NNs,
we're explicitly skipping the specific NNs and just using the logical NNs.
I revised the comment and also changed the name of the method to
getNameServiceUris, since I think that's generally more clear.
bq. getNNUris should use HdfsConstants#HDFS_URI_SCHEME
Done.
bq. testGetNNUris needs HA-enabled and non-HA-enabled cases
This is already tested, since the configuration which is tested includes
multiple name services, one HA-enabled and the other not.
I also ran the following tests to verify this change, and they all passed:
TestDNFencing,TestDNFencingWithReplication,TestEditLogsDuringFailover,TestEditLogTailer,TestFailureOfSharedDir,TestFailureToReadEdits,TestHAConfiguration,TestHAFsck,TestHAMetrics,TestHASafeMode,TestHAStateTransitions,TestHAWebUI,TestPipelinesFailover,TestQuotasWithHA,TestStandbyCheckpoints,TestStandbyIsHot,TestDFSClientFailover,TestBalancerWithHANameNodes,TestDFSUtil,TestBalancer,TestBalancerWithMultipleNameNodes
> HA: Balancer should use logical uri for creating failover proxy with HA
> enabled.
> --------------------------------------------------------------------------------
>
> Key: HDFS-2979
> URL: https://issues.apache.org/jira/browse/HDFS-2979
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: balancer, hdfs client
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Uma Maheswara Rao G
> Assignee: Aaron T. Myers
> Attachments: HDFS-2979-HDFS-1623.patch, HDFS-2979-HDFS-1623.patch,
> HDFS-2979.patch
>
>
> Presently Balancer uses real URI for creating the failover proxy.
> Since the failover proxy checks for uri consistency, we should pass logical
> uri for creating failover proxy instead of instead of real URI. Presently
> will work only with default port.
> java.io.IOException: Port 49832 specified in URI hdfs://127.0.0.1:49832 but
> host '127.0.0.1' is a logical (HA) namenode and does not use port information.
> at
> org.apache.hadoop.hdfs.HAUtil.getFailoverProxyProviderClass(HAUtil.java:224)
> at org.apache.hadoop.hdfs.HAUtil.createFailoverProxy(HAUtil.java:247)
> at
> org.apache.hadoop.hdfs.server.balancer.NameNodeConnector.<init>(NameNodeConnector.java:80)
> at
> org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:1401)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
