[ https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677392#comment-16677392 ]
Erik Krogen edited comment on HDFS-14017 at 11/6/18 11:07 PM: -------------------------------------------------------------- Cool, I like the v004 patch, glad we were able to get on the same page! Sorry for jumping to conclusions about your intentions earlier. I have a few comments on the patch but looking good overall: * Can we extend the Javadoc of {{ObserverReadProxyProviderWithIPFailover}} to explain the logic more fully (CFPP/ORPP-like behavior for the observers, IPFPP-like behavior for the active)? * Seems like there is some duplication between the two {{initializeProxy}} methods. The main difference is just how {{nameNodeProxies}} is constructed. How about the following, which I think makes it more clear what the actual differences are: ** We have a {{initializeNameNodeProxies()}} method that is overridden in ORPPWithIPFailover. ** The fields you made {{protected}} can go back to being {{private}} ** The shared logic can be in ORPP ** I think {{cloneDelegationTokenForHA()}} can occur within the overridden {{initializeNameNodeProxies()}} ? * Shouldn't use star import in ORPPWithIPFailover * Do you have a plan to solve the TODO in {{ORPPWithIPFailover#getConfiguredAddresses()}}? It seems that the key in the {{addressList}} map should be able to be used to solve this? was (Author: xkrogen): * Can we extend the Javadoc of {{ObserverReadProxyProviderWithIPFailover}} to explain the logic more fully (CFPP/ORPP-like behavior for the observers, IPFPP-like behavior for the active)? * Seems like there is some duplication between the two {{initializeProxy}} methods. The main difference is just how {{nameNodeProxies}} is constructed. How about the following, which I think makes it more clear what the actual differences are: ** We have a {{initializeNameNodeProxies()}} method that is overridden in ORPPWithIPFailover. ** The fields you made {{protected}} can go back to being {{private}} ** The shared logic can be in ORPP ** I think {{cloneDelegationTokenForHA()}} can occur within the overridden {{initializeNameNodeProxies()}} ? * Shouldn't use star import in ORPPWithIPFailover * Do you have a plan to solve the TODO in {{ORPPWithIPFailover#getConfiguredAddresses()}}? It seems that the key in the {{addressList}} map should be able to be used to solve this? > ObserverReadProxyProviderWithIPFailover should work with HA configuration > ------------------------------------------------------------------------- > > Key: HDFS-14017 > URL: https://issues.apache.org/jira/browse/HDFS-14017 > Project: Hadoop HDFS > Issue Type: Sub-task > Reporter: Chen Liang > Assignee: Chen Liang > Priority: Major > Attachments: HDFS-14017-HDFS-12943.001.patch, > HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch, > HDFS-14017-HDFS-12943.004.patch > > > Currently {{ObserverReadProxyProviderWithIPFailover}} extends > {{ObserverReadProxyProvider}}, and the only difference is changing the proxy > factory to use {{IPFailoverProxyProvider}}. However this is not enough > because when calling constructor of {{ObserverReadProxyProvider}} in > super(...), the follow line: > {code:java} > nameNodeProxies = getProxyAddresses(uri, > HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY); > {code} > will try to resolve the all configured NN addresses to do configured > failover. But in the case of IPFailover, this does not really apply. > > A second issue closely related is about delegation token. For example, in > current IPFailover setup, say we have a virtual host nn.xyz.com, which points > to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS, > there is always only one DT being exchanged, which has hostname nn.xyz.com. > Server only issues this DT, and client only knows the host nn.xyz.com, so all > is good. But in Observer read, even with IPFailover, the client will no > longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and > nn2.xyz.com. During this process, current code will look for DT associated > with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT > given by NN. causing Token authentication to fail. This happens in > {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy > provider will need to resolve this as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org