[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689892#comment-16689892
]
Erik Krogen edited comment on HDFS-14017 at 11/16/18 7:02 PM:
--
Even stranger now: This time around it claims that everything passed, but it
only ran 84 tests, whereas it should be around 6000+. Even more strange, it
doesn't report that it is skipping them, it seems to think those 6000 tests
don't exist... Not sure where to go from here.
I think it's pretty safe to say we didn't break anything existing considering
that we are only modifying a completely new class, so I'm still +1 on
committing.
was (Author: xkrogen):
Even stranger now: This time around it claims that everything passed, but it
only ran 84 tests, whereas it should be around 6000+. Even more strange, it
doesn't report that it is skipping them, it seems to think those 6000 tests
don't exist... Not sure where to go from here.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch,
> HDFS-14017-HDFS-12943.009.patch, HDFS-14017-HDFS-12943.010.patch,
> HDFS-14017-HDFS-12943.011.patch, HDFS-14017-HDFS-12943.012.patch,
> HDFS-14017-HDFS-12943.013.patch, HDFS-14017-HDFS-12943.014.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689764#comment-16689764
]
Erik Krogen edited comment on HDFS-14017 at 11/16/18 5:58 PM:
--
Hm.. Something is pretty wrong with Jenkins. It's not actually running any of
the tests, failing with errors like:
{code}
[ERROR] ExecutionException The forked VM terminated without properly saying
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-client &&
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx2048m
-XX:+HeapDumpOnOutOfMemoryError -jar
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/surefire/surefirebooter375579229167329239.jar
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/surefire
2018-11-16T17-42-57_928-jvmRun1 surefire586051240617267363tmp
surefire_07291752059438872666tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
{code}
It looks like the last patch where it actually ran tests was v009. We were
seeing the same issue on HDFS-14035, but I don't see it on other Jenkins runs
against trunk (rather than the HDFS-12943 branch).
[~vagarychen], I see you didn't merge in trunk after committing HDFS-14035, I'm
going to do so now and then re-run Jenkins and see if things get better.
was (Author: xkrogen):
Hm.. Something is pretty wrong with Jenkins. It's not actually running any of
the tests, failing with errors like:
{code}
[ERROR] ExecutionException The forked VM terminated without properly saying
goodbye. VM crash or System.exit called?
[ERROR] Command was /bin/sh -c cd
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-client &&
/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx2048m
-XX:+HeapDumpOnOutOfMemoryError -jar
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/surefire/surefirebooter375579229167329239.jar
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs-client/target/surefire
2018-11-16T17-42-57_928-jvmRun1 surefire586051240617267363tmp
surefire_07291752059438872666tmp
[ERROR] Error occurred in starting fork, check output in log
[ERROR] Process Exit Code: 1
{code}
It looks like the last patch where it actually ran tests was v009. We were
seeing the same issue on HDFS-14035, but I don't see it on other Jenkins runs
against trunk (rather than the HDFS-12943 branch).
[~chliang], I see you didn't merge in trunk after committing HDFS-14035, I'm
going to do so now and then re-run Jenkins and see if things get better.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch,
> HDFS-14017-HDFS-12943.009.patch, HDFS-14017-HDFS-12943.010.patch,
> HDFS-14017-HDFS-12943.011.patch, HDFS-14017-HDFS-12943.012.patch,
> HDFS-14017-HDFS-12943.013.patch, HDFS-14017-HDFS-12943.014.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
>
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16688316#comment-16688316
]
Erik Krogen edited comment on HDFS-14017 at 11/15/18 4:37 PM:
--
Hey [~vagarychen], I think the new approach looks great! My comments are below.
* {{DFS_CLIENT_FAILOVER_IPFAILOVER_VIRTUAL_ADDRESS}} should probably live
within {{HdfsClientConfigKeys.Failover}}. Then it can also make use of
{{Failover.PREFIX}}.
* The change within {{ObserverReadProxyProvider}} is no longer necessary.
* The Javadoc on the ORPPWithIPFailover constructor doesn't make sense, it says
that IPFPP is the default for ORPP
* {{ORPPWithIPFailover#getProxyAddresses}} is looking the same as
{{AbstractNNFailoverProxyProvider#getProxyAddresses}}, is there any reason that
we need to override this? It seems that besides the new {{virtual-address}}
config, the rest of the configuration between ORPP and ORPPWithIPFailover is
the same, we should be able to just re-use this logic?
was (Author: xkrogen):
Hey [~vagarychen], I think the new approach looks great! My comments are below.
* {{DFS_CLIENT_FAILOVER_IPFAILOVER_VIRTUAL_ADDRESS}} should probably live
within {{HdfsClientConfigKeys.Failover}}. Then it can also make use of
{{Failover.PREFIX}}.
* The change within {{ObserverReadProxyProvider}} is no longer necessary.
* The Javadoc on the ORPPWithIPFailover constructor doesn't make sense, it says
that IPFPP is the default for ORPP
* {{ORPPWithIPFailover#getProxyAddresses}} is looking the same as
{{AbstractNNFailoverProxyProvider}}, is there any reason that we need to
override this? It seems that besides the new {{virtual-address}} config, the
rest of the configuration between ORPP and ORPPWithIPFailover is the same, we
should be able to just re-use this logic?
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch,
> HDFS-14017-HDFS-12943.009.patch, HDFS-14017-HDFS-12943.010.patch,
> HDFS-14017-HDFS-12943.011.patch, HDFS-14017-HDFS-12943.012.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685962#comment-16685962
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/14/18 1:06 AM:
--
For the record, my assumptions in the comment above were incorrect based on my
recent evaluation of the state of the art. Here is how IPFailoverPP is
configured:
{code:java}
// Client uses only these two lines from core-site.xml
fs.defaultFS = virtual-address-nn.in.com:8020
dfs.client.failover.proxy.provider.virtual-address-nn.in.com =
o.a.h...IPFailoverProxyProvider
// Standard HA configuration for the NameNode in hdfs-site-xml
dfs.nameservices = my-cluster
dfs.ha.namenodes.my-cluster = nn1, nn2
dfs.namenode.rpc-address.my-cluster.nn1 = physical-address-ha1.in.com:8020
dfs.namenode.rpc-address.my-cluster.nn2 = physical-address-ha2.in.com:8020
{code}
>From HDFS-6334 I understand IPFPP was intentionally made to look like it talks
>to a single NameNode. Which looks hacky now. We have multiple NameNodes and
>the Proxy provider is in control which NN it should direct the call, so using
>NN's logical name (aka nameserviceID) seems the right way for newly developed
>proxy providers. We should still support current way for IPFPP for backward
>compatibility, so be it.
For ORPPwithIPF we still need to know virtual address for NameNode failover. I
suggest we add a new parameter for that, adding it to the config above:
{code:java}
dfs.client.failover.ipfailover.virtual-address.my-cluster =
virtual-address-nn.in.com:8020
{code}
So the ORPP part will use {{dfs.nameservices}} to obtain physical addresses of
NNs, and the IPF part will instantiate IPFPP based on
{{dfs.client.failover.ipfailover.virtual-address}} parameter.
And we can still support traditional IPFPP (without Observer) using current
{{core-site.xml}} configuration.
was (Author: shv):
For the record, my assumptions in the comment above were incorrect based on my
recent evaluation of the state of the art. Here is how IPFailoverPP is
configured:
{code:java}
// Client uses only these two lines from core-site.xml
fs.defaultFS = virtual-address-nn.in.com:8020
dfs.client.failover.proxy.provider.virtual-address-nn.in.com =
o.a.h...IPFailoverProxyProvider
// Standard HA configuration for the NameNode in hdfs-site-xml
dfs.nameservices = my-cluster
dfs.ha.namenodes.my-cluster = nn1, nn2
dfs.namenode.rpc-address.my-cluster.nn1 = physical-address-ha1.in.com:8020
dfs.namenode.rpc-address.my-cluster.nn2 = physical-address-ha2.in.com:8020
{code}
>From HDFS-6334 I understand IPFPP was intentionally made to look like it talks
>to a single node. Which looks hacky now. We have multiple NameNodes and the
>Proxy provider is in control which NN it should direct the call, so using NN's
>logical name (aka nameserviceID) seems the right way for newly developed proxy
>providers. We should still support current way for IPFPP for backward
>compatibility, so be it.
For ORPPwithIPF we still need to know virtual address for NameNode failover. I
suggest we add a new parameter for that, adding it to the config above:
{code:java}
dfs.client.failover.ipfailover.virtual-address.my-cluster =
virtual-address-nn.in.com:8020
{code}
So the ORPP part will use {{dfs.nameservices}} to obtain physical addresses of
NNs, and the IPF part will instantiate IPFPP based on
{{dfs.client.failover.ipfailover.virtual-address}} parameter.
And we can still support traditional IPFPP (without Observer) using current
{{core-site.xml}} configuration.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch,
> HDFS-14017-HDFS-12943.009.patch, HDFS-14017-HDFS-12943.010.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16685962#comment-16685962
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/14/18 1:06 AM:
--
For the record, my assumptions in [the comment
above|https://issues.apache.org/jira/browse/HDFS-14017?focusedCommentId=16682588=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16682588]
were incorrect based on my recent evaluation of the state of the art. Here is
how IPFailoverPP is configured:
{code:java}
// Client uses only these two lines from core-site.xml
fs.defaultFS = virtual-address-nn.in.com:8020
dfs.client.failover.proxy.provider.virtual-address-nn.in.com =
o.a.h...IPFailoverProxyProvider
// Standard HA configuration for the NameNode in hdfs-site-xml
dfs.nameservices = my-cluster
dfs.ha.namenodes.my-cluster = nn1, nn2
dfs.namenode.rpc-address.my-cluster.nn1 = physical-address-ha1.in.com:8020
dfs.namenode.rpc-address.my-cluster.nn2 = physical-address-ha2.in.com:8020
{code}
>From HDFS-6334 I understand IPFPP was intentionally made to look like it talks
>to a single NameNode. Which looks hacky now. We have multiple NameNodes and
>the Proxy provider is in control which NN it should direct the call, so using
>NN's logical name (aka nameserviceID) seems the right way for newly developed
>proxy providers. We should still support current way for IPFPP for backward
>compatibility, so be it.
For ORPPwithIPF we still need to know virtual address for NameNode failover. I
suggest we add a new parameter for that, adding it to the config above:
{code:java}
dfs.client.failover.ipfailover.virtual-address.my-cluster =
virtual-address-nn.in.com:8020
{code}
So the ORPP part will use {{dfs.nameservices}} to obtain physical addresses of
NNs, and the IPF part will instantiate IPFPP based on
{{dfs.client.failover.ipfailover.virtual-address}} parameter.
And we can still support traditional IPFPP (without Observer) using current
{{core-site.xml}} configuration.
was (Author: shv):
For the record, my assumptions in the comment above were incorrect based on my
recent evaluation of the state of the art. Here is how IPFailoverPP is
configured:
{code:java}
// Client uses only these two lines from core-site.xml
fs.defaultFS = virtual-address-nn.in.com:8020
dfs.client.failover.proxy.provider.virtual-address-nn.in.com =
o.a.h...IPFailoverProxyProvider
// Standard HA configuration for the NameNode in hdfs-site-xml
dfs.nameservices = my-cluster
dfs.ha.namenodes.my-cluster = nn1, nn2
dfs.namenode.rpc-address.my-cluster.nn1 = physical-address-ha1.in.com:8020
dfs.namenode.rpc-address.my-cluster.nn2 = physical-address-ha2.in.com:8020
{code}
>From HDFS-6334 I understand IPFPP was intentionally made to look like it talks
>to a single NameNode. Which looks hacky now. We have multiple NameNodes and
>the Proxy provider is in control which NN it should direct the call, so using
>NN's logical name (aka nameserviceID) seems the right way for newly developed
>proxy providers. We should still support current way for IPFPP for backward
>compatibility, so be it.
For ORPPwithIPF we still need to know virtual address for NameNode failover. I
suggest we add a new parameter for that, adding it to the config above:
{code:java}
dfs.client.failover.ipfailover.virtual-address.my-cluster =
virtual-address-nn.in.com:8020
{code}
So the ORPP part will use {{dfs.nameservices}} to obtain physical addresses of
NNs, and the IPF part will instantiate IPFPP based on
{{dfs.client.failover.ipfailover.virtual-address}} parameter.
And we can still support traditional IPFPP (without Observer) using current
{{core-site.xml}} configuration.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch,
> HDFS-14017-HDFS-12943.009.patch, HDFS-14017-HDFS-12943.010.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682588#comment-16682588
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/10/18 9:08 PM:
--
I meant nameserviceID, rather than namespace. Here is an example:
{code:java}
dfs.nameservices = virtual-address-nn.g.li.com
dfs.ha.namenodes.virtual-address-nn.g.li.com = nn1, nn2
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn1 =
physical-address-ha1.g.li.com
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn2 =
physical-address-ha2.g.li.com
{code}
Here {{virtual-address-nn.g.li.com}} plays the role of the nameserviceID, which
turns out to be the virtual address of NN. Does your patch works with that
configuration? We should really unit test this.
So for ORPPwithIPF, the IPF will use {{virtual-address-nn.g.li.com}} as the
NN's virtual address, while the ORPP part will work with the physical addresses
{{physical-address-ha*.g.li.com}}.
was (Author: shv):
I meant nameserviceID, rather than namespace. Here is an example:
{code:java}
dfs.nameservices = virtual-address-nn.g.li.com
dfs.ha.namenodes.virtual-address-nn.g.li.com = nn1, nn2
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn1 =
physical-address-ha1.g.li.com
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn2 =
physical-address-ha2.g.li.com
{code}
Here {{virtual-address-nn.g.li.com}} plays the role of the nameserviceID, which
turns out to be the virtual address of NN. Does your patch works with that
configuration? We should really unit test this.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682168#comment-16682168
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/10/18 8:52 PM:
--
Thanks for sharing [~shv].
What did you mean by "the virtual address of the NameName in current IFPP comes
from the _nameserviceID_"? How's that different from the current patch? maybe
an example? And yes, current IPProxy does not care about physical addresses,
that's the whole point of it. But in observer read, then are to we able to talk
to different NNs without knowing physical addresses, or are we only talking
about the failover part here?
cc. [~xkrogen]
was (Author: vagarychen):
Thanks for sharing [~shv].
What did you mean by "the virtual address of the NameName in current IFPP comes
from the namespaceID"? How's that different from the current patch? maybe an
example? And yes, current IPProxy does not care about physical addresses,
that's the whole point of it. But in observer read, then are to we able to talk
to different NNs without knowing physical addresses, or are we only talking
about the failover part here?
cc. [~xkrogen]
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682145#comment-16682145
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/10/18 8:51 PM:
--
Had an offline discussion with Erik. The gist of the problem is that the
virtual address of the NameName in current IFPP comes from the _nameserviceID_,
and it doesn't look further into physical addresses of the NameNodes. So if we
want to keep this behavior, we should do the same for ORPPWithIPF.
I suggest that we double-check this is the case. If so, Erik says it is, let's
go this route, even though it seems hacky, but let's finally document this
properly.
was (Author: shv):
Had an offline discussion with Erik. The gist of the problem is that the
virtual address of the NameName in current IFPP comes from the namespaceID, and
it doesn't look further into physical addresses of the NameNodes. So if we want
to keep this behavior, we should do the same for ORPPWithIPF.
I suggest that we double-check this is the case. If so, Erik says it is, let's
go this route, even though it seems hacky, but let's finally document this
properly.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678668#comment-16678668
]
Chen Liang edited comment on HDFS-14017 at 11/7/18 7:22 PM:
Hmm...I don't see why it does not work, as long as both nn1 and nn2 shows up in
{{dfs.namenode.rpc-address.\*.\*}} configs, there will be two proxies created,
and client should be able to find them. I did a quick check in the test
cluster, seemed to work, although it was checking from command line and won't
tell about DT.
I agree that with all the {{dfs.namenode.rpc-address}} configs we have all
information. In fact, in current patch, the call to
{{DFSUtilClient.getAddresses(...)}} is already getting ALL the host physical
addresses for ALL the nameservices. All these addresses will have proxies
created and DT cloned. The issue I had was that, it may not be true that all
these addresses should be considered (and clone DT for) in case of federation.
Probably should be based on different {{fs.defaultFS}}, which is what
IPFailover is largely relying on, maybe only a subset should be checked. This
is what I meant by the need to tie fs.defaultFS to some name service/physical
address. And this, needs to be answered under the context of how IPFailover
should work with Federation.
was (Author: vagarychen):
Hmm...I don't see it does not work, as long as both nn1 and nn2 shows up in
{{dfs.namenode.rpc-address.\*.\*}} configs, there will be two proxies created,
and client should be able to find them. I did a quick in the test cluster, seem
to worked, although it was checking from command line and won't tell about DT.
I agree that with all the {{dfs.namenode.rpc-address}} configs we have all
information. In fact, in current patch, the call to
{{DFSUtilClient.getAddresses(...)}} is already getting ALL the host physical
addresses for ALL the nameservices. All these addresses will have proxies
created and DT cloned. The issue I had was that, it may not be true that all
these addresses should be considered (and clone DT for) in case of federation.
Probably should be based on different {{fs.defaultFS}}, which is what
IPFailover is largely relying on, maybe only a subset should be checked. This
is what I meant by the need to tie fs.defaultFS to some name service/physical
address. And this, needs to be answered under the context of how IPFailover
should work with Federation.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678601#comment-16678601
]
Chen Liang edited comment on HDFS-14017 at 11/7/18 6:23 PM:
What did you by a non-default FS? I meant the config fs.defaultFS, shouldn't it
always be there?
I think {{dfs.namenode.rpc-address.}} values are not VIP but actual
addresses, this was partly the reason I started making change to
{{ObserverReadProxyProviderWithIPFailover}}. Currently the code only checks
{{dfs.namenode.rpc-address.}} configs when defaultFS is determined a logic URI,
which is not the case for IPFailover, changing this seemed to affect many
places if I remember correctly (at least many tests will fail). In configured
setup, fs.defaultFS is a nameservice, and code will automatically replace nnID
as in {{dfs.namenode.rpc-address.}} with that fs.defaultFS nameservice
id. In IPFailover case, fs.defaultFS is not logical so
{{dfs.namenode.rpc-address.*}} are ignored completely. In our environment,
there is configured address for nameservice (which are for Datanodes). This is
where I needed to tie them together. Also, I have been trying to not explicitly
match URIs because VIP can be anything in theory.
was (Author: vagarychen):
What did you by a non-default FS? I meant the config fs.defaultFS, shouldn't it
always be there?
I think {{dfs.namenode.rpc-address.}} values are not VIP but actual
addresses, this was partly the reason I started making change to
{{ObserverReadProxyProviderWithIPFailover}}. Currently the code only checks
{{dfs.namenode.rpc-address.*}} configs when defaultFS is determined a logic
URI, which is not the case for IPFailover, changing this seemed to affect many
places if I remember correctly (at least many tests will fail). In configured
setup, fs.defaultFS is a nameservice, and code will automatically replace nnID
as in {{dfs.namenode.rpc-address.}} with that fs.defaultFS nameservice
id. In IPFailover case, fs.defaultFS is not logical so
{{dfs.namenode.rpc-address.*}} are ignored completely. In our environment,
there is configured address for nameservice (which are for Datanodes). This is
where I needed to tie them together. Also, I have been trying to not explicitly
match URIs because VIP can be anything in theory.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677392#comment-16677392
]
Erik Krogen edited comment on HDFS-14017 at 11/6/18 11:08 PM:
--
Cool, I like the v004 patch, glad we were able to get on the same page! Sorry
for jumping to conclusions about your intentions earlier. I have a few comments
on the patch but looking good overall:
* Can we extend the Javadoc of {{ObserverReadProxyProviderWithIPFailover}} to
explain the logic more fully (CFPP/ORPP-like behavior for the observers,
IPFPP-like behavior for the active)?
* Seems like there is some duplication between the two {{initializeProxy}}
methods. The main difference is just how {{nameNodeProxies}} is constructed.
How about the following, which I think makes it more clear what the actual
differences are:
** We have a {{protected List> initializeNameNodeProxies()}}
method that is overridden in ORPPWithIPFailover.
** The fields you made {{protected}} can go back to being {{private}}
** The shared logic can be in ORPP
** I think {{cloneDelegationTokenForHA()}} can occur within the overridden
{{initializeNameNodeProxies()}} ?
* Shouldn't use star import in ORPPWithIPFailover
* Do you have a plan to solve the TODO in
{{ORPPWithIPFailover#getConfiguredAddresses()}}? It seems that the key in the
{{addressList}} map should be able to be used to solve this?
was (Author: xkrogen):
Cool, I like the v004 patch, glad we were able to get on the same page! Sorry
for jumping to conclusions about your intentions earlier. I have a few comments
on the patch but looking good overall:
* Can we extend the Javadoc of {{ObserverReadProxyProviderWithIPFailover}} to
explain the logic more fully (CFPP/ORPP-like behavior for the observers,
IPFPP-like behavior for the active)?
* Seems like there is some duplication between the two {{initializeProxy}}
methods. The main difference is just how {{nameNodeProxies}} is constructed.
How about the following, which I think makes it more clear what the actual
differences are:
** We have a {{initializeNameNodeProxies()}} method that is overridden in
ORPPWithIPFailover.
** The fields you made {{protected}} can go back to being {{private}}
** The shared logic can be in ORPP
** I think {{cloneDelegationTokenForHA()}} can occur within the overridden
{{initializeNameNodeProxies()}} ?
* Shouldn't use star import in ORPPWithIPFailover
* Do you have a plan to solve the TODO in
{{ORPPWithIPFailover#getConfiguredAddresses()}}? It seems that the key in the
{{addressList}} map should be able to be used to solve this?
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail:
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677392#comment-16677392
]
Erik Krogen edited comment on HDFS-14017 at 11/6/18 11:07 PM:
--
Cool, I like the v004 patch, glad we were able to get on the same page! Sorry
for jumping to conclusions about your intentions earlier. I have a few comments
on the patch but looking good overall:
* Can we extend the Javadoc of {{ObserverReadProxyProviderWithIPFailover}} to
explain the logic more fully (CFPP/ORPP-like behavior for the observers,
IPFPP-like behavior for the active)?
* Seems like there is some duplication between the two {{initializeProxy}}
methods. The main difference is just how {{nameNodeProxies}} is constructed.
How about the following, which I think makes it more clear what the actual
differences are:
** We have a {{initializeNameNodeProxies()}} method that is overridden in
ORPPWithIPFailover.
** The fields you made {{protected}} can go back to being {{private}}
** The shared logic can be in ORPP
** I think {{cloneDelegationTokenForHA()}} can occur within the overridden
{{initializeNameNodeProxies()}} ?
* Shouldn't use star import in ORPPWithIPFailover
* Do you have a plan to solve the TODO in
{{ORPPWithIPFailover#getConfiguredAddresses()}}? It seems that the key in the
{{addressList}} map should be able to be used to solve this?
was (Author: xkrogen):
* Can we extend the Javadoc of {{ObserverReadProxyProviderWithIPFailover}} to
explain the logic more fully (CFPP/ORPP-like behavior for the observers,
IPFPP-like behavior for the active)?
* Seems like there is some duplication between the two {{initializeProxy}}
methods. The main difference is just how {{nameNodeProxies}} is constructed.
How about the following, which I think makes it more clear what the actual
differences are:
** We have a {{initializeNameNodeProxies()}} method that is overridden in
ORPPWithIPFailover.
** The fields you made {{protected}} can go back to being {{private}}
** The shared logic can be in ORPP
** I think {{cloneDelegationTokenForHA()}} can occur within the overridden
{{initializeNameNodeProxies()}} ?
* Shouldn't use star import in ORPPWithIPFailover
* Do you have a plan to solve the TODO in
{{ORPPWithIPFailover#getConfiguredAddresses()}}? It seems that the key in the
{{addressList}} map should be able to be used to solve this?
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
