[
https://issues.apache.org/jira/browse/HDFS-11094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15705818#comment-15705818
]
Eric Badger commented on HDFS-11094:
------------------------------------
{quote}
1. I discussed with Arpit Agarwal offline and he suggested us use the same
logic in updateActorStatesFromHeartbeat to update the active NN
bpServiceToActive, which has dealt with several cases carefully. Moreover, if
we are updating bpServiceToActive we should likely also update
lastActiveClaimTxId. To achieve this, I think we can pass
NNHAStatusHeartbeatProto instead of HAServiceStateProto in NamespaceInfoProto.
{quote}
[~liuml07], I actually did it this way in the patch on purpose. The entire
logic of updating {{bpServiceToActive}} will occur before any heartbeats start,
since we are doing this during the handshake between the DN and the NN. If we
send in an {{NNHAStatusHeartbeatProto}} instead of a {{HAServiceStateProto}}
then we will have to deal with the {{lastActiveClaimTxId}} as you have
mentioned. However, this would require more serious changes to the code, since
we would have to either set and send along a TxId on the NN side (extra code
change for what I see is negligible benefit) or we would need to arbitrarily
create one on the DN side (would need to set it to be below the first heartbeat
TxId, so it would have to be a negative number or would have to make extra
changes).
At this point, we want the DN to have an active before it starts trying to do
anything with it (the whole point of this fix). If, for whatever reason, both
NNs declare themselves as active, then it will choose the first one and ignore
the second. If the wrong assertion is made, then it will talk to the standby
and we will get a simple standby exception and then once the next heartbeat
comes we will update the correct active. So worst case scenario we get a
standby exception and retry, which is still loads better than the NPE that we
were getting before. I think that since this is such a small window that it is
unnecessary to make more changes with the TxId.
[~daryn] may have more thoughts on this.
{quote}
2. For the unit test, can we set a very large heartbeat interval in
configuration, and check the active NN is not null after
cluster.waitForActive()? Mocked tests are useful as well and can be kept.
Another idea is to drop heartbeat request against a spied HeartbeatManager.
{quote}
This should be fairly easy to do. I'll put up a patch shortly with this added
test.
> Send back HAState along with NamespaceInfo during a versionRequest as an
> optional parameter
> -------------------------------------------------------------------------------------------
>
> Key: HDFS-11094
> URL: https://issues.apache.org/jira/browse/HDFS-11094
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: datanode
> Reporter: Eric Badger
> Assignee: Eric Badger
> Attachments: HDFS-11094.001.patch, HDFS-11094.002.patch,
> HDFS-11094.003.patch, HDFS-11094.004.patch, HDFS-11094.005.patch,
> HDFS-11094.006.patch, HDFS-11094.007.patch, HDFS-11094.008.patch,
> HDFS-11094.009.patch
>
>
> The datanode should know which NN is active when it is connecting/registering
> to the NN. Currently, it only figures this out during its first (and
> subsequent) heartbeat(s) and so there is a period of time where the datanode
> is alive and registered, but can't actually do anything because it doesn't
> know which NN is active. A byproduct of this is that the MiniDFSCluster will
> become active before it knows what NN is active, which can lead to NPEs when
> calling getActiveNN().
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
