[
https://issues.apache.org/jira/browse/HDFS-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13187256#comment-13187256
]
Todd Lipcon commented on HDFS-2691:
-----------------------------------
In order to fix this, we need to get the {{targets}} list for incomplete blocks
replicated to the SBN. The most straightforward way to do this involves the DNs
somehow reporting the RBW replicas to both the SBN and the ANN. When the NN
receives a reported RBW replica, it adds to the targets list for an incomplete
block, thus allowing a recovery to succeed even after fail-over.
Two potential ways to get the RBW replicas reported:
*Solution 1*: whenever any pipeline is opened on the DNs, it reports the RBW
replica to the NNs -- similar to how it reports FINALIZED replicas on close.
This would be queued to both the active and standby namenodes. Advantage:
timely reporting of replicas. Disadvantage: adds slightly to NN load even in
common case.
*Solution 2*: when the DN detects that a failover has occurred to a new Active,
it reports all RBW replicas to the new active at that point. Advantage: doesn't
add to non-failover load. Disadvantage: there is some complexity in that there
is a window where the new active is in active state but hasn't yet heard from
all the DNs. Thus, it will serve incorrect reads or incorrectly handle a lease
recovery for a short window of time (since it doesn't know any locations for
the last block).
Given the potential for user-visible incorrect results for a window of time in
solution 2, I plan to pursue solution 1, unless someone has a better idea.
> HA: Tests and fixes for pipeline targets and replica recovery
> -------------------------------------------------------------
>
> Key: HDFS-2691
> URL: https://issues.apache.org/jira/browse/HDFS-2691
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Attachments: hdfs-2691.txt
>
>
> Currently there are some TODOs around pipeline/recovery code in the HA
> branch. For example, commitBlockSynchronization only gets sent to the active
> NN which may have failed over by that point. So, we need to write some tests
> here and figure out what the correct behavior is.
> Another related area is the treatment of targets in the pipeline. When a
> pipeline is created, the active NN adds the "expected locations" to the
> BlockInfoUnderConstruction, but the DN identifiers aren't logged with the
> OP_ADD. So after a failover, the BlockInfoUnderConstruction will have no
> targets and I imagine replica recovery would probably trigger some issues.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
