[
https://issues.apache.org/jira/browse/HDFS-839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12805659#action_12805659
]
Konstantin Shvachko commented on HDFS-839:
------------------------------------------
Eli and Todd asked for the notes from the meeting on HA. I'll post them here in
order to avoid recursive requests.
My understanding of the current state of the HA issue is that there is a lot of
unanswered questions and problems.
Passing block locations to BN raised by Wang in this issue is one of them.
Todd mentions the problem of lease recovery: how the recovery of unfinished
writes is done in the HA world.
Another problem mentioned by Dhruba is the race condition in removing replicas
of a block when control switches from primary to backup, as both of them can
decide to remove (potentially different) replicas of the same block.
If you know other problems please share.
I see there is a big interest in HA from many people / organizations.
Unfortunately, I don't see a real proposal, which attempts to answer these and
other potential problems.
In my personal opinion we should do HA in 3-4 steps:
1. Start from relatively simple task of implementing manual warm standby (see
above for classification). This can be implemented in 2-4 weeks.
2a, 2b. From there HDFS can evolve in either direction: automatic warm standby,
or manual hot standby. The latter (2b) is Facebook's priority as I understand.
3. After that the last two (2a+2b) technologies can be merged into automatic
hot standby.
> The NameNode should forward block reports to BackupNode
> -------------------------------------------------------
>
> Key: HDFS-839
> URL: https://issues.apache.org/jira/browse/HDFS-839
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: name-node
> Reporter: dhruba borthakur
> Assignee: dhruba borthakur
>
> The BackupNode (via HADOOP-4539) receives a stream of transactions from
> NameNode. However, the BackupNode does not have block locations of blocks. It
> would be nice if the NameNode can forward all block reports (that it receives
> from DataNodes) to the BackupNode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
