[
https://issues.apache.org/jira/browse/HDFS-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13736778#comment-13736778
]
Vinay commented on HDFS-4516:
-----------------------------
The current issue is more concerned about reading the data which is already
written to complete blocks and the synced data (if any) in the current block.
So what needs to be taken care is :
1. Client should be able to read all the previous blocks
2. Client should be able to read the data in current block if any sync call
from client is success.
3. Recovery of such files should be successful
So, Here is one simple solution,
1. One first sync call of new block, persist the length of the block.
2. While reading if there are no locations available for last under
construction block and the length is 0, this will be considered as crashed
block and will skip it. If length is non-zero, then exception will be thrown.
3. Same as #2, while recovery also, last under construction block can be
removed and file can be closed.
> Client crash after block allocation and NN switch before lease recovery for
> the same file can cause readers to fail forever
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-4516
> URL: https://issues.apache.org/jira/browse/HDFS-4516
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0, 2.0.3-alpha
> Reporter: Uma Maheswara Rao G
> Priority: Critical
> Attachments: HDFS-4516-Test.patch, HDFS-4516.txt
>
>
> If client crashes just after allocating block( blocks not yet created in DNs)
> and NN also switched after this, then new Namenode will not know about locs.
> Further details will be in comment.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
