[
https://issues.apache.org/jira/browse/HDFS-4452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17238894#comment-17238894
]
Konstantin Shvachko commented on HDFS-4452:
-------------------------------------------
This is an interesting observation [~honestman]. You are right in your scenario
the block creation will fail and the client will have to retry either
re-writing just the last block or the entire file. The good thing is that the
namespace remains in a consistent state. Which was the problem with the
original issue in this jira.
This is essentially a scenario for "Case 3" of {{analyzeFileState()}}. It would
be good to confirm with a unit test this is indeed possible. NameNode should
not violate the contract of persisting all the data that was successfully
reported to clients.
> getAdditionalBlock() can create multiple blocks if the client times out and
> retries.
> ------------------------------------------------------------------------------------
>
> Key: HDFS-4452
> URL: https://issues.apache.org/jira/browse/HDFS-4452
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.0.2-alpha
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Critical
> Fix For: 2.0.3-alpha
>
> Attachments: TestAddBlockRetry.java,
> getAdditionalBlock-branch2.patch, getAdditionalBlock.patch,
> getAdditionalBlock.patch, getAdditionalBlock.patch
>
>
> HDFS client tries to addBlock() to a file. If NameNode is busy the client can
> timeout and will reissue the same request again. The two requests will race
> with each other in {{FSNamesystem.getAdditionalBlock()}}, which can result in
> creating two new blocks on the NameNode while the client will know of only
> one of them. This eventually results in {{NotReplicatedYetException}} because
> the extra block is never reported by any DataNode, which stalls file creation
> and puts it in invalid state with an empty block in the middle.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
