[
https://issues.apache.org/jira/browse/HADOOP-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15090450#comment-15090450
]
Hudson commented on HADOOP-12678:
---------------------------------
FAILURE: Integrated in Hadoop-trunk-Commit #9076 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9076/])
HADOOP-12678. Handle empty rename pending metadata file during atomic
(cnauroth: rev f0fa6d869b9abb5a900ea1c9eb4eb19ec9831dc4)
*
hadoop-tools/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/NativeAzureFileSystemBaseTest.java
*
hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java
* hadoop-common-project/hadoop-common/CHANGES.txt
> Handle empty rename pending metadata file during atomic rename in redo path
> ---------------------------------------------------------------------------
>
> Key: HADOOP-12678
> URL: https://issues.apache.org/jira/browse/HADOOP-12678
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Reporter: madhumita chakraborty
> Assignee: madhumita chakraborty
> Priority: Critical
> Fix For: 2.8.0
>
> Attachments: HADOOP-12678.001.patch, HADOOP-12678.002.patch,
> HADOOP-12678.003.patch, HADOOP-12678.004.patch, HADOOP-12678.005.patch,
> HADOOP-12678.006.patch
>
>
> Handle empty rename pending metadata file during atomic rename in redo path
> During atomic rename we create metadata file for rename(-renamePending.json).
> We create that in 2 steps
> 1. We create an empty blob corresponding to the .json file in its real
> location
> 2. We create a scratch file to which we write the contents of the rename
> pending which is then copied over into the blob described in 1
> If process crash occurs after step 1 and before step 2 is complete - we will
> be left with a zero size blob corresponding to the pending rename metadata
> file.
> This kind of scenario can happen in the /hbase/.tmp folder because it is
> considered a candidate folder for atomic rename. Now when HMaster starts up
> it executes listStatus on the .tmp folder to clean up pending data. At this
> stage due to the lazy pending rename complete process we look for these json
> files. On seeing an empty file the process simply throws a fatal exception
> assuming something went wrong.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
