[
https://issues.apache.org/jira/browse/HADOOP-12678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-12678:
-----------------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.8.0
Status: Resolved (was: Patch Available)
+1 for patch v06. I have committed this to trunk, branch-2 and branch-2.8.
[~madhuch-ms], thank you for contributing this patch.
> Handle empty rename pending metadata file during atomic rename in redo path
> ---------------------------------------------------------------------------
>
> Key: HADOOP-12678
> URL: https://issues.apache.org/jira/browse/HADOOP-12678
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Reporter: madhumita chakraborty
> Assignee: madhumita chakraborty
> Priority: Critical
> Fix For: 2.8.0
>
> Attachments: HADOOP-12678.001.patch, HADOOP-12678.002.patch,
> HADOOP-12678.003.patch, HADOOP-12678.004.patch, HADOOP-12678.005.patch,
> HADOOP-12678.006.patch
>
>
> Handle empty rename pending metadata file during atomic rename in redo path
> During atomic rename we create metadata file for rename(-renamePending.json).
> We create that in 2 steps
> 1. We create an empty blob corresponding to the .json file in its real
> location
> 2. We create a scratch file to which we write the contents of the rename
> pending which is then copied over into the blob described in 1
> If process crash occurs after step 1 and before step 2 is complete - we will
> be left with a zero size blob corresponding to the pending rename metadata
> file.
> This kind of scenario can happen in the /hbase/.tmp folder because it is
> considered a candidate folder for atomic rename. Now when HMaster starts up
> it executes listStatus on the .tmp folder to clean up pending data. At this
> stage due to the lazy pending rename complete process we look for these json
> files. On seeing an empty file the process simply throws a fatal exception
> assuming something went wrong.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
