madhumita chakraborty created HADOOP-12678:
----------------------------------------------
Summary: Handle empty rename pending metadata file during atomic
rename in redo path
Key: HADOOP-12678
URL: https://issues.apache.org/jira/browse/HADOOP-12678
Project: Hadoop Common
Issue Type: Bug
Components: fs/azure
Reporter: madhumita chakraborty
Assignee: madhumita chakraborty
Priority: Critical
Handle empty rename pending metadata file during atomic rename in redo path
During atomic rename we create metadata file for rename(-renamePending.json).
We create that in 2 steps
1. We create an empty blob corresponding to the .json file in its real location
2. We create a scratch file to which we write the contents of the rename
pending which is then copied over into the blob described in 1
If process crash occurs after step 1 and before step 2 is complete - we will be
left with a zero size blob corresponding to the pending rename metadata file.
This kind of scenario can happen in the /hbase/.tmp folder because it is
considered a candidate folder for atomic rename. Now when HMaster starts up it
executes listStatus on the .tmp folder to clean up pending data. At this stage
due to the lazy pending rename complete process we look for these json files.
On seeing an empty file the process simply throws a fatal exception assuming
something went wrong.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
