[
https://issues.apache.org/jira/browse/HDFS-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoyu Yao updated HDFS-8531:
-----------------------------
Attachment: HDFS-8531.00.patch
Attach an initial patch that handles lease recovery upon append failure after
partial block for DFSClient that is the original lease holder and that is not
the original lease holder.
> Append failed due to unreleased lease from previous appender with quota
> exceeded exception
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-8531
> URL: https://issues.apache.org/jira/browse/HDFS-8531
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.7.0
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
> Attachments: HDFS-8531.00.patch
>
>
> Append operation fails if
> 1. Set SpaceQuota to 3G for /user/hrt_qa/heterogenous
> 2. Copy 1GB file to /user/hrt_qa/heterogenous
> 3. Run 'hdfs dfs -appendToFile' to append 1GB file to the already copied 1GB
> file in /user/hrt_qa/heterogenous
> 4. Append fails with Quota exceed message
> 5. Increase the Quota to 5G for /user/hrt_qa/heterogenous
> 6. Run the Append message again and the following error message comes.
> {code}
> appendToFile: Failed to APPEND_FILE
> /user/hrt_qa/quotaPerHerterogenousStorage/1GBFile_1/part-m-00000 for
> DFSClient_NONMAPREDUCE_1431576997_1 on 172.31.37.190 because this file lease
> is currently owned by DFSClient_NONMAPREDUCE_-231994503_1 on 172.31.37.190
> {code}
> 7. Wait for a while (lease soft limit -- 1 min) and retry append it succeeds
> after the previous lease expire.
> This seems to relate to HDFS-7587
> "
> When a client was trying to append to the file, the remaining space quota was
> very small. This caused a failure in prepareFileForWrite(), but after the
> inode was already converted for writing and a lease added. Since these were
> not undone when the quota violation was detected, the file was left in
> under-construction with an active lease without edit logging OP_ADD.
> A subsequent append() eventually caused a lease recovery after the soft limit
> period. This resulted in commitBlockSynchronization(), which closed the file
> with OP_CLOSE being logged."
> The fix in HDFS-7587 addressed the edit log corruption issue but does not
> handle the problem within soft limit period where other clients can't append.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
