[
https://issues.apache.org/jira/browse/HDFS-8531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaoyu Yao updated HDFS-8531:
-----------------------------
Affects Version/s: 2.7.0
> Append failed due to unreleased lease from previous appender with quota
> exceeded exception
> ------------------------------------------------------------------------------------------
>
> Key: HDFS-8531
> URL: https://issues.apache.org/jira/browse/HDFS-8531
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.0
> Reporter: Xiaoyu Yao
> Assignee: Xiaoyu Yao
>
> Append operation fails if
> 1. Set SpaceQuota to 3G for /user/hrt_qa/heterogenous
> 2. Copy 1GB file to /user/hrt_qa/heterogenous
> 3. Run 'hdfs dfs -appendToFile' to append 1GB file to the already copied 1GB
> file in /user/hrt_qa/heterogenous
> 4. Append fails with Quota exceed message
> 5. Increase the Quota to 5G for /user/hrt_qa/heterogenous
> 6. Run the Append message again and the following error message comes.
> {code}
> appendToFile: Failed to APPEND_FILE
> /user/hrt_qa/quotaPerHerterogenousStorage/1GBFile_1/part-m-00000 for
> DFSClient_NONMAPREDUCE_1431576997_1 on 172.31.37.190 because this file lease
> is currently owned by DFSClient_NONMAPREDUCE_-231994503_1 on 172.31.37.190
> {code}
> 7. Wait for a while (lease soft limit -- 1 min) and retry append it succeeds
> after the previous lease expire.
> This seems to relate to HDFS-7587
> "
> When a client was trying to append to the file, the remaining space quota was
> very small. This caused a failure in prepareFileForWrite(), but after the
> inode was already converted for writing and a lease added. Since these were
> not undone when the quota violation was detected, the file was left in
> under-construction with an active lease without edit logging OP_ADD.
> A subsequent append() eventually caused a lease recovery after the soft limit
> period. This resulted in commitBlockSynchronization(), which closed the file
> with OP_CLOSE being logged."
> The fix in HDFS-7587 addressed the edit log corruption issue but does not
> handle the problem within soft limit period where other clients can't append.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
