[
https://issues.apache.org/jira/browse/HDFS-7943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367953#comment-14367953
]
Jing Zhao commented on HDFS-7943:
---------------------------------
I kindly of have an initial patch fixing the issue following #1. Then I find
that more places have the assumption that the block's size upper limit is the
preferred size. E.g., when conversion between UC block and complete block
happens, we always follow this assumption to update quota usage. Also, when
computing storage usage of a UC file, we use the preferred size as the size of
the last UC block. These places require fixes as well.
Currently I prefer #2 because of its simplicity. Also I think it is good to
have the preferred block size as the upper limit. This upper limit is also kind
of respected by the client while writing data.
What do you think, [~szetszwo]?
> Append cannot handle the last block with length greater than the preferred
> block size
> -------------------------------------------------------------------------------------
>
> Key: HDFS-7943
> URL: https://issues.apache.org/jira/browse/HDFS-7943
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 2.7.0
> Reporter: Jing Zhao
> Assignee: Jing Zhao
> Priority: Blocker
> Attachments: HDFS-7943.000.patch
>
>
> In HDFS-3689, we remove the restriction from concat that all the source files
> should have the same preferred block size with the target file. This can
> cause a file to contain blocks with size larger than its preferred block size.
> If such block happens to be the last block of a file, and later we append
> data to the file without the {{CreateFlag.NEW_BLOCK}} flag (i.e., appending
> data to the last block), looks like the current client code will keep writing
> to the last block and never allocate a new block.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
