[
https://issues.apache.org/jira/browse/HADOOP-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12636962#action_12636962
]
Hairong Kuang commented on HADOOP-4292:
---------------------------------------
The complexity of append exists when the last checksum chunk is partial. In
this case, I plan to first read the partial chunk out, then truncate the data
file to exclude the last chunk and truncate the crc file to exclude the last 4
bytes. Afterwards write the partial chunk again and data to be append.
Since LocalFileSystem is a ChecksumFileSystem, I plan to implement the above
algorithm in ChecksumFileSystem so it is general to any checksum file system.
For any checksum file system to support append, its raw file system needs to
implement a TruncatableFileSystem interface that supports a truncate method:
FSDataOutputStream truncate(Path file, long len);
This method sets the give file's length be the given len and returns a
FSDataOutputStream, whose current position is equal to the given len.
> append() does not work for LocalFileSystem
> ------------------------------------------
>
> Key: HADOOP-4292
> URL: https://issues.apache.org/jira/browse/HADOOP-4292
> Project: Hadoop Core
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.18.0, 0.19.0
> Reporter: Raghu Angadi
> Assignee: Hairong Kuang
> Priority: Blocker
> Fix For: 0.18.2, 0.19.0
>
>
> append is supported by LocalFileSystem but it does not update crc when a file
> is appended.
> When you enable checksum verification {{TestLocalFileSystem.testAppend}}
> fails. Since HADOOP-4277 is a blocker for 0.17 I am planning to disable this
> test in HADOOP-4277.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
