[
https://issues.apache.org/jira/browse/HADOOP-16150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16778569#comment-16778569
]
Eric Yang commented on HADOOP-16150:
------------------------------------
It would be ok for ChecksumFileSystem to fail on concat. Other implementations
extend ChecksumFileSystem must implement their own concat and append logic. At
minimum, check writeChecksum flag to be false, then throw
UnsupportedOperationException to be backward compatible. This ensures that
public API doesn't allow silent corruption to occur unless explicitly set
writeChecksum flag which helps to track down callers that maybe affected by
this change.
> checksumFS doesn't wrap concat(): concatenated files don't have checksums
> -------------------------------------------------------------------------
>
> Key: HADOOP-16150
> URL: https://issues.apache.org/jira/browse/HADOOP-16150
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Priority: Major
>
> Followon from HADOOP-16107. FilterFS passes through the concat operation, and
> checksum FS doesn't override that call -so files created through concat *do
> not have checksums*.
> If people are using a checksummed fs directly with the expectations that they
> will, that expectation is not being met.
> What to do?
> * fail always?
> * fail if checksums are enabled?
> * try and implement the concat operation from raw local up at the checksum
> level
> append() just gives up always; doing the same for concat would be the
> simplest. Again, brings us back to "need a way to see if an FS supports a
> feature before invocation", here checksum fs would reject append and concat
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
