[
https://issues.apache.org/jira/browse/HADOOP-17999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17444793#comment-17444793
]
Abhishek Das commented on HADOOP-17999:
---------------------------------------
The initial implementation in the patch was to keep track of whether
verifyChecksum or writeChecksum values are set or not, and pass on the values
when the child filesystem is initialized.
[~shv] suggested that ViewFileSystem should not set the verifyChecksum or
writeChecksum flags for its child filesystems. This goes with the
implementation in ViewFs where both *setWriteChecksum* and *setVerifyChecksum*
are no-op.
We still want to keep overriding the two methods in ViewFileSystem (same as
ViewFs) so that we explicitly say these are no-ops and doesn't depend on the
FileSystem implementation (if anything changes in future)
[~shv] thoughts on this ?
> ViewFileSystem.setVerifyChecksum should not initialize all target filesystems
> -----------------------------------------------------------------------------
>
> Key: HADOOP-17999
> URL: https://issues.apache.org/jira/browse/HADOOP-17999
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Abhishek Das
> Assignee: Abhishek Das
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently setVerifyChecksum and setWriteChecksum initializes all target file
> systems which causes delay in hadoop shell copy commands such as get, put,
> copyFromLocal etc.
> This also eventually causes OOM.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
