[
https://issues.apache.org/jira/browse/HDDS-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022510#comment-17022510
]
Steve Loughran commented on HDDS-2935:
--------------------------------------
bq. Checksum calculations of local ozone files should be better similar to
whatever s3 is already doing/returning.
no, it's an etag with no guarantees.
for a cloudy distcp you should just publish your own checksum and have a
version of distcp which tracks the source and dest checksums from the last
upload rather than simply assume source == dest.
look at the various s3a etag/checksum JIRAs to see how our adding a checksum
broke distcp for anyone who didnt have -skipCrC on their CLI.
> Support for getFileChecksum in OzoneFS
> --------------------------------------
>
> Key: HDDS-2935
> URL: https://issues.apache.org/jira/browse/HDDS-2935
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Reporter: Srinivasu Majeti
> Priority: Major
>
> Support for getFileChecksum() and any other way to help distcp to avoid the
> copy of duplicate files even when the length is the same that of remote
> storage (cloud copy to s3). Checksum calculations of local ozone files should
> be better similar to whatever s3 is already doing/returning.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
