[
https://issues.apache.org/jira/browse/HADOOP-18149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17714243#comment-17714243
]
Chris Bevard commented on HADOOP-18149:
---------------------------------------
Honestly, I had forgotten about this pull request, then noticed it while I was
opening [HADOOP-18706|https://issues.apache.org/jira/browse/HADOOP-18706]. From
the title I couldn't remember why I had opened it, so I just decided to just
close it.
I remember now though why I created this ticket. I was prototyping Accumulo and
Datawave running on S3 using the S3AFileSystem. Datawave could populate the
distributed cache, but jobs weren't able to read from the cache because of the
way S3 updates the modification times when an object is stored. It was a very
niche use case that probably doesn't warrant a code change.
> The FSDownload verifyAndCopy method doesn't support S3
> ------------------------------------------------------
>
> Key: HADOOP-18149
> URL: https://issues.apache.org/jira/browse/HADOOP-18149
> Project: Hadoop Common
> Issue Type: Improvement
> Reporter: Chris Bevard
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The modification time comparison in FSDownload's verifyAndCopy method fails
> for S3, which prohibits distributed cache files being loaded from S3. This
> change allows S3 to be supported via a config change, that would replace the
> IO Exception with a warning log entry.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
