[
https://issues.apache.org/jira/browse/HADOOP-18706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719804#comment-17719804
]
ASF GitHub Bot commented on HADOOP-18706:
-----------------------------------------
steveloughran commented on PR #5563:
URL: https://github.com/apache/hadoop/pull/5563#issuecomment-1536085389
thanks, merged to trunk. can you cherrypick locally to branch-3.3 rerun the
tests and then put that up as a new PR? I will then merge that.
FWIW I still think what you are trying to do for recovery is "bold", the
rock climbing "I wonder if they will survive" meaning of the word, but the
filename tracking could be useful in other ways.
> The temporary files for disk-block buffer aren't unique enough to recover
> partial uploads.
> -------------------------------------------------------------------------------------------
>
> Key: HADOOP-18706
> URL: https://issues.apache.org/jira/browse/HADOOP-18706
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs/s3
> Reporter: Chris Bevard
> Assignee: Chris Bevard
> Priority: Minor
> Labels: pull-request-available
> Fix For: 3.4.0
>
>
> If an application crashes during an S3ABlockOutputStream upload, it's
> possible to complete the upload if fast.upload.buffer is set to disk by
> uploading the s3ablock file with putObject as the final part of the multipart
> upload. If the application has multiple uploads running in parallel though
> and they're on the same part number when the application fails, then there is
> no way to determine which file belongs to which object, and recovery of
> either upload is impossible.
> If the temporary file name for disk buffering included the s3 key, then every
> partial upload would be recoverable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
