[
https://issues.apache.org/jira/browse/HADOOP-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16918669#comment-16918669
]
Thomas Demoor commented on HADOOP-16193:
----------------------------------------
[~ehiggs] is correct.
On AWS, the multipart put, the overwriting regular put and the delete might hit
different servers, causing temporary inconsistencies. This is documented here:
[https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel
|https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel]Evidently,
strong consistency solves this problem.
What's the purpose of this test? Are we concerned about racing uploads on the
same name?
> add extra S3A MPU test to see what happens if a file is created during the MPU
> ------------------------------------------------------------------------------
>
> Key: HADOOP-16193
> URL: https://issues.apache.org/jira/browse/HADOOP-16193
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Fix For: 3.1.3
>
>
> Proposed extra test for the S3A MPU: if you create and then delete a file
> while an MPU is in progress, when you finally complete the MPU the new data
> is present.
> This verifies that the other FS operations don't somehow cancel the
> in-progress upload, and that eventual consistency brings the latest value out.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
