[
https://issues.apache.org/jira/browse/HADOOP-16193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ewan Higgs reopened HADOOP-16193:
---------------------------------
I must have made a mistake in testing this. Re-reviewing it, it's not possible
for this to work and indeed it fails when I test it again.
When writing to S3, the 'winner' will be the
last-started-upload-that-completed-successfully. In the case of MPU this start
is at init time, not complete time. So when the transient file comes in and is
deleted then the MPU becomes moot.
In the case of a versioned bucket, then the transient file doesn't unroll the
latest version to the MPU because all the 'latest' knowledge is based on init
time, not complete time.
I will revert the commit.
> add extra S3A MPU test to see what happens if a file is created during the MPU
> ------------------------------------------------------------------------------
>
> Key: HADOOP-16193
> URL: https://issues.apache.org/jira/browse/HADOOP-16193
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Fix For: 3.1.3
>
>
> Proposed extra test for the S3A MPU: if you create and then delete a file
> while an MPU is in progress, when you finally complete the MPU the new data
> is present.
> This verifies that the other FS operations don't somehow cancel the
> in-progress upload, and that eventual consistency brings the latest value out.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
