[
https://issues.apache.org/jira/browse/HDFS-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16662511#comment-16662511
]
Steve Loughran commented on HDFS-13713:
---------------------------------------
bq. "If you complete multiple objects to the same destination at the same time,
one of the files will be in the destination. The file that is available is
dependent on the target storage system." As opposed to "there will be an error"
or, worse, "the file will be corrupted".
Implementations could have multiple options here, and we shouldn't use the
current AWS model to dictate for all
How about we state that and list the expected possibilities
1. reject the second and subsequent attempts to initiate an MPU while an upload
is in progress.
2. fail in completion in the 2nd and later attempts
3. allow both to succeed with the final "winner" being determined by the
implementation on a case-by-case basis.
Furthermore, an FS instance does not have to be consistent over time: the
outcome may vary over time/different paths in the store.
The filesystem contract test XML files will have to add an option declaring
which outcome, and the test case use that to verify that the expected outcome
took place.
> Add specification of Multipart Upload API to FS specification, with contract
> tests
> ----------------------------------------------------------------------------------
>
> Key: HDFS-13713
> URL: https://issues.apache.org/jira/browse/HDFS-13713
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, test
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Ewan Higgs
> Priority: Blocker
> Attachments: HADOOP-13713-004.patch, HADOOP-13713-004.patch,
> HADOOP-13713-005.patch, HADOOP-13713-006.patch, HADOOP-13713-007.patch,
> HDFS-13713.001.patch, HDFS-13713.002.patch, HDFS-13713.003.patch,
> HDFS-13713.008.patch, multipartuploader.md
>
>
> There's nothing in the FS spec covering the new API. Add it in a new .md file
> * add FS model with the notion of a function mapping (uploadID -> Upload),
> the operations (list, commit, abort). The [TLA+
> mode|https://issues.apache.org/jira/secure/attachment/12865161/objectstore.pdf]l
> of HADOOP-13786 shows how to do this.
> * Contract tests of not just the successful path, but all the invalid ones.
> * implementations of the contract tests of all FSs which support the new API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
