[
https://issues.apache.org/jira/browse/HDFS-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16603504#comment-16603504
]
Ewan Higgs commented on HDFS-13713:
-----------------------------------
{quote}duplicate entries: should the s3a one do a check & fail consistently? Or
call out that it's a MUST fail with IllegalArgumentException or IOE. (I'd
prefer a consistent IllegalArgumentException as this check is straightforward
to do client side){quote}
Agreed. I think {{IllegalArgumentException}} fits best here.
{quote}add a marker to stop a file going in there later. {quote} Goodness, I
hope we don't have to do that. A feature here is that no file exists until the
complete method is called!
If you're working on a system where people will trample your destination files
with directories, I would prefer the onus to be on the client to create names
that won't be interfered with (e.g. containing a UUID).
{quote}Or, and its an interesting thought: don't do the checks at init time,
but postpone them until commit. {quote}
Yes, It's a feature here that the destination file doesn't exist until the
complete method is called. So it makes sense that this is when all the checks
happen. The parent directory for a file needs to exist at init time though
because that's where we put the temp directory with the parts.
{quote}All this is straightforward to test, obviously, which is why having
consistent exceptions is nice. (and its why I like these FS specs, they
identify those corner cases we can trivially derive tests from, and, use as the
reference when trying to decide whether a test failure is a bug in the FS vs
the test itself{quote} Absolutely!
> Add specification of Multipart Upload API to FS specification, with contract
> tests
> ----------------------------------------------------------------------------------
>
> Key: HDFS-13713
> URL: https://issues.apache.org/jira/browse/HDFS-13713
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, test
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Ewan Higgs
> Priority: Blocker
> Attachments: HDFS-13713.001.patch, HDFS-13713.002.patch,
> multipartuploader.md
>
>
> There's nothing in the FS spec covering the new API. Add it in a new .md file
> * add FS model with the notion of a function mapping (uploadID -> Upload),
> the operations (list, commit, abort). The [TLA+
> mode|https://issues.apache.org/jira/secure/attachment/12865161/objectstore.pdf]l
> of HADOOP-13786 shows how to do this.
> * Contract tests of not just the successful path, but all the invalid ones.
> * implementations of the contract tests of all FSs which support the new API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
