[
https://issues.apache.org/jira/browse/HDFS-13713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16529944#comment-16529944
]
Steve Loughran commented on HDFS-13713:
---------------------------------------
Tests
* abort twice is OK
* commit 2x is bad
* abort after commit?
* what happens if an invalid path handle gets in? That is: malicious code
trying a serialization attack? Fpr
{{org.apache.hadoop.fs.s3a.commit.files.PersistentCommitData}} there's a lot of
validation on there to defend against malicious payloads
* can commit after initiate but before uploading data (true? If not, add that
to spec & expect this operation to fail)
* that len(committed upload) = listFiles == len in listStatus == len
getFileStatus == length of data
* that you get back all the bytes you uploaded, in order, even for large files
* That whatever mechanism exists to determine the min upload size, works
* And if there is a limit on the #of parts of an upload, that is somehow
available too.
* that you can serialize a path handle, deserialize it. and commit from that
deser'd value. (ie the data round trips properly)
This is essentially a move of {{AbstractSystemMultipartUploaderTest}} to being
a contract test, with more cases underneath
> Add specification of new API to FS specification, with contract tests
> ---------------------------------------------------------------------
>
> Key: HDFS-13713
> URL: https://issues.apache.org/jira/browse/HDFS-13713
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: fs, test
> Affects Versions: 3.2.0
> Reporter: Steve Loughran
> Assignee: Ewan Higgs
> Priority: Blocker
>
> There's nothing in the FS spec covering the new API. Add it in a new .md file
> * add FS model with the notion of a function mapping (uploadID -> Upload),
> the operations (list, commit, abort). The [TLA+
> mode|https://issues.apache.org/jira/secure/attachment/12865161/objectstore.pdf]l
> of HADOOP-13786 shows how to do this.
> * Contract tests of not just the successful path, but all the invalid ones.
> * implementations of the contract tests of all FSs which support the new API.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
