[
https://issues.apache.org/jira/browse/HADOOP-13974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16283715#comment-16283715
]
Steve Loughran commented on HADOOP-13974:
-----------------------------------------
Looks pretty good
General
* check the import ordering w.r.t the style rules
* I can see that retry stuff is useful, but also it's going to have to be
something we keep an eye on maintenance-wise.
* need to move {{S3ATestUtils.listMultipartUploads()}} to this
* there's some shiny new Java 8 code in S3AUtils, like applyLocatedFiles(),
which work on RemoteIterator<LocatedFileStatus>.
These could be expanded to take any RemoteIterator/subclass thereof, maybe,
which would actually be something to have in hadoop common for broader use.
{{MultipartTestUtils.assertNoUploadsAt()}} would be an example use
* MultipartUtils L210: can simplify to {{return batchIterator.hasNext();}}.
S3AFileSystem L772. Good point. They used to be bonded to the destination path,
but we've moved off that: we could just create a single instance here. Want to
change it?
ITestS3GuardToolLocal:
* good Q. about using eventually() in listings. I don't know what happens
there, but I'd hope that there's more list consistency here. (Fewer entries and
you certainly need to be able to map on subsequent posts to the outstanding MPU
* If uploadCommandAssertCount really is at risk of failing, the output should
be logged or included in the fail(). Simplest to log
> S3a CLI to support list/purge of pending multipart commits
> ----------------------------------------------------------
>
> Key: HADOOP-13974
> URL: https://issues.apache.org/jira/browse/HADOOP-13974
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Steve Loughran
> Assignee: Aaron Fabbri
> Attachments: HADOOP-13974.001.patch, HADOOP-13974.002.patch,
> HADOOP-13974.003.patch, HADOOP-13974.004.patch
>
>
> The S3A CLI will need to be able to list and delete pending multipart
> commits.
> We can do the cleanup already via fs.s3a properties. The CLI will let scripts
> stat for outstanding data (have a different exit code) and permit batch jobs
> to explicitly trigger cleanups.
> This will become critical with the multipart committer, as there's a
> significantly higher likelihood of commits remaining outstanding.
> We may also want to be able to enumerate/cancel all pending commits in the FS
> tree
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
