[
https://issues.apache.org/jira/browse/HADOOP-19254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17875468#comment-17875468
]
Steve Loughran commented on HADOOP-19254:
-----------------------------------------
this helps for CLI testability.
that file thing...I was thinking about that. the pure unix way would just be to
use stdin ignoring newlines and # comment lines. so you can
{code}
hadoop fs -bulkdelete s3a://bucket/path < file list
rg s3a://bucket logs | hadoop fs -bulkdelete s3a://bucket/path
{code}
looks like it is fairly straightforward:
https://www.baeldung.com/java-console-input-output
and if more than one path was passed on the cli we'd just delete them in order
{code}
hadoop fs -bulkdelete s3a://bucket/path s3a://bucket/path/file1
s3a://bucket/path/file2 ...
{code}
to keep things simple, no need for options, docs. or testing different configs
* log to std out files which were deleted after their successful bulkDelete()
call
* on an error: log at error files which were rejected then fail
* always pretty print iostatistics at the end
testing could be added as extra cases in the bulk delete contract test as
that'd help validate behaviour with page size > 1
> Implement bulk delete command as hadoop fs command operation
> -------------------------------------------------------------
>
> Key: HADOOP-19254
> URL: https://issues.apache.org/jira/browse/HADOOP-19254
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Affects Versions: 3.4.1
> Reporter: Mukund Thakur
> Assignee: Mukund Thakur
> Priority: Major
> Fix For: 3.4.2
>
>
> hadoop fs -bulkdelete <base-url> <file>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
