[
https://issues.apache.org/jira/browse/HADOOP-13611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490493#comment-15490493
]
Steve Loughran commented on HADOOP-13611:
-----------------------------------------
This is easy to do, but low priority, as it is generally used in testing rather
than production.
if adding to FileSystem, adding some logging at the same time could be useful,
as it would help show why things were taking time.
Test-wise, a FilterFileSystem subclass could make the method public;
experiments could made to verify it works even after the file to delete has
been removed. Ideally, this could be done in a new FS contract test, so that
the correct operation of all filesystems are verified.
> FileSystem/s3a processDeleteOnExit to skip the exists() check
> -------------------------------------------------------------
>
> Key: HADOOP-13611
> URL: https://issues.apache.org/jira/browse/HADOOP-13611
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
> Affects Versions: 2.7.3
> Reporter: Steve Loughran
> Priority: Minor
>
> If you look at {{FileSystem.processDeleteOnExit()}}, it does an exists()
> check for each entry, before calling delete().
> That exists() check is superfluous; on s3 it add an extra 1-4 HTTP GETs
> This could be fixed with a subclass in s3a to avoid it, but as the call is
> superfluous in *all* filesystems, it could be removed in {{FileSystem}} and
> so picked up by all object stores.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
