[
https://issues.apache.org/jira/browse/HIVE-1852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12973329#action_12973329
]
Ning Zhang commented on HIVE-1852:
----------------------------------
Yes, it's because the oldPath could be from a different file system. In this
patch, a new file system is got from oldPath and it is used to delete oldPath.
It should be equivalent to FsShell. Hairong mentioned that FsShell is
relatively slow and more expensive that calling FileSystem.delete.
Yongqiang, is there a test case for HIVE-1707?
> Reduce unnecessary DFSClient.rename() calls
> -------------------------------------------
>
> Key: HIVE-1852
> URL: https://issues.apache.org/jira/browse/HIVE-1852
> Project: Hive
> Issue Type: Improvement
> Reporter: Ning Zhang
> Assignee: Ning Zhang
> Attachments: HIVE-1852.2.patch, HIVE-1852.3.patch, HIVE-1852.patch
>
>
> In Hive client side (MoveTask etc), DFSCleint.rename() is called for every
> file inside a directory. It is very expensive for a large directory in a busy
> DFS namenode. We should replace it with a single rename() call on the whole
> directory.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
