[
https://issues.apache.org/jira/browse/HIVE-24669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17268605#comment-17268605
]
Peter Varga commented on HIVE-24669:
------------------------------------
Local performance measurement results:
Doing insert select from unpartition table on S3 to a partitioned table on S3,
creating 100 new dynamic partition.
Avg loadPartitionInternal before patch: 2433 ms
Avg loadPartitionInternal after patch: 1279 ms
> Improve Filesystem usage in Hive::loadPartitionInternal
> -------------------------------------------------------
>
> Key: HIVE-24669
> URL: https://issues.apache.org/jira/browse/HIVE-24669
> Project: Hive
> Issue Type: Sub-task
> Reporter: Peter Varga
> Assignee: Peter Varga
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> * Use native recursive listing instead doing it on the Hive side
> * Reuse the file list determined for writeNotificationlogs in quickstat
> generation
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
