[
https://issues.apache.org/jira/browse/HIVE-22819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22819:
------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~Marton Bod], and [~ste...@apache.org] for the review!
> Refactor Hive::listFilesCreatedByQuery to make it faster for object stores
> --------------------------------------------------------------------------
>
> Key: HIVE-22819
> URL: https://issues.apache.org/jira/browse/HIVE-22819
> Project: Hive
> Issue Type: Improvement
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22819.1.patch, HIVE-22819.2.patch,
> HIVE-22819.3.patch, HIVE-22819.4.patch, HIVE-22819.5.patch,
> HIVE-22819.6.patch, HIVE-22819.7.patch, HIVE-22819.8.patch
>
>
> {color:#0000ff}Hive::listFilesCreatedByQuery{color} does an exists(), an
> isDir() and then a listing call. This can be expensive in object stores. We
> should instead directly list the files in the directory (we'd have to handle
> an exception if the directory does not exists, but issuing a single call to
> the object store would most likely still end up being more performant).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
