[
https://issues.apache.org/jira/browse/HIVE-24368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17230399#comment-17230399
]
Rajesh Balamohan commented on HIVE-24368:
-----------------------------------------
Appears that this is handled in master codebase via
https://issues.apache.org/jira/browse/HIVE-23791
h1.
> Optimise AcidUtils::getAcidFilesForStats for ACID tables
> --------------------------------------------------------
>
> Key: HIVE-24368
> URL: https://issues.apache.org/jira/browse/HIVE-24368
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Reporter: Rajesh Balamohan
> Priority: Major
>
> After insert, hive gathers statistics for ACID table and that becomes
> expensive over time, due to number of delta folders and scanning .
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L2648]
>
> {noformat}
> public static List<FileStatus> getAcidFilesForStats(
> Table table, Path dir, Configuration jc, FileSystem fs) throws
> IOException {
> ...
> Directory acidInfo = AcidUtils.getAcidState(fs, dir, jc, idList, null,
> false, hdfsDirSnapshots);
> ...
> ..+ other calls
> ...
> }
> {noformat}
>
> Runtime keeps increasing as more deltas are generated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
