[ https://issues.apache.org/jira/browse/HIVE-22081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16908407#comment-16908407 ]
Rajkumar Singh commented on HIVE-22081: --------------------------------------- Thanks [~pvary], I have uploaded the fresh patch with the suggested changes for a clean run. > Hivemetastore Performance: Compaction Initiator Thread overwhelmed if there > are too many Table/partitions are eligible for compaction > -------------------------------------------------------------------------------------------------------------------------------------- > > Key: HIVE-22081 > URL: https://issues.apache.org/jira/browse/HIVE-22081 > Project: Hive > Issue Type: Improvement > Components: Transactions > Affects Versions: 3.1.1 > Reporter: Rajkumar Singh > Assignee: Rajkumar Singh > Priority: Major > Attachments: HIVE-21917.01.patch, HIVE-21917.02.patch, > HIVE-21917.03.patch, HIVE-22081.patch > > > if Automatic Compaction is turned on, Initiator thread check for potential > table/partitions which are eligible for compactions and run some checks in > for loop before requesting compaction for eligibles. Though initiator thread > is configured to run at interval 5 min default, in case of many objects it > keeps on running as these checks are IO intensive and hog cpu. > In the proposed changes, I am planning to do > 1. passing less object to for loop by filtering out the objects based on the > condition which we are checking within the loop. > 2. Doing Async call using future to determine compaction type(this is where > we do FileSystem calls) -- This message was sent by Atlassian JIRA (v7.6.14#76016)