[
https://issues.apache.org/jira/browse/HIVE-25801?focusedWorklogId=696591&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-696591
]
ASF GitHub Bot logged work on HIVE-25801:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Dec/21 12:43
Start Date: 15/Dec/21 12:43
Worklog Time Spent: 10m
Work Description: veghlaci05 opened a new pull request #2879:
URL: https://github.com/apache/hive/pull/2879
### What changes were proposed in this pull request?
This PR fixes the issue that the statistics gathering for Query based
compactions are not honouring the custom queue settings. It also deprecates the
hive.mr.compactor.gather.stats setting which is honoured only by the MR based
compactor, and introduces the hive.compactor.gather.stats setting, which is
honoured by both MR and Query based compactors. The values of the two settings
are combined with logical OR in case of MR based compaction. The stats
gathering also changed since there is no need for column based statistics
gathering in case of compaction, the basic statistics should be enough.
### Why are the changes needed?
hive.compactor.job.queue config limits resources available for compaction,
so users can limit the effects of compaction on the cluster. However this
setting does not affect stats collection which uses Driver.
HIVE-25595 is addressing the above issue for MR-based compaction. We need to
incorporate the same thing for the Query-based compaction.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
It was tested using the already existing automated tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 696591)
Remaining Estimate: 0h
Time Spent: 10m
> Custom queue settings is not honoured by Query based compaction StatsUpdater
> ----------------------------------------------------------------------------
>
> Key: HIVE-25801
> URL: https://issues.apache.org/jira/browse/HIVE-25801
> Project: Hive
> Issue Type: Bug
> Reporter: László Végh
> Assignee: László Végh
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{hive.compactor.job.queue}} config limits resources available for
> compaction, so users can limit the effects of compaction on the cluster.
> However this settings does not affect stats collection which uses Driver.
> HIVE-25595 is addressing the above issue for MR-based compaction. We need to
> incorporate the same thing for the Query-based compaction.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
