[
https://issues.apache.org/jira/browse/HIVE-26471?focusedWorklogId=815486&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-815486
]
ASF GitHub Bot logged work on HIVE-26471:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 11/Oct/22 07:29
Start Date: 11/Oct/22 07:29
Worklog Time Spent: 10m
Work Description: deniskuzZ commented on code in PR #3645:
URL: https://github.com/apache/hive/pull/3645#discussion_r991911089
##########
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/metrics/CompactionMetricData.java:
##########
@@ -67,34 +70,53 @@ static CompactionMetricData
of(List<ShowCompactResponseElement> compacts) {
private void init() {
final Map<String, ShowCompactResponseElement> lastElements = new
HashMap<>();
- poolCount = new HashMap<>();
+ initiatedPoolCount = new HashMap<>();
+ workingPoolCount = new HashMap<>();
+ oldestEnqueueTimePerPool = new HashMap<>();
+ oldestWorkingTimePerPool = new HashMap<>();
oldestEnqueueTime = OLDEST_TIME_NO_VALUE;
oldestWorkingTime = OLDEST_TIME_NO_VALUE;
oldestCleaningTime = OLDEST_TIME_NO_VALUE;
+
+ long currentTime = System.currentTimeMillis();
for (ShowCompactResponseElement element : compacts) {
final String key = element.getDbname() + "/" + element.getTablename() +
(element.getPartitionname() != null ? "/" +
element.getPartitionname() : "");
// If new key, add the element, if there is an existing one, change to
the element if the element.id is greater than old.id
lastElements.compute(key, (k, old) -> (old == null) ? element :
(element.getId() > old.getId() ? element : old));
- // find the oldest elements with initiated and working states
String state = element.getState();
- if (TxnStore.INITIATED_RESPONSE.equals(state) && (oldestEnqueueTime >
element.getEnqueueTime())) {
- oldestEnqueueTime = element.getEnqueueTime();
- poolCount.compute(element.getPoolName(), (k, old) -> (old == null) ? 1
: old + 1);
+ if (TxnStore.INITIATED_RESPONSE.equals(state)) {
+ final int enqueueSeconds = (int) ((currentTime -
element.getEnqueueTime()) / 1000);
+ oldestWorkingTimePerPool.compute(element.getPoolName(), (k, old) ->
(old == null) ? enqueueSeconds : Math.max(enqueueSeconds, old));
+
+ initiatedPoolCount.compute(element.getPoolName(), (k, old) -> (old ==
null) ? 1 : old + 1);
+ // find the oldest element
+ if (oldestEnqueueTime > element.getEnqueueTime()) {
+ oldestEnqueueTime = element.getEnqueueTime();
+ }
}
if (element.isSetStart()) {
- if (TxnStore.WORKING_RESPONSE.equals(state) && (oldestWorkingTime >
element.getStart())) {
- oldestWorkingTime = element.getStart();
+ if (TxnStore.WORKING_RESPONSE.equals(state)) {
+ final int startSeconds = (int) ((currentTime - element.getStart()) /
1000);
+ oldestEnqueueTimePerPool.compute(element.getPoolName(), (k, old) ->
(old == null) ? startSeconds : Math.max(startSeconds, old));
+
+ workingPoolCount.compute(element.getPoolName(), (k, old) -> (old ==
null) ? 1 : old + 1);
+ // find the oldest element
+ if (oldestWorkingTime > element.getStart()) {
+ oldestWorkingTime = element.getStart();
+ }
}
}
if (element.isSetCleanerStart()) {
- if (TxnStore.CLEANING_RESPONSE.equals(state) && (oldestCleaningTime >
element.getCleanerStart())) {
- oldestCleaningTime = element.getCleanerStart();
+ if (TxnStore.CLEANING_RESPONSE.equals(state)) {
Review Comment:
why is this change?
Issue Time Tracking
-------------------
Worklog Id: (was: 815486)
Time Spent: 3.5h (was: 3h 20m)
> New metric for Compaction pooling
> ---------------------------------
>
> Key: HIVE-26471
> URL: https://issues.apache.org/jira/browse/HIVE-26471
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: László Végh
> Assignee: László Végh
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-2
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> To be able to properly supervise the pool based compaction, a new metric is
> required:
> Number of 'Initiated' compaction requests per compaction pool.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
