[
https://issues.apache.org/jira/browse/HIVE-26770?focusedWorklogId=828650&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-828650
]
ASF GitHub Bot logged work on HIVE-26770:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Nov/22 09:37
Start Date: 24/Nov/22 09:37
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3803:
URL: https://github.com/apache/hive/pull/3803#discussion_r1031290340
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/MetaStoreCompactorThread.java:
##########
@@ -133,4 +134,28 @@ protected static long updateCycleDurationMetric(String
metric, long startedAt) {
}
return 0;
}
+
+ // This method was created to reduce duplication between
+ // Compactor classes Cleaner and Initiator
+ protected long getThreadSleepTime(long elapsedTime, AtomicBoolean stop,
CompactorUtil.CompactorThreadType type) {
+ long checkInterval;
+ switch (type) {
+ case INITIATOR:
+ checkInterval =
conf.getTimeVar(HiveConf.ConfVars.HIVE_COMPACTOR_CHECK_INTERVAL,
TimeUnit.MILLISECONDS);
+ break;
+ case CLEANER:
+ checkInterval = conf.getTimeVar(
+ HiveConf.ConfVars.HIVE_COMPACTOR_CLEANER_RUN_INTERVAL,
TimeUnit.MILLISECONDS);
+ break;
+ default:
+ throw new IllegalArgumentException("Unknown type: " + type);
+ }
+
+ if (elapsedTime < checkInterval && !stop.get()) {
+ return checkInterval - elapsedTime;
+ } else {
+ //In case where Thread.sleep is not required
+ return -1;
+ }
+ }
Review Comment:
1. Move this method to `CompactorThread`
2. Field `stop` is acccessible in `CompactorThread`, so you don't need to
pass it.
3. Restore `checkinterval set in `init()` methods, so no need to set it here
again in every iteration
4. Move `checkinterval` to `CompactorThread` and for `Worker` set it to 0 in
`Worker.init()`
5. `Thread.sleep()` should happen here
6. Logging should happen here like `LOG.debug(type + " loop took "...)`
7. The method name should be sth like `doPostLoopActions`and it should be
void
8. Worker should call this method as well
Issue Time Tracking
-------------------
Worklog Id: (was: 828650)
Time Spent: 40m (was: 0.5h)
> Make "end of loop" compaction logs appear more selectively
> ----------------------------------------------------------
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
> Reporter: Akshat Mathur
> Assignee: Akshat Mathur
> Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
