[
https://issues.apache.org/jira/browse/HIVE-26770?focusedWorklogId=828740&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-828740
]
ASF GitHub Bot logged work on HIVE-26770:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Nov/22 14:44
Start Date: 24/Nov/22 14:44
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3803:
URL: https://github.com/apache/hive/pull/3803#discussion_r1031563926
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java:
##########
@@ -219,4 +226,17 @@ protected LockRequest createLockRequest(CompactionInfo ci,
long txnId, LockType
!conf.getBoolVar(HiveConf.ConfVars.TXN_WRITE_X_LOCK));
return requestBuilder.build();
}
+
+ protected void doPostLoopActions(long elapsedTime, CompactorThreadType type)
throws InterruptedException {
+ if (elapsedTime < checkInterval && !stop.get()) {
+ Thread.sleep(checkInterval - elapsedTime);
+ }
+
+ if(elapsedTime < MAX_WARN_LOG_TIME){
+ LOG.debug(type.name() + " loop took " + elapsedTime/1000 + " seconds to
finish.");
+ } else {
+ LOG.warn("Possible "+ type.name() +" slowdown, loop took "+
elapsedTime/1000 + " seconds to finish.");
Review Comment:
nit: spaces around + operator
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java:
##########
@@ -61,6 +61,13 @@ public abstract class CompactorThread extends Thread
implements Configurable {
protected String hostName;
protected String runtimeVersion;
+ //Time threshold for compactor thread log
+ //In milliseconds:
+ protected Integer MAX_WARN_LOG_TIME = 1200000; //20 min
+
+ protected long checkInterval;
+
+ public enum CompactorThreadType {INITIATOR, WORKER, CLEANER}
Review Comment:
What about introducing an abstract CompactorThreadType getThreadType()
method, implement in Cleaner, Worker, Initiator, and use it in
doPostLoopActions()? This way CompactorThreadType type parameter would no
longer necessary. However, I don't have a strong preference over this, the
current approach is also fine.
Issue Time Tracking
-------------------
Worklog Id: (was: 828740)
Time Spent: 1h (was: 50m)
> Make "end of loop" compaction logs appear more selectively
> ----------------------------------------------------------
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
> Reporter: Akshat Mathur
> Assignee: Akshat Mathur
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
