[
https://issues.apache.org/jira/browse/HIVE-26770?focusedWorklogId=828649&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-828649
]
ASF GitHub Bot logged work on HIVE-26770:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 24/Nov/22 09:36
Start Date: 24/Nov/22 09:36
Worklog Time Spent: 10m
Work Description: veghlaci05 commented on code in PR #3803:
URL: https://github.com/apache/hive/pull/3803#discussion_r1031246443
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorThread.java:
##########
@@ -61,6 +61,9 @@ public abstract class CompactorThread extends Thread
implements Configurable {
protected String hostName;
protected String runtimeVersion;
+ //Time threshold for compactor thread log
+ //In milli sec
Review Comment:
nit: milliseconds
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Cleaner.java:
##########
@@ -181,10 +178,16 @@ public void run() {
}
// Now, go back to bed until it's time to do this again
long elapsedTime = System.currentTimeMillis() - startedAt;
- if (elapsedTime < cleanerCheckInterval && !stop.get()) {
- Thread.sleep(cleanerCheckInterval - elapsedTime);
+ long threadSleepTime = getThreadSleepTime(elapsedTime, stop,
CompactorUtil.CompactorThreadType.CLEANER);
+
+ if(threadSleepTime != -1 ) Thread.sleep(threadSleepTime);
+
+
+ if(elapsedTime < MAX_WARN_LOG_TIME){
+ LOG.debug("Cleaner loop took " + elapsedTime + " milli sec to
finish.");
Review Comment:
Please log the value in seconds, also in all other occurrences.
##########
ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorUtil.java:
##########
@@ -34,6 +34,8 @@
public class CompactorUtil {
public static final String COMPACTOR = "compactor";
+
+ public enum CompactorThreadType {INITIATOR, WORKER, CLEANER}
Review Comment:
Move this to CompactorThread
Issue Time Tracking
-------------------
Worklog Id: (was: 828649)
Time Spent: 0.5h (was: 20m)
> Make "end of loop" compaction logs appear more selectively
> ----------------------------------------------------------
>
> Key: HIVE-26770
> URL: https://issues.apache.org/jira/browse/HIVE-26770
> Project: Hive
> Issue Type: Improvement
> Reporter: Akshat Mathur
> Assignee: Akshat Mathur
> Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently Initiator, Worker, and Cleaner threads log something like "finished
> one loop" on INFO level.
> This is useful to figure out if one of these threads is taking too long to
> finish a loop, but expensive in general.
>
> Suggested Time: 20mins
> Logging this should be changed in the following way
> # If loop finished within a predefined amount of time, level should be DEBUG
> and message should look like: *Initiator loop took \{ellapsedTime} seconds to
> finish.*
> # If loop ran longer than this predefined amount, level should be WARN and
> message should look like: *Possible Initiator slowdown, loop took
> \{ellapsedTime} seconds to finish.*
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
