cryptoe commented on code in PR #15180:
URL: https://github.com/apache/druid/pull/15180#discussion_r1366459439
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/indexing/MSQWorkerTaskLauncher.java:
##########
@@ -348,6 +351,68 @@ public boolean isTaskLatest(String taskId)
}
}
+ public static class WorkerStats
+ {
+ String workerId;
+ TaskState state;
+ long duration;
+
+ /**
+ * For JSON deserialization only
+ */
+ public WorkerStats()
+ {
+ }
+
+ public WorkerStats(String workerId, TaskState state, long duration)
+ {
+ this.workerId = workerId;
+ this.state = state;
+ this.duration = duration;
+ }
+
+ @JsonProperty()
+ public String getWorkerId()
+ {
+ return workerId;
+ }
+
+ @JsonProperty()
+ public TaskState getState()
+ {
+ return state;
+ }
+
+ @JsonProperty("durationMs")
+ public long getDuration()
+ {
+ return duration;
+ }
+ }
+
+ public Map<Integer, List<WorkerStats>> getWorkerStats()
+ {
+ final Map<Integer, List<WorkerStats>> workerStats = new TreeMap<>();
+
+ for (Map.Entry<String, TaskTracker> taskEntry : taskTrackers.entrySet()) {
+
+ TaskTracker taskTracker = taskEntry.getValue();
+
+ long duration = (taskTracker.status.getDuration() == -1
Review Comment:
MSQ tracks the start time as the time it requested to launch the job. I
currently do not know if duration counter in the overlord is started as soon as
the overlord gets the request or when ever that task goes into running state.
To test it what you could do is
1. Make changes to the MSQWorkerTaskLauncher to log the report every time
liveReports is called.
2. Start a cluster with lets say 4 task slots.
3. Use 2 slots for other ingestions.
4. Schedule an MSQ job with 4 slots. (The MSQ job will wait and the worker
task launcher will set the start time for each task ).
5. Kill the job running on the other 2 slots.
Check if the taskDuration in the report is going backward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
