npawar commented on code in PR #8949:
URL: https://github.com/apache/pinot/pull/8949#discussion_r903159292
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/PinotHelixTaskResourceManager.java:
##########
@@ -531,6 +531,48 @@ public synchronized Map<String, TaskDebugInfo>
getTasksDebugInfo(String taskType
return taskDebugInfos;
}
+ /**
+ * Given a taskType and a tableNameWithType, helper method to debug all the
HelixJobs for
+ * the taskType and tableNameWithType. For each of the HelixJobs, collects
status of
+ * the (sub)tasks in the taskbatch.
+ *
+ * @param taskType Pinot taskType / Helix JobQueue
+ * @param tableNameWithType Table name with type to filter on
+ * @param verbosity By default, does not show details for completed tasks.
+ * If verbosity > 0, shows details for all tasks.
+ * @return Map of Pinot Task Name to TaskDebugInfo. TaskDebugInfo contains
details for subtasks.
+ */
+ public synchronized Map<String, TaskDebugInfo> getTasksDebugInfoByTable(
+ String taskType, String tableNameWithType, int verbosity) {
+ Map<String, TaskDebugInfo> taskDebugInfos = new TreeMap<>();
+ WorkflowContext workflowContext =
_taskDriver.getWorkflowContext(getHelixJobQueueName(taskType));
+ if (workflowContext == null) {
+ return taskDebugInfos;
+ }
+
+ Map<String, TaskState> helixJobStates = workflowContext.getJobStates();
+ for (String helixJobName : helixJobStates.keySet()) {
+ String pinotTaskName = getPinotTaskName(helixJobName);
+
+ // Iterate through all task configs associated with this task name
+ for (PinotTaskConfig taskConfig : getTaskConfigs(pinotTaskName)) {
+ Map<String, String> pinotConfigs = taskConfig.getConfigs();
+
+ // Filter task configs that matches this table name
+ if (pinotConfigs != null) {
+ String tableNameConfig = pinotConfigs.get(TABLE_NAME);
+ if (tableNameConfig != null &&
tableNameConfig.equals(tableNameWithType)) {
+ // Found a match. Add task debug info to the result
+ // TODO: we may want to filter out debug info that does not belong
to the given table.
Review Comment:
didn't follow this TODO. why would we get info from `getTaskDebugInfo`
that's not for this table?
##########
pinot-controller/src/main/java/org/apache/pinot/controller/api/resources/PinotTaskRestletResource.java:
##########
@@ -215,6 +215,18 @@ public Map<String,
PinotHelixTaskResourceManager.TaskDebugInfo> getTasksDebugInf
return _pinotHelixTaskResourceManager.getTasksDebugInfo(taskType,
verbosity);
}
+ @GET
+ @Path("/tasks/{taskType}/{tableNameWithType}/debug")
+ @ApiOperation("Fetch information for the given task type and table")
+ public Map<String, PinotHelixTaskResourceManager.TaskDebugInfo>
getTasksDebugInfo(
+ @ApiParam(value = "Task type", required = true) @PathParam("taskType")
String taskType,
+ @ApiParam(value = "Table name with type", required = true)
@PathParam("tableNameWithType")
+ String tableNameWithType,
+ @ApiParam(value = "verbosity (By default, prints for running and error
tasks. Value of >0 prints for all tasks)")
+ @DefaultValue("0") @QueryParam("verbosity") int verbosity) {
+ return _pinotHelixTaskResourceManager.getTasksDebugInfoByTable(taskType,
tableNameWithType, verbosity);
Review Comment:
I tried this out with RealtimeQuickstartWithMinion, and saw it return 3
COMPLETED tasks, for both 0 and 1 verbosity. Only difference was level of
detail. Is this `By default, prints for running and error tasks. Value of >0
prints for all tasks)` incorrect or does the impl need some change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
