cryptoe commented on code in PR #17581:
URL: https://github.com/apache/druid/pull/17581#discussion_r1919488732
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java:
##########
@@ -329,6 +331,27 @@ public void run(final QueryListener queryListener) throws
Exception
}
// Call onQueryComplete after Closer is fully closed, ensuring no
controller-related processing is ongoing.
queryListener.onQueryComplete(reportPayload);
+
+ long totalProcessedBytes =
reportPayload.getCounters().copyMap().values().stream()
Review Comment:
I am not sure ingest/processed/bytes make sense for select query.
MSQ runs using a DAG of stages.
https://github.com/apache/druid/blob/cedf9bbfe366452760de77cfb5c7042430755b18/extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/kernel/QueryDefinition.java#L48
has a
`private final Map<StageId, StageDefinition> stageDefinitions;`
And each stage definition would have `private final List<InputSpec>
inputSpecs;`
I think the metric makes sense when we check if the input spec is not a
`StageInputSpec` and only then plumb the input bytes to the final summary
metric.
A UT like this can help you debug stuff :
https://github.com/apache/druid/blob/cedf9bbfe366452760de77cfb5c7042430755b18/extensions-core/multi-stage-query/src/test/java/org/apache/druid/msq/exec/MSQInsertTest.java#L404
Attach breakpoint to
https://github.com/apache/druid/blob/cedf9bbfe366452760de77cfb5c7042430755b18/extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/exec/ControllerImpl.java#L372
to see the query definition in action.
Hope it helps.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
