Akshat-Jain commented on code in PR #17038:
URL: https://github.com/apache/druid/pull/17038#discussion_r1760986762
##########
extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/querykit/WindowOperatorQueryFrameProcessor.java:
##########
@@ -213,7 +213,8 @@ Current approach with R&C and operators materialize a
single R&C for processing.
Most of the window operations like SUM(), RANK(), RANGE() etc. can be
made with 2 passes of the data. We might think to reimplement them in the MSQ
way so that we do not have to materialize so much data.
*/
- if (partitionColumnNames.isEmpty()) {
+ // todo: need to cleanup unused code and comments
+ if (true) {
Review Comment:
Based on offline discussion, maxRowsMaterialized check should be moved to
the glueing operator.
But on further thoughts, the same can't be done in Windowing.java as we
don't have access to `MultiStageQueryContext.MAX_ROWS_MATERIALIZED_IN_WINDOW`
and `Limits.MAX_ROWS_MATERIALIZED_IN_WINDOW`
Another potential option which was coined was changing
`OperatorFactory#wrap` interface method signature from `Operator wrap(Operator
op)` to `Operator wrap(Operator op, Map<String, Object> context)`, and
Windowing can just add a PartitioningFactory whose `wrap` method makes the
decision between Naive and Glueing partitioning operators. Will have to look
into this, if we have all the data to make this approach work or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
