yashmayya commented on issue #18685: URL: https://github.com/apache/pinot/issues/18685#issuecomment-4626455088
The `numDocsScanned` would've been much higher if it were indeed doing a full scan on all segments. [SelectionQuerySegmentPruner](https://github.com/apache/pinot/blob/fe761d249a272574e9de77575e44102ef4bb9c1f/pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SelectionQuerySegmentPruner.java#L46) correctly does not prune when a filter is present, but [MinMaxValueBasedSelectionOrderByCombineOperator](https://github.com/apache/pinot/blob/fe761d249a272574e9de77575e44102ef4bb9c1f/pinot-core/src/main/java/org/apache/pinot/core/operator/combine/MinMaxValueBasedSelectionOrderByCombineOperator.java#L57) does still function as expected - in the case above, I think the issue is more that the boundary value only exists after one segment fully completes - a thread publishes its boundary only after `operator.nextBlock()` returns >= K rows. But all ~N combine worker threads claim their first segment immediately and in parallel. So ~`numWorkerThreads` segments are always fully executed b efore any skip can happen. I'm still surprised about the time taken though, is this because of the DESC ordering limitation? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
