stuhood commented on issue #27109: URL: https://github.com/apache/datafusion/issues/27109#issuecomment-6643552894
> For example, when an incremental top-`K` probe scan feeds a chain of `HashJoinExec`s under `ORDER BY ... LIMIT K`, `LimitPushdown` can set `fetch = Some(K)` only on the outermost join; inner joins have `fetch = None`, so their `LimitedBatchCoalescer` buffers matches and repeatedly pulls from the probe scan until `8192` matches accumulate or the scan is exhausted. This is a rewrite that we have implemented downstream: is it something that is worth proposing upstream as well? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
