alamb commented on PR #12399: URL: https://github.com/apache/datafusion/pull/12399#issuecomment-2343778532
> I think this may negate some of the performance improvements gained by the initial PR that introduced these bugs, but I think once this is merged, we can refactor the pushdown_limit_helper function slightly to keep the correct behavior while pulling in the improvements again @itsjunetime I wonder if you can elaborate on this or file a ticket. In order to avoid the final `GlobalLimitExec` I believe we would have to add `offset` support into `SortPreservingMerge` and `TopK` -- which we could do, but I think the benefit might be relatively low The implementation of Limit is pretty straightforward: https://github.com/apache/datafusion/blob/f24f2cbeff92937f59c5b22f27d23ebf347c3013/datafusion/physical-plan/src/limit.rs#L342-L404 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
