ming535 commented on PR #2629: URL: https://github.com/apache/arrow-datafusion/pull/2629#issuecomment-1139241758
I think a better way to solve this issue https://github.com/apache/arrow-datafusion/issues/2624 might be to add another LogicalPlan, for example `LimitWithOffset`. The semantic of these three LogicalPlan is different: 1. `Limit`: only set an upper bounds on the stream's data 2. `Offset` only set a lower bounds on the stream's data 3. `LimitWithOffset` set a lower bounds AND upper bounds on the stream's data As of `limit_push_down`, it wont' touch `Offset`. It will push down `limit` of `Limit`. It will adjust `LimitWithOffset`'s `limit` based on its offset and push the limit down to TableScan; if the push down happens, we replace it with `Offset`. As of physical plan, we need to implement 3 physical plan accordingly. (Or may be we should just ) I am not quite sure whether this is a good way to solve this issue, any thoughts? @alamb @andygrove @Ted-Jiang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
