palaska commented on PR #1059: URL: https://github.com/apache/datafusion-ballista/pull/1059#issuecomment-2374878019
> > I've just looked at this, I think the never completing queries just try to return too many rows and they take too long. That is because the logical optimizer pushes down Limit to Sort but on logical plan deserialization, we seem to be ignoring it. This might be a quick fix: [apache/datafusion#12626](https://github.com/apache/datafusion/pull/12626). > > It works fine with the main branch though. Was this bug not present in earlier DataFusion versions? I think this bug was always there but Datafusion v42's logical optimizer removes Limit logical node after pushing down it's `fetch` while v41 doesn't remove it. Even though we haven't been deserializing `Sort` properly, having the wrapping `Limit` was saving us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
