zhuqi-lucas commented on PR #16196: URL: https://github.com/apache/datafusion/pull/16196#issuecomment-2921143644
> Thank you @zhuqi-lucas and @pepijnve I think this PR does actually solve the cancellation issue > > I left some suggestions for how it could be improved, but the basic idea looks great to me and I think we could merge this PR and handle the results as a follow on > > I also tested with https://github.com/pepijnve/datafusion_cancel_test from @pepijnve > > With this PR checked out locally, the test now shows proper cancel behavior for me > > ```diff > diff --git a/Cargo.toml b/Cargo.toml > index 8855aee..f815883 100644 > --- a/Cargo.toml > +++ b/Cargo.toml > @@ -6,7 +6,8 @@ edition = "2024" > [dependencies] > futures = "0.3.31" > arrow = "55.0.0" > -datafusion = "47.0.0" > +# datafusion = "47.0.0" > +datafusion = { path = "/Users/andrewlamb/Software/datafusion/datafusion/core" } > > [dependencies.tokio] > version = "1.45.1" > ``` > > ``` > Running query; will time out after 5 seconds > InfiniteStream::poll_next 10000 times > InfiniteStream::poll_next 20000 times > InfiniteStream::poll_next 30000 times > InfiniteStream::poll_next 40000 times > InfiniteStream::poll_next 50000 times > InfiniteStream::poll_next 60000 times > InfiniteStream::poll_next 70000 times > InfiniteStream::poll_next 80000 times > InfiniteStream::poll_next 90000 times > InfiniteStream::poll_next 100000 times > InfiniteStream::poll_next 110000 times > InfiniteStream::poll_next 120000 times > Timeout reached! > No result (cancelled or empty) > Exiting, stream will be dropped now > ``` > > Without those changes > > ``` > Running query; will time out after 5 seconds > InfiniteStream::poll_next 10000 times > InfiniteStream::poll_next 20000 times > InfiniteStream::poll_next 30000 times > InfiniteStream::poll_next 40000 times > InfiniteStream::poll_next 50000 times > InfiniteStream::poll_next 60000 times > InfiniteStream::poll_next 70000 times > InfiniteStream::poll_next 80000 times > InfiniteStream::poll_next 90000 times > InfiniteStream::poll_next 100000 times > InfiniteStream::poll_next 110000 times > InfiniteStream::poll_next 120000 times > InfiniteStream::poll_next 130000 times > InfiniteStream::poll_next 140000 times > InfiniteStream::poll_next 150000 times > InfiniteStream::poll_next 160000 times > InfiniteStream::poll_next 170000 times > InfiniteStream::poll_next 180000 times > InfiniteStream::poll_next 190000 times > InfiniteStream::poll_next 200000 times > ... > (never cancels) > ``` Thank you @alamb for review and checking! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
